Best Practices In Technology

𝗡𝗮𝗶𝘃𝗲 𝗥𝗔𝗚 𝘄𝗼𝗿𝗸𝘀 𝗶𝗻 𝗮 𝗱𝗲𝗺𝗼. 𝗜𝘁 𝗳𝗮𝗶𝗹𝘀 𝘁𝗵𝗲 𝗺𝗼𝗺𝗲𝗻𝘁 𝗿𝗲𝗮𝗹 𝘂𝘀𝗲𝗿𝘀 𝘀𝗵𝗼𝘄 𝘂𝗽. Embed → retrieve → generate looks clean in a notebook. Real requirements break it: → Questions whose answer is spread across many documents → Industry terms that embeddings get wrong → Bad chunks the pipeline never catches → Answers that live in how things connect, not in any single chunk → PDFs full of tables and images a text-only index cannot read These 5 architectures are how serious teams stay ahead in the agentic AI era: 𝟬𝟭 𝗛𝘆𝗯𝗿𝗶𝗱 𝗥𝗔𝗚 → Dense vectors find meaning. BM25 finds exact words. → Reciprocal Rank Fusion combines both ranked lists. → A safe baseline for almost every team. 𝟬𝟮 𝗚𝗿𝗮𝗽𝗵𝗥𝗔𝗚 → Pull entities and their relationships into a knowledge graph. → Retrieve subgraphs and community summaries, not chunks. → Best when the answer lives in how things connect. 𝟬𝟯 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 → A planner agent picks the right tool: vector, web, or SQL. → A reasoner agent keeps trying until the answer is solid. → Retrieval becomes a plan, not a single step. 𝟬𝟰 𝗖𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝘃𝗲 𝗥𝗔𝗚 (𝗖𝗥𝗔𝗚) → Grade every retrieval before you trust it. → Correct → answer. Unclear → rewrite the query. Wrong → search the web. → This is what production RAG actually looks like. 𝟬𝟱 𝗠𝘂𝗹𝘁𝗶𝗺𝗼𝗱𝗮𝗹 𝗥𝗔𝗚 → One embedding model (CLIP, ColPali) for text, images, and tables. → One vector index. One multimodal LLM. → No more separate pipelines for PDFs with charts. I built a runnable example for each of the five patterns. GitHub link in the first comment. The best teams in 2026 do not pick one. They combine them — hybrid retrieval inside an agentic loop, with a corrective grader, over a multimodal index. Naive RAG is a starting point, not a finish line. That is why most enterprise GenAI projects stall at the demo. Which of these five becomes the default RAG stack in the next 18 months — and which stays a specialized tool?

The Mobile Telco Model Is Broken....And That Might Not Be a Bad Thing I had a good conversation with a senior Telco CEO yesterday. We both agreed: the mobile telecom model isn’t just under pressure... it’s broken. And not recently. It’s been structurally broken for over 15 years. Revenues? Growing at 2% a year globally, below inflation. ROIC? Still under the cost of capital for most operators. NPS? Telcos hover around 20–30 while tech companies push 60+. Opex? In many markets, it eats up 80% of revenue. And price erosion? We’ve normalized losing 30–40% of GB value per year. And what are we still selling? Buckets of data. Now, when something is broken, you need to go to foundational aspects of our business. First, telcos must become digital-first. Customers don’t compare you to other telcos, they compare you to Uber, Amazon, and Apple. That’s the bar. If it takes 12 clicks to activate a line or change a plan, you’ve already lost. Second, you need deep personalization at scale. Spotify knows your mood. Telcos still offer “Youth Plan 3.0.” Microsegments aren’t enough. You need individual-level context. Every offer, touchpoint, and upsell should feel tailor-made. Third, get out of the legacy trap. The OSS/BSS model is dead. Nobody has three years to rebuild a CRM or billing engine. Those projects fail. The only path is cloud-native, agile execution, rapid deployment, fail-fast loops, and zero patience for waterfall digital transformation roadmaps. Amazon Changes 2.5 million prices every day and I can assure you they don't have a group of consultants running a 3 year transformation project to make it happen. Fourth, adopt customer obsession as a core value, not a slide. Amazon isn’t customer-centric because it says so; it’s because the product, support, and incentives are aligned with the user every step of the way. Telcos need to unlearn product-first thinking. Finally, stop thinking of your network as a commodity. It’s not a GB plan. It’s a programmable platform. One that can offer privacy, identity, location, QoS, and trust. If you can expose that as a service, you stop being a utility and start being essential infrastructure again. The model is broken. But that might be the best thing that’s happened to this industry in years.

How To Handle Sensitive Information in your next AI Project It's crucial to handle sensitive user information with care. Whether it's personal data, financial details, or health information, understanding how to protect and manage it is essential to maintain trust and comply with privacy regulations. Here are 5 best practices to follow: 1. Identify and Classify Sensitive Data Start by identifying the types of sensitive data your application handles, such as personally identifiable information (PII), sensitive personal information (SPI), and confidential data. Understand the specific legal requirements and privacy regulations that apply, such as GDPR or the California Consumer Privacy Act. 2. Minimize Data Exposure Only share the necessary information with AI endpoints. For PII, such as names, addresses, or social security numbers, consider redacting this information before making API calls, especially if the data could be linked to sensitive applications, like healthcare or financial services. 3. Avoid Sharing Highly Sensitive Information Never pass sensitive personal information, such as credit card numbers, passwords, or bank account details, through AI endpoints. Instead, use secure, dedicated channels for handling and processing such data to avoid unintended exposure or misuse. 4. Implement Data Anonymization When dealing with confidential information, like health conditions or legal matters, ensure that the data cannot be traced back to an individual. Anonymize the data before using it with AI services to maintain user privacy and comply with legal standards. 5. Regularly Review and Update Privacy Practices Data privacy is a dynamic field with evolving laws and best practices. To ensure continued compliance and protection of user data, regularly review your data handling processes, stay updated on relevant regulations, and adjust your practices as needed. Remember, safeguarding sensitive information is not just about compliance — it's about earning and keeping the trust of your users.

Maybe you can WRITE SQL, but are you writing ✨GOOD SQL✨? SQL is more than just writing a query without errors… Here’s 10 query optimization tips: 1. Avoid SELECT * and instead list desired columns 2. Use INNER JOINs over LEFT JOINs when applicable 3. Use WHERE and LIMIT to filter rows 4. Filter as much as possible as early as possible (consider the order of execution) 5. Avoid ORDER BY (especially in subqueries and CTEs) 6. Avoid using DISTINCT unless necessary (especially when it’s already implied like in GROUP BY & UNION) 7. Use CTEs when you’ll have to refer to a table/ouput multiple times 8. Avoid using wildcards at the beginning of a string (‘%jess%’ vs. ‘jess%’) 9. Use EXISTS instead of COUNT and IN 10. Avoid complex logic Obviously you can’t ALWAYS avoid these, and they each have their use cases, but these are good things to think about when optimizing your queries.

With 30 years of experience in the technology sector, including in engineering & operations, I’ve developed my own best practices that help organizations build trust with the communities who will use their technology.  In this week’s special TIME Magazine Davos issue, I outlined a framework based on those hard-won lessons to help ensure AI development is responsible, thoughtful, and benefits humanity, including: - Embrace Early Collaboration: Bringing outside voices into the development process early helps to create technology that better reflects the breadth and depth of the human experience. Ensuring you partner with - and listen to - experts & local communities can help mitigate potential risks. - Operationalize Care: The success of AI projects often hinges on how well organizations implement systems that operationalize their commitment to care. For example, at Google DeepMind, we have developed frameworks that embed ethical considerations and safety measures into the fabric of any research and development process - as fundamental building blocks, not bolted-on afterthoughts. - Build Trust Through Real-World Impact: The antidote to apprehension around AI is to build products that solve real problems, and then highlight those solutions. When people understand how AI is adding clear value to their lives, the conversation can focus both on positive  opportunities and managing risk. I very much appreciated the opportunity to share my thoughts, and you can read more here:

Dear data engineers, you’ll thank yourself later if you spend time learning these today: ⥽ SQL (Advanced) & Query Optimization > AI can help you write SQL, but only you can tune a query to avoid those nightmare full-table scans. ⥽ Distributed Data Processing (Spark, Flink, Beam, etc.) > When datasets grow beyond RAM, knowing Spark or Beam inside out is what lets you scale from gigabytes to terabytes. No AI prompt will save you from shuffling bottlenecks if you don’t get the fundamentals. ⥽ Data Warehousing (Snowflake, BigQuery, Redshift, etc.) > Modern warehouses change the game, partitioning, clustering, and streaming ingestion. Know how and when to use each, or you’ll pay for it (literally, in cloud bills). ⥽ Kafka, Kinesis, or Pub/Sub > Real-time pipelines live and die on event streaming. AI can set up a topic, but only experience teaches you how to avoid data loss, lag, and dead-letter nightmares. ⥽ Airflow & Orchestration > Scheduling DAGs, managing retries, and tracking lineage are what separate side-projects from production. Copilot won’t explain why your pipeline is missing yesterday’s data. ⥽ Parquet, Avro & Data Formats > Efficient formats are what make your pipelines affordable and fast. Learn how and when to use each. AI won’t optimize your storage costs. ⥽ Schema Evolution & Data Contracts > When teams change code, schemas break. Schema evolution is where production pipelines break. Practice versioning, validation, and enforcing data contracts. ⥽ Monitoring & Data Quality > “It loaded, but did it load right?” > AI can’t spot silent data drift or null spikes. Only real monitoring and quality checks will save your job. ⥽ ETL vs ELT > Sometimes you transform before loading, sometimes after. Understand tradeoffs: it’s money, time, and data accuracy. ⥽ Partitioning & Indexing > With big data, these two can make or break your pipeline speed. AI can suggest a partition key, but only hands-on will teach you why it matters. ⥽ SCDs, CDC & Data Versioning > Slowly Changing Dimensions, Change Data Capture, historical accuracy—know how to track what changed, when, and why. ⥽ Cloud Data Platforms (AWS, GCP, Azure) > Learn managed services, IAM, cost controls, and infra basics. Cloud AI tools are great, but you have to make them work together. ⥽ Data Lake Design & Governance > Not all data belongs in a warehouse. Know how to set up, secure, and govern a data lake, or your company will end up with a data swamp. ⥽ Data Privacy & Compliance > GDPR, CCPA, masking, encryption, one slip here, and it’s not just code review, it’s legal. ⥽ CI/CD for Data Pipelines & Git > Automated testing for data flows, rollback for broken jobs, versioning for reproducibility, learn this before a failed deploy ruins your week. Write those data pipelines, break schemas, tune storage, and trace why something failed in prod. That’s how you build instincts. AI will make you faster. But these fundamentals make you irreplaceable.

The OpenAI and Hugging Face disclosure may be one of the most useful insights our industry has published this year. A frontier model, tested with reduced safeguards, escaped its sandbox during a cyber evaluation and reached production infrastructure. Both companies put the details in the open. That takes some courage, and the rest of us should treat it as a free lesson instead of a headline. Here is what I am telling security teams and boards to do with it: 1️⃣ Design for the agent that does not stop on its own. It had no real stopping point, so it kept working the problem until it found a gap. Treat that as the default. Cap session time and tool calls for the agents you operate, and pair the caps with monitoring, since long tasks can split across sessions. Log the full trajectory. It will not stop an escape, but it is what lets you catch and reconstruct one. 2️⃣Build detection that works before you can attribute. Hugging Face contained this while the attacker was still unknown. That is the standard now. Instrument for anomalous behavior. If your detection depends on naming the adversary first, it is already too slow. 3️⃣Pre-authorize decision rights before the incident. Machine speed attacks do not wait for an approval chain. Decide now who can isolate a system, revoke credentials, or pull a service, and under what conditions. Write it down. Rehearse it. The middle of an incident is the worst time to learn you need three signatures to act. 4️⃣Assume breakout and limit the blast radius. Scoped credentials. Real segmentation. Least privilege that is actually enforced. If something gets out of its box, you want it to stay small and get noticed fast. 5️⃣Hold test and dev to production controls. This started inside a lab. Your pre-production environments run real code with real access and thinner guardrails, and attackers know it. Bring them into scope. 6️⃣Confirm your defensive tooling will work under fire. During response, some frontier models blocked analysis of the live payloads, because the safety filters could not tell a defender from an attacker. Test your defensive AI against real incident artifacts before you need it. 7️⃣Rehearse the response, including out of band. Assume your primary channels may be noisy or compromised. Have an out of band way to coordinate and decide. Run the tabletop. The teams that do well in the first hour are the ones who practiced it. None of this is about panic, and none of it is about any single lab. The capability is real and it is only going up. The gap that decides outcomes is visibility and control. That is where boards should put attention and budget this year. If your team read this incident and felt behind, you’re not alone. Use it. It is the cheapest lesson you will get all year.

Stop building GenAI apps like it’s a weekend hobby—start building them like 𝘀𝗰𝗮𝗹𝗮𝗯𝗹𝗲 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲. Here is the structure: Moving from a basic notebook to a production-ready application is where most developers hit a wall. If your project structure is a mess, your AI will be too. Think of your GenAI project like a 𝗵𝗶𝗴𝗵-𝗲𝗻𝗱 𝗿𝗲𝘀𝘁𝗮𝘂𝗿𝗮𝗻𝘁: ✅ 𝗧𝗵𝗲 𝗰𝗼𝗻𝗳𝗶𝗴/ (𝗧𝗵𝗲 𝗠𝗲𝗻𝘂): This is where you define your LLM providers and parameters. You don’t rewrite the menu every time a guest sits down; you keep it centralized. ✅ 𝗧𝗵𝗲 𝘀𝗿𝗰/𝗰𝗼𝗿𝗲/ (𝗧𝗵𝗲 𝗘𝘅𝗲𝗰𝘂𝘁𝗶𝘃𝗲 𝗖𝗵𝗲𝗳): Your model factory logic. It decides whether to call GPT-4, Claude, or a local Llama instance based on the "order." ✅ 𝗧𝗵𝗲 𝗱𝗮𝘁𝗮/𝘃𝗲𝗰𝘁𝗼𝗿𝗱𝗯/ (𝗧𝗵𝗲 𝗣𝗮𝗻𝘁𝗿𝘆): Where your specialized ingredients (embeddings) live. Without a clean pantry, your RAG system will serve "hallucinated" dishes. 🏗️ 𝗧𝗵𝗲 𝗔𝗻𝗮𝘁𝗼𝗺𝘆 𝗼𝗳 𝗮 𝗣𝗿𝗼 𝗚𝗲𝗻𝗔𝗜 𝗥𝗲𝗽𝗼 As shown in this incredible visual by Priyanka Vergadia, a robust structure separates the 𝗯𝗿𝗮𝗶𝗻 from the 𝗽𝗹𝘂𝗺𝗯𝗶𝗻𝗴: ✅ 𝗠𝗼𝗱𝘂𝗹𝗮𝗿 𝗖𝗼𝗿𝗲: Don't hardcode API calls. Use a model factory to switch providers seamlessly. ✅ 𝗣𝗿𝗼𝗺𝗽𝘁 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁: Treat prompts like code, not strings. Store them in prompts/templates file for versioning and reusability. ✅ 𝗥𝗔𝗚 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲: Keep your embedder, indexer, and retriever logic distinct. It makes debugging "retrieval failure" 10x faster. ✅ 𝗖𝗼𝗻𝘁𝗮𝗶𝗻𝗲𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Use Docker to ensure your vector DB and environment stay consistent across dev and prod. 𝗖𝗹𝗲𝗮𝗻 𝗰𝗼𝗱𝗲 𝗶𝘀 𝘁𝗵𝗲 𝘀𝗲𝗰𝗿𝗲𝘁 𝘀𝗮𝘂𝗰𝗲 𝘁𝗼 𝗿𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗔𝗜. When your structure is modular, you spend less time fixing broken imports and more time optimizing your context windows. Like this post? Consider resharing with your network and follow Priyanka for more cloud and ai tips.

If you're an AI engineer building RAG pipelines, this one’s for you. RAG has evolved from a simple retrieval wrapper into a full-fledged architecture for modular reasoning. But many stacks today are still too brittle, too linear, and too dependent on the LLM to do all the heavy lifting. Here’s what the most advanced systems are doing differently 👇 🔹 Naïve RAG → One-shot retrieval, no ranking or summarization. → Retrieved context is blindly appended to prompts. → Breaks under ambiguity, large corpora, or multi-hop questions. → Works only when the task is simple and the documents are curated. 🔹 Advanced RAG → Adds pre-retrieval modules (query rewriting, routing, expansion) to tighten the search space. → Post-processing includes reranking, summarization, and fusion, reducing token waste and hallucinations. → Often built using DSPy, LangChain Expression Language, or custom prompt compilers. → Far more robust, but still sequential, limited adaptivity. 🔹 Modular RAG → Not a pipeline- a DAG of reasoning operators. → Think: Retrieve, Rerank, Read, Rewrite, Memory, Fusion, Predict, Demonstrate. → Built for interleaved logic, recursion, dynamic routing, and tool invocation. → Powers agentic flows where reasoning is distributed across specialized modules, each tunable and observable. Why this matters now ⁉️ → New LLMs like GPT-4o, Claude 3.5 Sonnet, and Mistral 7B Instruct v2 are fast — so bottlenecks now lie in retrieval logic and context construction. → Cohere, Fireworks, and Together are exposing rerankers and context fusion modules as inference primitives. → LangGraph and DSPy are pushing RAG into graph-based orchestration territory — with memory persistence and policy control. → Open-weight models + modular RAG = scalable, auditable, deeply controllable AI systems. 💡 Here are my 2 cents- for engineers shipping real-world LLM systems: → Upgrade your retriever, not just your model. → Optimize context fusion and memory design before reaching for finetuning. → Treat each retrieval as a decision, not just a static embedding call. → Most teams still rely on prompting to patch weak context. But the frontier of GenAI isn’t prompt hacking, it’s reasoning infrastructure. Modular RAG brings you closer to system-level intelligence, where retrieval, planning, memory, and generation are co-designed. 🛠️ Arvind and I are kicking off a hands-on workshop on RAG This first session is designed for beginner to intermediate practitioners who want to move beyond theory and actually build. Here’s what you’ll learn: → How RAG enhances LLMs with real-time, contextual data → Core concepts: vector DBs, indexing, reranking, fusion → Build a working RAG pipeline using LangChain + Pinecone → Explore no-code/low-code setups and real-world use cases If you're serious about building with LLMs, this is where you start. 📅 Save your seat and join us live: https://lnkd.in/gS_B7_7d

This year, India’s defense sector unveiled advancements in AI that are reshaping military strategies & boosting national security. Here’s what the data tells us: --> AI is now central to defense modernization. --> Collaboration across sectors is driving innovation. Let’s explore these in detail. 1️⃣ AI-Powered Technologies Transforming Defense India’s armed forces are deploying AI across critical areas: ➤ Autonomy in operations: AI-enabled systems like swarm drones & autonomous intercept boats enhance mission precision, reduce human risk, & improve tactical outcomes. ➤ Intelligence, Surveillance, & Reconnaissance (ISR): AI-based motion detection & target identification systems provide real-time alerts for better situational awareness along borders. ➤ Advanced robotics: Silent Sentry, a 3D-printed AI rail-mounted robot, supports automated perimeter security & intrusion detection. Example: Swarm drones use distributed AI algorithms for dynamic collision avoidance, target identification, & coordinated aerial maneuvers, providing versatility in both offensive & defensive tasks. 2️⃣ Collaboration as the Catalyst for Innovation India’s AI advancements are the result of partnerships between the government, private industries, & research institutions. ➤ Indigenous solutions: 100% indigenously developed systems like the Sapper Scout UGV for mine detection. ➤ Startups and SMEs: Innovative contributions from tech firms and startups have fueled projects like AI-enabled predictive maintenance for naval ships and drones. ➤ Global export potential: Systems like Project Drone Feed Analysis and maritime anomaly detection tools are export-ready, positioning India as a major global defense tech player. 3️⃣ The Data-Driven Case for AI ➤ Efficiency: AI-driven systems exponentially improve surveillance coverage and reduce operational time. For example, the Drone Feed Analysis system decreases mission costs while expanding surveillance areas. ➤ Safety: Predictive AI systems in vehicles and maritime platforms enhance safety by identifying potential risks before failures occur. ➤ Economic impact: AI-powered predictive maintenance for critical assets like naval ships and aircraft maximizes uptime while minimizing costs. Real Impact ➤ Swarm drones: Affordable, scalable, and capable of BVLOS operations, offering precision in combat. ➤ AI-enabled maritime systems: Detect anomalies in vessel traffic, securing trade routes and protecting economic interests. ➤ AI-driven mine detection: Enhances soldier safety while automating high-risk tasks. What does this mean for defense organizations? AI isn’t just modernizing defense; it’s placing it firmly in the global defense innovation market. With bold policies, dedicated budgets, and a growing ecosystem of public and private sector players, this will help lead the next wave of AI-driven defense technologies. But the question remains: How do we ensure these technologies are deployed ethically and responsibly? Agree?