<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Modern Data 101  on Medium]]></title>
        <description><![CDATA[Stories by Modern Data 101  on Medium]]></description>
        <link>https://medium.com/@community_md101?source=rss-550c0299513d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*M8mSi4GdQ0MbXoDcl6YWiQ.jpeg</url>
            <title>Stories by Modern Data 101  on Medium</title>
            <link>https://medium.com/@community_md101?source=rss-550c0299513d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 11:50:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@community_md101/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[5 Data Management Best Practices to Improve AI ROI]]></title>
            <link>https://medium.com/@community_md101/5-data-management-best-practices-to-improve-ai-roi-af29f7ec86d5?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/af29f7ec86d5</guid>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[data-modeling]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 14:07:42 GMT</pubDate>
            <atom:updated>2026-11-05T14:07:42.856Z</atom:updated>
            <content:encoded><![CDATA[<h4>Five architectural shifts required to turn raw enterprise data into highly actionable, hallucination-free AI reasoning layers.</h4><p>Author: <a href="https://www.linkedin.com/in/rakesh-vishvakarma/">Rakesh Vishvakarma</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/ai-in-data-management-best-practices"><strong><em>Modern Data 101</em></strong></a><strong><em> Website.</em></strong></p><h3>TL;DR</h3><p>Major AI projects still fail:<a href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-2024"> <strong>60% of AI projects fail due to data readiness issues</strong></a></p><p>Three shifts make this unavoidable:</p><p>1. First, data volume and complexity have outgrown human interpretation. Modern systems generate structured, semi-structured, and unstructured data continuously. Without AI-assisted management, understanding, classifying, and governing this at scale becomes impractical.</p><p>2. Second, AI systems (especially <a href="https://www.moderndata101.com/blogs/5-ways-ai-agents-will-transform-data-management-analytics">LLMs and agents) are now primary consumers of data</a>, not just analysts. They depend on consistent semantics, quality signals, and lineage to produce reliable outputs. If the data layer is ambiguous, AI amplifies that ambiguity into automation errors.</p><p>3. Third, organisations are moving from static reporting to autonomous decision loops. This requires data that is continuously validated, contextualised, and governed, instead of just stored and queried.</p><p>This article covers five AI data management best practices that data leaders can begin applying immediately, and for each one, how a modern data platform is the foundational layer that makes it scalable and sustainable.</p><p>Take a pause and do checkout the <a href="https://www.moderndata101.com/reports/ai-governance-benchmark-enterprise-readiness">AI Governance Benchmark 2026: Why Only 4% of Enterprises Are AI-Ready</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dPx13LetsSzI585v4Hi9iw.png" /></figure><h3>Best Practices for AI in Data Management</h3><h4>1. Consolidate Your Data Foundation Before Scaling AI</h4><p>AI is seen to increase the impact of fragmented platforms, duplicated pipelines, inconsistent definitions, and isolated datasets spread across cloud, on-premises, SaaS, and operational systems.</p><p><a href="https://www.moderndata101.com/blogs/ai-data-management-vs-traditional-data-management-which-one-saves-time">Traditional automation was built for stable environments</a> with predictable inputs. Modern data environments are neither. Data structures change continuously, new sources appear constantly, and business context shifts faster than static rules can keep up. Maintaining thousands of brittle transformations and integrations becomes operational drag rather than enablement.</p><figure><img alt="Comparison table contrasting traditional and AI-driven data management across seven dimensions." src="https://cdn-images-1.medium.com/max/1024/0*UDGQWty_eySQdbZx" /><figcaption><em>Traditional vs. Ai-driven data management | Source: </em><a href="https://www.moderndata101.com/blogs/ai-data-management-vs-traditional-data-management-which-one-saves-time#:~:text=At%20a%20Glance%3A%20How%20AI%20and%20Traditional%20Data%20Management%20Compare"><em>Modern Data 101 Articles</em></a></figcaption></figure><p>AI changes data management by shifting systems from rule execution to adaptive interpretation. Instead of manually defining every exception, AI systems can detect anomalies, infer relationships, classify unstructured content, and identify quality issues dynamically as the ecosystem evolves.</p><p>But none of this works in fragmented architectures. <a href="https://moderndata101.substack.com/p/modern-data-report-2026">Research across 500+ data practitioners</a> shows the same pattern repeatedly: excessive tooling, integration sprawl, and disconnected platforms limit how effectively organisations can operationalise AI. Consolidation onto a modern, interoperable data platform is no longer an optimisation layer. It is the operational foundation for AI readiness.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/what-is-ai-readiness-and-how-to-be-ai-ready">What is AI-Readiness and How to Be AI-Ready</a></blockquote><h4>2. Shift from Datasets to Data Products to Make AI a First-Class Consumer</h4><figure><img alt="Matrix charting stakeholder groups across impact and influence levels for enterprise AI change management." src="https://cdn-images-1.medium.com/max/746/0*dyv2eRPhaC_HVXQA" /><figcaption><em>Impact-Influence Matrix for Change Management in Enterprise AI | </em><a href="https://moderndata101.substack.com/p/first-strategy-piece-of-enterprise-ai"><em>Source</em></a></figcaption></figure><p>At scale, AI only works when data is structured as products.</p><p>A <a href="https://www.moderndata101.com/blogs/building-robust-data-products-5-pillars-every-data-engineer-should-apply">data product is a managed unit with an owner</a>, contract, defined consumers, and lifecycle; versioning, deprecation, and controlled evolution are included. It carries what AI systems otherwise reconstruct: schema, semantics, quality, freshness, and lineage.</p><p>This shifts the context from documentation to the architectural layer. This is managed, versioned, discoverable, and can be connected to, queried by, and relied on by anything that needs to reason about the company’s data.</p><blockquote><em>The </em><a href="https://www.moderndata101.com/modern-data-report-2026"><em>Modern Data Report 2026 is unambiguous</em></a><em>: semantic layer beats shiny tools every time. Machines do not inherit tribal knowledge. If definitions, context, and trust are not encoded in the data layer, automation cannot scale.</em></blockquote><p>AI agents intensify this. Each additional agent on poorly defined data increases the inconsistency cost. Their autonomy depends on embedded governance, quality contracts, and clear action boundaries.</p><p>Start small: pick one domain, define a reliability problem, add monitoring and governance boundaries, separate autonomous vs human-approved actions, then scale once stable.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/lean-ai-building-a-scalable-data-platform-for-enterprise-ai-roi">Lean AI: Building a Scalable Data Platform for Enterprise AI ROI</a></blockquote><h4>3. Embed a Semantic Layer to Make Data AI-Actionable</h4><p><strong>What is a semantic layer in data management, and why do AI systems depend on it?</strong></p><p><a href="https://www.moderndata101.com/blogs/why-enterprises-need-a-semantic-data-model-to-drive-business-success">A semantic layer is the business context layer between raw data and consumption systems</a>. It standardises definitions, metrics, relationships, and meaning across the organisation.</p><p>AI systems depend on it because AI needs business context, not just raw data. The semantic layer helps AI interpret data consistently, reduce hallucinations, and generate reliable, trustworthy outputs.</p><figure><img alt="Diagram showing how conceptual modeling bridges business understanding, enterprise ontology, and data solution design." src="https://cdn-images-1.medium.com/max/1024/0*Dw51Qnqm0sKo8Q6j" /><figcaption><em>How Data Models and Ontologies Connect to Build Semantic Foundations | </em><a href="https://moderndata101.substack.com/p/semantic-foundations-with-data-models-or-ontology?utm_source=publication-search"><em>Source</em></a></figcaption></figure><p>Poor contextual understanding of data renders technically accurate outputs that are operationally misleading.</p><p>For AI systems specifically, a semantic layer changes the nature of interaction entirely. Models operate on business-aware representations of customers, products, transactions, risks, policies, and operational events. That context is what makes AI outputs usable, explainable, and adaptable to evolving business questions.</p><p>However, the underlying data might remain inconsistent, delayed, or poorly governed. This is why semantic consistency must be paired with trustworthy data products that enforce quality, ownership, versioning, and governance upstream.</p><p>This is also where a <a href="https://www.moderndata101.com/blogs/metrics-focused-data-strategy-with-model-first-data-products">model-first approach becomes important</a>. The business meaning, relationships, and intended AI consumption patterns are designed into the data product itself from the beginning. The focus shifts from merely moving data to shaping data that can reliably support reasoning systems, automation, analytics, and AI agents at scale.</p><figure><img alt="Infographic comparing core concepts and structural differences between semantic and relational data models." src="https://cdn-images-1.medium.com/max/612/0*RablxYA0o4B1sIYw" /><figcaption><em>Fundamental difference between semantic and relational models | Source: </em><a href="https://www.linkedin.com/posts/olga-maydanchik-23b3508_the-majority-of-organizations-today-are-deeply-activity-7154898186178850816-ce9k?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAADGpeHEBnx8wQxspRs3v5o9LUcdl8f9dc-A"><em>Olga Maydanchik</em></a></figcaption></figure><h4>4. Adopting a Data Lakehouse Architecture</h4><p>Traditional data warehouses delivered reliability but struggled with scale, flexibility, and unstructured data. Data lakes solved storage flexibility, but often lacked governance and trust.</p><p>Data lakehouses are accelerating AI readiness for <a href="https://www.dremio.com/press-releases/new-state-of-the-data-lakehouse-in-the-ai-era-report-shows-data-lakehouses-accelerating-ai-readiness-for-85-of-firms/">85% of firms that have adopted them</a>.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/data-lakehouse-vs-data-warehouse-vs-data-mart">Data Lakehouse vs Data Warehouse vs Data Mart</a></blockquote><p>The<a href="https://www.moderndata101.com/blogs/what-are-the-challenges-of-data-lakes-how-data-lakehouses-help"> lakehouse combines the scalability of a data lake with the reliability and transactional control</a> of a warehouse in a single architecture. For AI, this matters because structured and unstructured data must coexist to support analytics, machine learning, and AI agents. Customer records, logs, documents, images, and real-time events can all live within the same environment.</p><figure><img alt="Diagram contrasting vendor-locked format conformity with flexible, interoperable lakehouse architectures." src="https://cdn-images-1.medium.com/max/1024/0*3ZzTa6nXVLeG-JHI" /><figcaption><a href="https://www.moderndata101.com/blogs/lakehouse-2-0-the-open-system-that-lakehouse-1-0-was-meant-to-be-part-1"><em>Source</em></a></figcaption></figure><p>But the architecture has evolved. Early lakehouses often recreated lock-in by tightly coupling compute, storage, metadata, and governance. The <a href="https://www.moderndata101.com/blogs/introducing-lakehouse-2-0-what-changes">next-generation lakehouse shifts toward openness</a>: open table formats, decoupled compute and storage, interoperable governance, and portable data products.</p><p>The lakehouse is the foundation, but modern data platforms sit above it, enabling semantic consistency, governance, and reusable data products that remain portable across tools, engines, and evolving AI ecosystems.</p><h4>5. Implement AI-Augmented Data Governance</h4><p>Traditional data governance frameworks were built for compliance. They answered: <em>“Can we prove our data was handled correctly?”</em> Modern AI-augmented governance answers a more demanding question: <em>“Can we guarantee that every AI system operating on our data will produce trustworthy outputs?”</em> These are fundamentally different problems.</p><p>Without a comprehensive data strategy with executive support, AI ambitions will struggle and have a high probability of failing. Many organisations share the struggle of having too many tools, resulting in overly complex solutions that fundamentally limit which AI use cases can be pursued.</p><p><a href="https://www.moderndata101.com/blogs/what-is-the-ai-data-governance-gap-why-it-keeps-getting-worse">AI-augmented governance makes governance continuous</a>, automated, and embedded rather than periodic, manual, and bolted on:<br>‍</p><ul><li><strong>Automated lineage tracking</strong>: AI traces data from source to consumption across complex multi-system environments, surfacing dependencies that manual documentation misses.</li><li><strong>Policy enforcement at ingestion</strong>: Governance rules apply at the point of data entry, rejecting non-conforming records before they enter the pipeline.</li><li><strong>Access control and privacy</strong>: ML models classify data sensitivity automatically, applying RBAC and PII controls without requiring manual tagging of every asset.</li><li><strong>Regulatory readiness</strong>: For GDPR, CCPA, and HIPAA environments, AI governance layers maintain audit trails and demonstrate compliance continuously.</li></ul><h4>The Common Thread: A Modern Data Platform as the Foundation</h4><p>Each of these five <strong>AI data management best practices</strong> points to the same architectural conclusion. The data layer itself must evolve. AI models are only as capable as the data they operate on, and data is only as capable as the infrastructure it runs within.</p><p>The organisations succeeding with AI in data management share a common foundation: a <strong>modern data platform</strong> that consolidates data across environments, enforces quality continuously, makes data discoverable through semantic contracts, governs access proactively, and serves data as products to both human and AI consumers.</p><h3>FAQs</h3><h4>Q1. What are 5 best practices for synthetic data use</h4><p>Define a clear use case + metric. Anchor synthetic data to real distributions. Preserve schema, semantics, and business constraints. Govern generation with versioning and lineage. Continuously validate against real data drift and update generators accordingly.</p><h4>Q2. What is AI for data management?</h4><p>AI for data management is the use of machine learning and automation to discover, classify, clean, govern, and optimise data across systems, making data more reliable, contextual, and usable for analytics, applications, and AI agents at scale.</p><h4>Q3. What are the 5 C’s of data management?</h4><p>The “5 C’s of data management” commonly refer to:</p><ul><li><strong>Collect</strong> — capture relevant data from systems and sources</li><li><strong>Cleanse</strong> — remove errors, duplicates, and inconsistencies</li><li><strong>Consolidate</strong> — integrate data into a unified view</li><li><strong>Contextualise</strong> — add meaning, metadata, and business definitions</li><li><strong>Consume</strong> — make data accessible for analytics, AI, and decision-making</li></ul><h4>MD101 Connect ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><p><em>From MD101</em> <em>team</em> 🧡</p><h4><strong>AI Governance Benchmark 2026</strong></h4><p>60% of companies scale AI, yet only 4% possess governance mature <strong>enough to manage the risk.</strong> Discover how industry leaders bridge the gap to capture 3.6x higher returns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*8_w0whwV-bQcH7gr.png" /><figcaption><a href="https://www.moderndata101.com/reports/ai-governance-benchmark-enterprise-readiness">Download full report</a></figcaption></figure><p>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af29f7ec86d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Incentive Map Every Data Platform PM Should Draw]]></title>
            <link>https://medium.com/@community_md101/the-incentive-map-every-data-platform-pm-should-draw-8d9f1b71ae95?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d9f1b71ae95</guid>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Tue, 03 Nov 2026 14:29:20 GMT</pubDate>
            <atom:updated>2026-11-03T14:29:20.111Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://moderndata101.substack.com/s/experts-desk/?utm_source=substack&amp;utm_medium=menu">Expert’s Desk ⭐️</a></h4><h4>Revealing the Invisible Layer That Powers (or Blocks) Platform Success</h4><p>Author: <a href="https://www.linkedin.com/in/annabergevin/">Anna Bergevin</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://moderndata101.substack.com/p/incentive-map-for-data-pm"><strong><em>Modern Data 101</em></strong></a><strong><em> Substack.</em></strong></p><h4>About Our Contributing Expert</h4><h3>Anna Bergevin | Sr. Manager Data &amp; AI Product Management</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*POeiuUjYintht_Rl.png" /></figure><p><a href="https://www.linkedin.com/in/annabergevin/">Anna Bergevin</a> is a data and AI product leader currently serving as Sr. Manager of Data &amp; AI Product Management at <em>Children’s Mercy</em>. She <strong>leads platform product strategy</strong> focused on enabling secure, compliant, and scalable data systems that power analytics and AI across the enterprise.</p><p>Previously, she <strong>led global Data &amp; AI platform product</strong> initiatives at <em>ResMed</em>, guiding roadmap and governance strategy for a distributed engineering organization supporting business teams in over 140 countries. Her work centers on semantic modeling, metric standardization, metadata-driven governance, and designing data products that balance regulatory rigor with innovation.</p><p>Anna writes and speaks about data product management, sociotechnical AI adoption, and the discipline required to build enterprise data systems that scale on <a href="https://dataprodmgmt.substack.com/"><em>When Data Met Product</em></a>. She believes the future of AI depends less on model sophistication and more on clear definitions, strong governance, and product-led stewardship of data infrastructure. We’re thrilled to feature her unique insights on <a href="https://moderndata101.substack.com/">Modern Data 101</a>!</p><blockquote>We actively collaborate with data experts to bring the best resources to a <a href="https://www.moderndata101.com/">15,000+ strong community</a> of data leaders and practitioners. If you have something to share, reach out!<br>🫴🏻 Share your ideas and work: <a href="http://community@moderndata101.com/">community@moderndata101.com</a></blockquote><p>Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality &amp; relevance. We keep it information-first and do not support any promotions, paid or otherwise.</p><h3>Let’s Dive In</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VoFPX71Nt4y9j6wV.png" /></figure><p>Data platform product managers have an impossible job: <strong>drive adoption and impact without formal authority over the teams they serve.</strong></p><p>To succeed, you need to think like a game theorist. <a href="https://moderndata101.substack.com/i/150092028/a-light-introduction-to-game-theory">Game theory</a>, the study of strategic decision-making, assumes people act rationally given their incentives and constraints. Behavior that may appear confusing on the surface makes sense if we dig deeper to understand the incentives that drive their behavior.</p><p>The problem isn’t that your stakeholders are irrational. It’s that you often don’t understand the games they’re playing.</p><blockquote><em>🔖 </em><strong><em>Related Read</em></strong></blockquote><blockquote><a href="https://moderndata101.substack.com/p/dont-trust-decentralisation-yet-game">Don’t Trust Decentralisation Yet? Game Theory Might Change Your Stance</a></blockquote><p>Software engineers aren’t ignoring your metadata standards because they’re lazy. They’re <strong>optimizing for velocity</strong> because that’s how they get promoted, and the standards require changes that feel burdensome to adopt.</p><p>Analysts aren’t creating warehouse chaos because they don’t care about governance. They’re solving today’s urgent business question because that’s what their VP is waiting for. They aren’t thinking about how their quick solutions create warehouse issues at scale.</p><figure><img alt="Illustration showing a product manager facing a wall of resistance while stakeholders run paths driven by incentives." src="https://cdn-images-1.medium.com/max/1024/0*ATOYNmuQDf0A2RQ5.png" /><figcaption>The Data PM’s Dilemma | Adapted from concepts shared by Author | Curated by Modern Data 101</figcaption></figure><p><em>Your stakeholders are rational, and you just need to understand their incentives better.</em></p><p><strong>The Incentive Map is the invisible layer that powers organizational behavior.<br></strong>Once you see it, your job as a platform PM gets infinitely easier. You stop fighting human nature and start designing with it. You build products that consider the needs of the diverse set of people engaging with the system, and not just the obvious users at the center.</p><figure><img alt="Cross-section diagram showing surface office behaviors influenced by an underlying X-Ray network of organizational incentives." src="https://cdn-images-1.medium.com/max/1024/0*CIBKqfAxK6fchyvk.png" /><figcaption>The Invisible Layer of Organizational Behavior | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Building this map? That requires systematic observation across four dimensions: structure, success metrics, systems, and people.</p><h3>The Four Dimensions of Incentive Mapping</h3><figure><img alt="Compass graphic mapping the four incentive dimensions: Structure, Success Metrics, Systems, and People." src="https://cdn-images-1.medium.com/max/1024/0*uSslc2JP846r25c1.png" /><figcaption>The Four Dimensions of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><h4>1. Structure</h4><p><strong>Data platform teams serve the entire company,</strong> which means understanding org structure across the business is critical. Organization structure reveals who sets incentives, how information flows, and what leadership believes about how teams should work together. It gives us clues about where collaboration will be easy or difficult.</p><p>When I join a new company (or hire someone onto my team), one of my first priorities is understanding the key people who surround my role. Eventually, as individuals, but first and foremost, the teams — who report into whom and how that intersects with my team.</p><blockquote>- “Ah, okay she works on DevX, which reports to the VP Eng Platform alongside the data platform. All up through the CIO, interesting.”</blockquote><blockquote>- “Alright so this analyst is embedded in finance, but supply chain has their own finance analysts, what’s that dynamic?”</blockquote><blockquote>- “He’s an architect on the software side….reports into a product leader….looks like we don’t have a CTO, okay so software eng is seen here as a function under CPO, okay.”</blockquote><p>These mappings uncover how the organization thinks about each group. I’ve worked on data teams under both a CTO and a CIO. Under the CTO, all of engineering (software, enterprise applications, data) shared the same org and goals.</p><p>Under the CIO, we aligned with enterprise applications while the software engineers were being driven by goals set by the product leadership. That changed the collaboration dynamic significantly.</p><figure><img alt="Diagram contrasting CTO-aligned engineering teams with siloed Data and Software teams split between CIO and CPO reporting lines." src="https://cdn-images-1.medium.com/max/1024/0*6_nFsIIFnRnck5em.png" /><figcaption>The Structural Dimension of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Organizational structure determines:</p><ul><li>Who sets incentives, and what those incentives reward</li><li>How planning and prioritization happen</li><li>Where silos form and where context needs bridging</li><li>Whether cross-team dependencies are visible or hidden</li></ul><p>Org design decisions and changes happen for many reasons: strategy shifts, layoffs, mergers, cost reduction. But regardless of the trigger, leaders are making choices about which teams sit together, who reports to whom, and how work should flow. Those choices reshape incentives and communication patterns, whether that’s the primary goal or a side effect.</p><h4>2. Success Metrics</h4><p>Understanding the org structure tells you who sets incentives. Understanding success metrics tells you <strong>what those incentives actually reward.</strong> What are the called-out KPIs for different teams? Is there clarity or ambiguity? Do some people feel pulled in conflicting directions? Are individual performance measures aligned with team goals, or are they at odds?</p><p>Once you understand how the company measures success, you can see why certain tensions exist.</p><h4>Example: When I didn’t understand the success metrics</h4><p>Early in my product platform career, I was PM-ing the communications platform supporting email, in-app notifications, and mobile push for all product development teams.</p><p>Very quickly, I discovered a tension between Marketing and Product around communications ownership.</p><ul><li>Product felt they should own user engagement comms. Like many product orgs, they were measured by Monthly Active Users, and email was one of the only tools to re-engage dormant users.</li><li>Marketing also wanted to interact with individual users and drive engagement.</li></ul><p>At first, I was perplexed. In a B2B SaaS company, why would Marketing care about individual user emails when buyers make the purchasing decisions?</p><p>Then I had a breakthrough. I realized marketing was being measured by <strong>account retention</strong>, which in our B2B edtech product was deeply tied to usage rates. Enterprise buyers purchased licenses, but renewals depended on whether people actually used the product.</p><p>Marketing didn’t just want to email buyers — they wanted leverage to drive individual usage. And frankly, it seemed like they didn’t trust Product to do comms as well as they could.</p><figure><img alt="Scale balancing Marketing’s Account Retention metric against Product’s Monthly Active Users goal." src="https://cdn-images-1.medium.com/max/1024/0*ChxcnZngXeSZJtet.png" /><figcaption>Success Metrics, Second Dimension of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Once I realized this, I had a lot more empathy. Marketing’s success metric was hard to impact through buyer-only lifecycle emails. And they had a point: individual product teams were sending emails that were loose on branding, lacked a comms strategy, and had no coordination across teams. We were just giving them an API and letting them do whatever. I decided to change my strategy with marketing to see if we could collaborate more closely, improve brand alignment, utilize their expertise in comms strategy on critical emails, etc.</p><h4>The lesson: Conflicting success metrics create predictable tensions.</h4><p><strong>The reality:<br></strong>Companies are imperfect at measuring success. As data professionals, we know why: lack of skill at picking the right KPI, office politics driving selection, data quality or availability issues that make measuring what matters difficult.</p><p>Even as companies iterate on metrics over time, tensions will probably still exist. But once you understand the metrics, you can strategize and find common ground that lets both teams find success and helps the company reach its overall goals.</p><h4>3. Systems</h4><p>While structure and success metrics are slower-changing dimensions, organizations have many cycles that create pressure and shift priorities and incentives for our colleagues.</p><figure><img alt="Timeline illustrating how Planning, Launch, and Board cycles drive high-pressure behaviors." src="https://cdn-images-1.medium.com/max/1024/0*-552z-3SHBXtH94y.png" /><figcaption>Systems, Third Dimension of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Cycles like planning, budgets, promotions, and performance reviews — these create unique pressures on individuals, and they aren’t uniform. A few examples to illustrate:</p><h4>Planning</h4><p>You learn once again that one of your analysts has created an expensive, monstrous ingest query into Tableau. Why do they keep doing this? Didn’t you just have a great meeting talking about investing in data products, shifting left, and maximizing efficiency for the company?</p><p>Well, yeah, you did, but what you didn’t learn is that leadership just pivoted strategy hard in the quarterly planning cycle, and your analyst had to start all over with the dashboard for their group.</p><p>The leaders are stressed and want metrics asap so they can see if the pivot is working or not. Your analyst can’t wait to build a curated model on this one; they just had to get it up and running as fast as possible.</p><p>Once you understand this, you can meet them where they are. Help them ship fast now, then collaborate on refactoring it into a proper data product next quarter. It’s not resistance to your platform strategy — they just had to deliver under pressure.</p><h4>Product / Launch Cycle</h4><p>You’ve been working for months on a new metadata as code project. You are super excited because you’ve been talking to a few teams and think you have a prototype that will work to let them push their metadata to you through a GitHub action, and you’ll validate it from there and push to the rest of the system.</p><p>But you approach one team, and they shut you down hard. It’s confusing, you thought they were on board, and other teams have been open to testing.</p><p>You ask your manager, and they remind you that the team owns the top priority feature that is launching at a user conference at the end of the quarter. They have zero margin for error. It has to be perfect and demo-able, without any embarrassing bugs. They just cannot take on something “extra” like a governance ask that feels like a distraction they can’t afford right now.</p><p>Work with other teams instead, wish them good luck, and circle back next quarter to celebrate their successful launch and bring up metadata as code again. By then, you might even have learned from the early adopters and have a second prototype for them to test.</p><h4>Leadership Pushing AI Initiatives</h4><p>Your leaders are bringing up AI across the board, and the pressure is intensifying. Even though you try to showcase progress and point to other work you’re juggling (cost savings, shipping important data pipelines, etc), they just keep coming back to “<em>How are things going on the GenAI pilots?</em>”</p><p>The tech is new, and everyone is learning, so progress is slow, and you are irritated — we’ll get there when we get there. Why is he being so intense about this?</p><p>What you aren’t seeing is that your leaders have a board meeting and budget cycle coming up. They are worried the board will ask hard questions, and they won’t have clear progress to show on an AI feature that resonates with that audience.</p><p>They don’t want a lukewarm update. <strong>They want a wow factor.</strong> Additionally, with the annual budgeting cycle, they need to justify renewal of that GenAI spending (or maybe an increase) and are worried lukewarm results will make that difficult.</p><p>If you understand the pressures from those board/budget cycles, the increasing AI pressure amid steady progress (but without a wow factor) makes sense. Maybe you can achieve something that feels more like a milestone they can run with?</p><p>In all these scenarios, the pattern is the same. Behavior that appears irrational, but suddenly comes into focus once you understand the pressure they’re under from these outside cycles. When you understand that, you can pivot your strategy: what you’re doing, the timeline, your priorities, etc and find a path forward.</p><h4>4. People</h4><p>The first three dimensions give you a systematic map of incentives: where people sit in the org, what they’re measured on, what cycles they’re navigating. But organizations aren’t just systems. They’re made up of humans with egos, ambitions, histories, and relationships that don’t show up on any org chart.</p><figure><img alt="Network map showing informal power, grudges, and alliances centered around an influential architect." src="https://cdn-images-1.medium.com/max/1024/0*pnb1ySxHuw-lgG4V.png" /><figcaption>The Human Element of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Some people are more influential than their title suggests. Others have reputations, deserved or not, that shape how their ideas are received. There are grudges from past projects, alliances between teams, and personalities that either smooth collaboration or create friction.</p><p>You can’t template this layer. What you can do is stay curious. Pay attention to who holds informal power, which relationships are strained, and where past conflicts still echo.</p><p><em>Your incentive map isn’t complete without accounting for the fact that people are people: complex, political, and rarely as rational as game theory would predict.</em></p><h3>The Four Stakeholder Archetypes</h3><p>The four dimensions reviewed above give you the structural foundation of your incentive map — how your organization is designed, what it rewards, the cycles that create pressure, and the people who have influence. But to make the map <em>actionable</em>, you need to understand the <em>players</em> navigating that structure.</p><figure><img alt="Diagram showing four player archetypes: The Engineer, The Analyst, The Leader, and The Business." src="https://cdn-images-1.medium.com/max/1024/0*76_WBjiHE3DAVQ-n.png" /><figcaption>The Four Stakeholder Archetypes of Incentive Mapping | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>While every individual is unique, I have found that certain roles share predictable incentives, constraints, and pain points that shape how they operate. The four stakeholder archetypes below will help you interpret the behavior of your colleagues: what they optimize for, what drains them, and what they commonly misunderstand about each other.</p><p>And at the end of your day, understanding your users deeply and empathizing with their pain points is what product is all about.</p><p><em>That’s how you design sticky products people love, by building solutions that work with human nature instead of against it.</em></p><h4>1. Software Engineers: Optimizing for Velocity</h4><figure><img alt="Card diagram outlining software engineer incentives, downstream blind spots, and automated platform strategies." src="https://cdn-images-1.medium.com/max/1024/0*Zd89Y6hUYfEbeJOC.png" /><figcaption>Archetype 1: Software Engineers | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>This group includes software engineers, enterprise application engineers, solution architects, and other engineers outside the data team. For these users, the key is understanding that they are <em>incentivized to ship business-prioritized features</em>.</p><p>New work is seen as the most valuable, interesting, and engaging–while maintenance and tech debt are viewed as less desirable.</p><p>Software engineers value solving complex problems with autonomy, automation over manual processes, and minimizing context switching. They find many meetings burdensome, and processes a distraction; they’ll push back against unexpected pivots with no notice, and face invisible constraints (infosec, dev rules) that make “simple” requests more complex than one might think.</p><h4>The common misunderstanding</h4><p>Data platform PMs sometimes don’t realize that software engineers are <em>blind to downstream data usage</em>. They’re building for the <em>application</em> use case and are often not thinking of or even aware of data dependencies on their application data.</p><p>They don’t <em>want</em> to break dashboards or ML models, but they have no visibility into those dependencies. If you haven’t built data contracts integrated into their development lifecycle, you can’t really expect them to prevent breakage.</p><h4>The design implication</h4><p>Don’t create manual governance processes and expect compliance. Build automated tooling that fits their workflow: data contracts, metadata as code, and programmatic data access. Make the optimal path the <em>easiest</em> path, and you may be surprised by the uptake.</p><p><strong>An example: </strong>For most businesses, one of the greatest barriers we face in getting value from data is a lack of context on the data. One of the most requested improvements nearly every stakeholder wants is better table and column level descriptions, richer metadata and more context overall.</p><p>The challenge data teams face? We don’t own that system, and when we contact the teams that do, we hear things like:</p><ul><li>We have a Confluence page, but it’s out of date</li><li>Todd knows everything, so if you have questions, message Todd</li><li>Honestly, I don’t think it’s written down in any one place (implication: context lives in the code base comments, Jira tickets, and scattered resources).</li></ul><p><strong>So how do we overcome this?</strong><br>In the short term, you can do what I call documentation sprints. You have them send you their messy docs, no matter the format, and you work on cleaning them up and validating them.</p><p>You interview Todd and write down his answers. You beg the tech lead to populate a spreadsheet. Even if you are successful, this only solves it for a single point in time; there’s still no plan for keeping it up to date.</p><p>But we had a hypothesis that if we made it easier to send us that metadata programmatically, through a standard format, that would be better received. We also thought that we’d make the update as simple as possible: follow the format, and we’ll manage validating it, handling versioning against prior metadata commits, and cascading it downstream to the warehouse and data catalog.</p><p>When we did user research with our software engineers, this idea was <em>very</em> sticky. So sticky that we were hit with comments like “<em>Oh my gosh, this would replace our Confluence page, we could use this for our own documentation</em>” and even “<em>I love this, it would be even better if you could add [insert many feature requests]</em>.” Suddenly, our software engineers were engaged and eager to use this solution, because it was designed to match their incentives:</p><ul><li>Solves a core problem they have for their application development work: documenting the system in a way that is more structured than a wiki; helping preserve official context to share across the team</li><li>Solves the problem in ways that make sense for their preferred workflow (sending us their metadata as a code file (yaml or json) via GitHub action that manages everything else. Minimizing the burden on them to learn a complex system, while delivering benefits for themselves and the business</li></ul><p>Understanding software engineers’ unique incentives and pain points is critical to driving collaboration between source system owners and downstream data consumers.</p><p><em>Secondary data usage may not be their focus, but we can drive engagement by reducing barriers to entry and ensuring their efforts get visibility with leadership as critical enablers of analytics and ML development</em></p><h4>2. Analysts: Balancing Speed and Precision</h4><figure><img alt="Infographic showing analysts balancing speed and precision, their blind spots, and curation strategies." src="https://cdn-images-1.medium.com/max/1024/0*RevJLdngruMvpzcL.png" /><figcaption>Archetype 2: Analysts and Data Scientists | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Analysts (and data scientists) are the <a href="https://dataprodmgmt.substack.com/p/what-analysts-really-need-from-your">core users of the data platform</a>. They are the final mile that takes all the data work we’ve done and turns it into insights and AI models that have real business impact.</p><p>For these users, the key to understanding them is that they are <em>incentivized to solve business problems quickly, but with precision</em>. The business needs their deliverables quickly, and with their focus on precision, they will do whatever wrangling is needed in order to build what the business is asking for and hit a timeline.</p><p>They are the rare user who has a foot in both worlds: deeply engaged with the business and its problems, with a strong technical understanding of what it takes to build well with data.</p><p>Analysts value the puzzle of finding the <em>right</em> data to bring to the problem and finding the <em>right</em> model, visualization, metric, forecast, etc. They enjoy the process of exploring and uncovering value for the business and then returning to the business to share those assets to guide decisions, power products, and spark critical conversations.</p><p><strong><em>They do not want to wait on the comparatively slow data platform team to build them pipelines</em></strong><em>, though if you have a curated, well-documented asset ready now, they are often excited to take the shortcut (if they can trust it).</em></p><h4>The common misunderstanding</h4><p>Data platform PMs sometimes don’t realize that data analysts often have no idea what the downstream implications are of their decisions. They don’t realize how the actions of many analysts compound to generate warehouse clutter, out-of-control query costs, and challenges with managing assets over their full lifecycle.</p><p>They are deep and narrow, focused on the problems within their remit and not thinking about the health of the ecosystem overall.</p><blockquote><em>💡 Platform PMs think analysts should wait for “perfect golden data products” or that analysts don’t care about data quality.</em></blockquote><p>Wrong on both counts. Analysts need speed, BUT they also care deeply about precision and quality. They may also feel threatened by self-service narratives that position their work as “almost automated.”</p><p>Particularly as AI demos show rapid-fire analysis and people declare the death of dashboards. Meanwhile, skilled analysts know that 80% of the work is just getting that dataset ready and knowing enough context to use it well, and we are far from being able to automate that aspect of their work.</p><h4>The design implication</h4><p>Plan for speed &amp; scale. Acknowledge that analysts may need to curate just-in-time assets, AND they also need to realize that those curations need to be managed to keep the ecosystem healthy.</p><p>Build a platform that supports <a href="https://dataprodmgmt.substack.com/p/the-curation-spectrum">the curation spectrum</a>: quick access for exploratory work PLUS a pathway to sustainable, scalable solutions (semantic layers, metrics views). Don’t force them to choose between speed and governance. And help them see career evolution, not job elimination.</p><p><em>Note: Data Scientists share many of the same incentives and needs as analysts — requiring speed, precision, and the ability to explore data iteratively. However, they have unique platform requirements around stability, pipeline reliability, and model deployment infrastructure that warrant additional consideration. More </em><a href="https://dataprodmgmt.substack.com/p/beyond-the-hype-how-data-platforms"><em>here</em></a><em>.</em></p><h4>3. Leadership: Proving ROI and Managing Risk</h4><figure><img alt="Infographic detailing leadership incentives around ROI and risk, key pain points, and strategic alignment advice." src="https://cdn-images-1.medium.com/max/1024/0*Ucp2bfxHhNkKvvam.png" /><figcaption>Archetype 3: Leadership | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Leaders are optimizing for fundamentally different outcomes than individual contributors or middle managers. They’re thinking about margin, growth, and the long-term viability of the business. They answer to boards, investors, and their own bosses. To influence leadership, you need to understand what they’re being measured on and what keeps them up at night.</p><h4>What are they worried about</h4><p>Leaders face constant pressure around hard decisions: org design, layoffs, budget allocation, and strategic prioritization. They want to be data-driven, but they don’t understand the complexity.</p><p>In particular, right now they have a lot of uncertainty around AI — are they investing in the right areas, are we getting ROI, do we have the right talent for this next phase, when will our industry and market transform, and how?</p><p>In short, they need information and confidence, and in particular, they need to know that when it comes to data &amp; AI, the business is moving in the right direction and allocating our scarce resources where needed. We need to provide reassurance that the data team can be:</p><ul><li><strong>An</strong> <strong>essential driver of innovation and operations</strong>, not just a service desk that builds nice-to-have reports</li><li>A <strong>strategic partner</strong> that helps leadership have the insights they need while finding the right opportunities to leverage data &amp; AI to help the business and our customers</li></ul><h4>The critical misunderstanding</h4><p>Data platform PMs often think too narrowly. They think it’s enough to say, “<em>I’m delivering what your team asked for.</em>” What they really need to think about is:</p><ul><li>Am I <strong>adding strategic value</strong> to leaders in my org?</li><li>Am I <strong>making sure their most important initiatives are getting their data needs met</strong> — not just responding to requests, but <strong>proactively helping identify opportunities</strong></li><li>Do they <strong>understand the full scope of what our team delivers</strong> for the business so that the timing of their request makes sense in the context of our overall capacity and demand on our team’s resources?</li></ul><h4>The design implication</h4><p><strong>Reserve capacity for comms and relationship building.</strong> Leadership doesn’t understand your world — you have to build that narrative. You have to connect platform work to business outcomes they care about (margin, growth, competitive advantage).</p><p>Communications from the data platform team are critical to the success of the team and your business. Make <a href="https://dataprodmgmt.substack.com/p/go-to-market-for-internal-platforms">optimizing those communication channels</a> a priority.</p><h4>Business Stakeholders: To Each Their Own</h4><figure><img alt="Diagram breaking down departmental priorities across Sales, Marketing, Finance, and Product." src="https://cdn-images-1.medium.com/max/1024/0*hVtWmXSRAHcg0p89.png" /><figcaption>Archetype 4: Business Stakeholders | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Finally, general business stakeholders, though this isn’t really one archetype. The whole point is that each one is different, and you need to spend the time to build the relationship and deeply understand their pain points. What is each stakeholder optimizing for?</p><ul><li>For sales, it’s all about closing deals and optimizing revenue.</li><li>For marketing, it’s about strategic growth levers and making sure we’re growing top of funnel and building brand presence.</li><li>For the product, it’s about making sure we’re shipping what our users need and understanding where we are getting traction and what’s not quite hitting the mark.</li><li>For finance, it’s about a multi-lens view of the business and the overall health, and making sure that those numbers are absolutely precise and can be trusted. And so on and so on for every different department.</li></ul><p>You have to do the work to deeply understand your stakeholder’s department level strategy, their OKRs and KPIs, their foundational analytics gaps, and the opportunities AI brings and whether they are tackling them.</p><p><em>Think of yourself as a mini-data strategy consultant for each department, partnering with them to grow their vertical.</em></p><h4>The critical misunderstanding</h4><p>Data Platform PMs limit their impact when they treat business stakeholders as a homogeneous group or just respond to inbound requests. Wrong. Every meeting is a consulting engagement: get up to speed fast, find opportunities, solve together. Hear requests but also help them think bigger about data &amp; AI’s role in their department.</p><h4>The design implication</h4><p>When you are staffing your data platform team and have multiple PMs, it may be tempting to align them to different platform features. And maybe some of your PMs can do that (at my last company, we had one PM over the AI platform feature set, for example), but when it comes to data products and engagement with the business, it is beneficial to <strong>align PMs to business departments</strong> for lasting relationships and deep domain knowledge.</p><p>You can’t be impactful without knowing them deeply. This isn’t scalable through templates and ticket queues. It requires dedicated relationship investment.</p><h3>Influence Without Authority</h3><p>Remember the opening premise: game theory assumes people act rationally given their incentives and constraints. The problem was never that your stakeholders were difficult or irrational. It was that you couldn’t see the games they were playing.</p><p>Once you have your incentive map, everything shifts. You stop being surprised when engineers resist governance or analysts bypass your platform. You understand why. And more importantly, you know how to design solutions that work with their incentives instead of against them.</p><p><em>Your core job as a product manager is to “</em><strong><em>influence without authority</em></strong><em>” and you can only do that if you empathize deeply with the humans you are working with.</em></p><p>The incentive map gives you that empathy at scale, helping you see patterns across individuals, anticipate resistance before it happens, and find the alignment points that let everyone succeed.</p><p>The map is never complete. It evolves as people change roles, structures shift, and strategies pivot. Your job is to keep updating it, stay curious about what motivates the people around you, and be the bridge-builder who sees all sides. That’s how platform PMs turn organizational complexity from an obstacle into an advantage.</p><h4>MD101 Connect ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><p><em>From MD101</em> <em>team</em> 🧡</p><h4><strong>AI Governance Benchmark 2026</strong></h4><p>60% of companies scale AI, yet only 4% possess governance mature <strong>enough to manage the risk.</strong> Discover how industry leaders bridge the gap to capture 3.6x higher returns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*UegPnmB0Nhm3Sgpu.png" /><figcaption><a href="https://www.moderndata101.com/reports/ai-governance-benchmark-enterprise-readiness">Download full report</a></figcaption></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d9f1b71ae95" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[5 Ways AI Agents Will Transform Data Management & Analytics]]></title>
            <link>https://medium.com/@community_md101/5-ways-ai-agents-will-transform-data-management-analytics-dc8e6df8669b?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc8e6df8669b</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Thu, 22 Oct 2026 14:26:04 GMT</pubDate>
            <atom:updated>2026-10-22T14:26:04.041Z</atom:updated>
            <content:encoded><![CDATA[<h4>From brittle pipelines to self-healing architectures, the 2026 and 2027 blueprint for building agent-ready data products.</h4><p>Authors: <a href="https://www.linkedin.com/in/akshay-chame-b43bb8209/">Akshay Chame</a>, <a href="https://www.linkedin.com/in/ritwika-c-18104a1b3/">Ritwika Chowdhury</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/5-ways-ai-agents-will-transform-data-management-analytics"><strong><em>Modern Data 101</em></strong></a><strong><em> Website.</em></strong></p><h4>TL;DR</h4><ul><li><strong>The Paradigm Shift:</strong> Traditional dashboards are dying. <strong>AI agents</strong> are transforming <strong>enterprise data management</strong> from a passive reporting layer into an autonomous, self-healing, and action-oriented ecosystem.</li><li><strong>Massive Adoption:</strong> Agentic AI is no longer abstract; Gartner projects that <a href="https://www.gartner.com/en/newsroom/press-releases/2025-08-05-gartner-hype-cycle-identifies-top-ai-innovations-in-2025"><strong>33% of enterprise software applications will feature agentic AI by 2028</strong></a>, a massive leap from under 1% in 2024.</li><li><strong>The 5 Pillars of Transformation:</strong> AI agents are replacing brittle manual engineering, automating <strong>data management</strong> and governance at scale, eliminating decision latency through real-time analytics, truly democratising data access using natural language, and deploying coordinated multi-agent architectures for end-to-end data discovery.</li><li><strong>The Real Bottleneck:</strong> While nearly two-thirds of enterprises are experimenting with <strong>ai for data management</strong>, <a href="https://www.mckinsey.com/capabilities/mckinsey-technology/our-insights/building-the-foundations-for-agentic-ai-at-scale">fewer than 10% have successfully scaled</a>. The reason? A severe lack of agent-ready, governed data products.</li></ul><blockquote><strong>📕 Related read:</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/7-ways-ai-helps-improve-data-quality"><strong>60% of AI Projects Fail: 7 Ways to Fix Data Quality Before It Kills Your Models</strong></a></blockquote><p>The era of passive dashboards and reactive BI tools is ending. AI agents, autonomous systems that perceive, reason, and act without constant human prompting, are the most significant force reshaping enterprise data infrastructure right now. <a href="https://www.gartner.com/en/newsroom/press-releases/2025-08-05-gartner-hype-cycle-identifies-top-ai-innovations-in-2025">Gartner named agentic AI its top strategic technology trend for 2025</a>, and the data industry is feeling it most acutely. By 2028, Gartner projects 33% of enterprise software applications will include agentic AI, up from less than 1% in 2024.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ihffmsSUA8dL_i_j" /><figcaption><em>Agentic AI adoption is expected to accelerate rapidly, reshaping how enterprises build and operate intelligent systems | Source: Authors</em></figcaption></figure><p>But <a href="https://www.moderndata101.com/blogs/lean-ai-building-a-scalable-data-platform-for-enterprise-ai-roi">most of the conversation around AI agents is still too abstract</a>. Here is what is concretely changing across data management and analytics, and why it matters now.</p><h4>1. Self-Healing Pipelines: How AI Data Management Eliminates Manual Engineering</h4><p>Traditional data pipelines are brittle. They break silently, require specialist intervention, and scale poorly under volume. AI agents change the operating model entirely.</p><p>Agentic systems now monitor pipeline health in real time, detect anomalies, trace failures back to root causes, and apply remediation, all without a human in the loop. The<a href="https://www.moderndata101.com/blogs/the-power-combo-of-ai-agents-and-the-modular-data-stack-ai-that-reasonsthe-power-combo-of-ai-agents-and-the-modular-data-stack-ai-that-reasons"> modular Data Developer Platform architecture enables purpose-driven agents</a> to be layered over existing infrastructure, handling compute, orchestration, and debugging as discrete, composable building blocks. This is adaptive intelligence embedded into the pipeline itself.</p><p>The practical payoff: data engineering teams spend less time firefighting and more time building. For organisations running at scale on data management platforms or modern data operating systems, this shift is already live.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/ai-data-management-vs-traditional-data-management-which-one-saves-time"><strong>AI vs. Traditional Data Management: Which One Actually Saves Time?</strong></a></blockquote><h4>2. Continuous Governance: Using AI Agents to Operationalise Data Quality at Scale</h4><p>Poor data quality costs enterprises <a href="https://www.gartner.com/en/data-analytics/topics/data-quality">an average of $12.9 million per year</a>. Manual governance, human stewards reviewing lineage, updating access controls, profiling datasets, cannot keep pace with modern data volumes or complexity.</p><p>AI agents solve this through continuous, contextual monitoring. They profile datasets as data moves through pipelines, flag anomalies in real time, classify data by sensitivity, apply access controls automatically, and maintain audit trails that satisfy regulatory requirements. McKinsey’s foundational research confirms the bottleneck: <a href="https://www.mckinsey.com/capabilities/mckinsey-technology/our-insights/building-the-foundations-for-agentic-ai-at-scale">eight in ten companies cite data limitations as a roadblock to scaling agentic AI</a>, which means governance is the unlock, not an afterthought.</p><figure><img alt="Diagram contrasting single-path human RBAC access with broken RBAC locks under multi-database agentic access." src="https://cdn-images-1.medium.com/max/1024/0*tPY-SDFVHVZ_e7rU" /><figcaption><em>The need of enterprises for AI-ready governance that supports autonomous data access, policies, and trust | Source: Authors</em></figcaption></figure><p>As <a href="https://www.moderndata101.com/blogs/governance-for-ai-agents-with-data-developer-platforms">AI agents become active consumers of enterprise data</a>, querying systems, invoking tools, and making decisions, governance frameworks designed for human users break down. The data contracts, RBAC/ABAC rules, and lineage tracking that govern human access need to be rebuilt for agentic actors. <a href="https://datadeveloperplatform.org/">The Data Developer Platform</a> provides a standards-based approach to exactly this problem.</p><h4>3. Autonomous Decision Intelligence: Slashing Analytics Latency by 48%</h4><p>The dashboard is an artifact of a world where humans were the last mile of analysis. AI agents eliminate that bottleneck.</p><p>Rather than surfacing data for human interpretation, agentic analytics systems continuously monitor information streams, identify patterns, assess significance, and trigger actions, inside CRMs, ERPs, and operational systems, without waiting for a human to run a query. Gartner’s 2025 analytics data shows organi<strong>s</strong>ations using autonomous agentic pipelines achieve a 48% reduction in decision latency compared to batch analytics environments. McKinsey research similarly reports that businesses deploying autonomous analytics agents in 2025 saw their insight-to-action cycle accelerate by 32%, with operational decisions becoming 21% more accurate.</p><p><a href="https://moderndata101.substack.com/p/data-products-the-essential-context">AI agents walking into a data warehouse have access to the tables but not the accumulated institutional context a human analyst carries</a>. The solution is not better models; it is better-structured data products with embedded metadata, lineage, and semantic meaning.</p><h4>4. AI Agents Democratise Data Access Across the Enterprise</h4><p>One of the most durable failures of the modern data stack has been democratisation. Despite years of investment in self-service BI, most business users still can’t get answers from data without analyst support.</p><p>AI agents change the access model. Natural language interfaces allow non-technical users to query data warehouses, validate assumptions, and generate reports through conversational prompts. The AI agent handles schema navigation, query construction, and result interpretation, translating intent into output without SQL expertise. <a href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai">McKinsey’s 2025 AI state-of-the-market report notes</a> that AI agent use is now most commonly deployed in IT and knowledge management, with deep research and query automation leading as early use cases across industries.</p><figure><img alt="Diagram showing an AI agent translating a natural language query into a generated report." src="https://cdn-images-1.medium.com/max/1024/0*-RFiT8V1Kr0KzHoG" /><figcaption><em>Agentic AI enables users to turn business questions into actionable insights without writing SQL, making enterprise analytics more accessible | Source: Authors</em></figcaption></figure><p>The implications for data team structure are significant. AI agents are not assistants that wait for a question; they are <a href="https://moderndata101.substack.com/p/new-data-roles-to-prep-for">autonomous consumers of enterprise data</a>. This fundamentally changes what data teams need to build and who they serve. The priority shifts from building pipelines for human analysts toward building governed, agent-ready data products.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/ai-data-management-how-to-make-data-ai-ready"><strong>AI Data Management: What It Actually Takes to Trust an AI Agent</strong></a></blockquote><h4>5. Multi-Agent Systems Enable End-to-End Data Discovery and Lineage</h4><p>The most advanced organisations are moving beyond single-agent deployments toward coordinated multi-agent architectures, systems where specialised agents handle discrete tasks (access control, semantic enrichment, relevance ranking, anomaly flagging) and share state across a broader workflow.</p><p>Think of a framework/ architecture where: an <a href="https://moderndata101.substack.com/p/new-data-roles-to-prep-for">access-control agent reads RBAC/ABAC rules</a> and produces a filtered view of permissible assets; a semantic enrichment agent layers business context; a ranking agent surfaces the most relevant assets for a given query; a feedback loop trains the system on actual usage patterns, turning agent recommendations into organisational memory. Over time, the system learns which datasets are trusted, which lead to incidents, and which are repeatedly relevant for which use cases.</p><p>This is how AI transforms data discovery from a search problem into an adaptive intelligence layer. And it only works with the kind of foundational infrastructure with metadata-first design, data contracts, and governed APIs that data developer platforms are built around.</p><h3>How to Move from Agent Experimentation to Scaled Value in Upcoming 2027</h3><p>The biggest confusion today might be about which agents to pick and where to place them in the stack, or rather, how to implement it altogether.</p><p>McKinsey reports nearly two-thirds of enterprises have experimented with agents, but fewer than <a href="https://www.mckinsey.com/capabilities/mckinsey-technology/our-insights/building-the-foundations-for-agentic-ai-at-scale">10% have scaled them to deliver value</a>. The gap here is data readiness.</p><p>Before deploying AI agents at scale, organisations must:</p><ul><li>Treat data products as first-class infrastructure with governance, versioning, and semantic context built in</li><li>Rebuild governance frameworks for agentic consumers, not just human users</li><li>Invest in metadata and lineage as the connective tissue that makes agents reliable</li><li>Pilot in high-value, bounded domains before scaling to enterprise-wide orchestration</li></ul><p>The organisations that close this gap first will have a structural analytics advantage that compounds, every decision an agent makes right improves the next one.</p><h3>FAQs</h3><h4>Q1. How are AI agents redefining data analysis?</h4><p>AI agents automate many parts of the analytics workflow, from discovering and preparing data to generating insights and recommending actions. Instead of manually building queries and reports, users can interact with data through natural language and receive faster, context-aware insights.</p><h4>Q2. What are the top 5 AI trends in data management in 2026?</h4><ol><li><strong>AI Agents for Data Operations</strong>: Agents automate data discovery, preparation, quality monitoring, and governance tasks.</li><li><strong>Semantic Layers &amp; AI Ontologies</strong>: Organisations are building business context layers to make AI outputs more accurate and trustworthy.</li><li><strong>Data Products for AI</strong>: Curated, reusable data products are becoming the preferred way to supply data to AI systems.</li><li><strong>Automated Data Quality Management</strong>: AI continuously detects, explains, and helps resolve data quality issues.</li><li><strong>Self-Service Data Platforms</strong>: AI-powered platforms enable users to find, access, and use data without relying heavily on central teams.</li></ol><h4>Q3. What is the role of data integration in AI data management?</h4><p>Data integration connects data across systems, creating a unified and consistent foundation for AI. It ensures AI models and agents can access complete, accurate, and timely data to generate reliable insights and decisions.</p><h4>MD101 Connect ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><p><em>From MD101</em> <em>team</em> 🧡</p><h4>AI Governance Benchmark 2026</h4><p>60% of companies scale AI, yet only 4% possess governance mature <strong>enough to manage the risk.</strong> Discover how industry leaders bridge the gap to capture 3.6x higher returns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*F0ijZyW-Q_urzgOK.png" /><figcaption><a href="https://www.moderndata101.com/reports/ai-governance-benchmark-enterprise-readiness">Download full report</a></figcaption></figure><p><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc8e6df8669b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reclaiming Freedom: Who Holds Veto Over Your Data Stack]]></title>
            <link>https://medium.com/@community_md101/reclaiming-freedom-who-holds-veto-over-your-data-stack-601e384de7d0?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/601e384de7d0</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[operating-systems]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 14:09:15 GMT</pubDate>
            <atom:updated>2026-10-15T14:09:15.087Z</atom:updated>
            <content:encoded><![CDATA[<h4>What Stallman understood about control that data tools hope you never ask.</h4><p>Author: <a href="https://www.linkedin.com/in/anismiles/">Animesh Kumar</a>, <a href="https://www.linkedin.com/in/samadritaghosh/">Samadrita Ghosh</a>, and <a href="https://www.linkedin.com/in/travis-w-thompson/">Travis Thompson</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://moderndata101.substack.com/p/reclaiming-freedom-who-holds-veto"><strong><em>Modern Data 101</em></strong></a><strong><em> Substack.</em></strong></p><h3>Freedom Has Always Been the Point</h3><p><a href="https://stallman.org/">Richard Stallman, who built GNU</a> (GNU <em>Not Unix</em> — a recursive acronym), came up with GNU to solve a more cultural problem (that later became a huge technical debt). His problem wasn’t with the software but the lock: the mechanism that let someone else decide who could use it, modify it, or build on it.</p><p><em>This is the first principle worth holding: </em><strong><em>control over a tool is control over everyone who depends on that tool.</em></strong><em> It doesn’t matter how good the tool is. Goodness and freedom are orthogonal. A gilded cage is still a cage.</em></p><p>Unix makes a great analogy for enterprise data platforms today. They work. Genuinely. That’s precisely what makes the control so dangerous. The better they are, the deeper the dependency, and the deeper the dependency, the less theoretical the risk becomes.</p><p>Stallman understood that software freedom isn’t about ideology, but more about <strong>who has veto power over your work.</strong> When a vendor can change pricing, deprecate an API, or simply cease to exist, that veto lives with them. You are building on borrowed ground.</p><p>The question was never “<em>does this platform perform well?</em>” The question is: <strong>when their interests and yours diverge, and they always eventually do, who wins?</strong></p><h4>The Argument for Freedom: How Linux Happened. Ergo, Road to the High-Power Operating Systems</h4><p><em>If you can’t see the code, you can’t trust the system. If you can’t change it, you’re not free.</em></p><p>AT&amp;T’s Unix was genuinely powerful. But by the 1980s, AT&amp;T locked it down. Stallman’s response was not to build something better than Unix. It was to <strong>rebuild Unix’s freedoms on top of Unix’s own </strong><a href="https://datadeveloperplatform.org/architecture/"><strong>architecture</strong>: <strong>three layers, from scratch</strong>, so no one could lock the door again</a>:</p><ul><li><strong>Utilities:</strong> the everyday tools and commands</li><li><strong>Shell</strong>: the interface between human and machine</li><li><strong>Kernel</strong>: the core that talks to hardware and manages memory</li></ul><blockquote><em>There is no one company that could have imagined all the different cases where computers are used these days, and Linux, </em><strong><em>thanks to its </em></strong><a href="https://moderndata101.substack.com/p/thriving-in-the-agentic-era-a-case"><strong><em>adaptability where everyone can just tweak it in little ways</em></strong></a><strong><em> to make it fit their use case, now covers all the use cases. ~ </em></strong><a href="https://www.linkedin.com/in/karsten-nohl/"><strong><em>Karsten Nohl | Hacker, Security Research Labs</em></strong></a></blockquote><h4>Linux spread not because it was marketed, but because <a href="https://moderndata101.substack.com/p/rise-of-the-context-architecture">adaptability without permission</a> is an irresistible force for builders.</h4><figure><img alt="Diagram showing the three GNU/Linux layers: Utilities, Shell, and Kernel, alongside a quote on adaptability." src="https://cdn-images-1.medium.com/max/1024/0*DWt-E1p3cE6Bm3VT.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><p>The public sector, in particular, adopted Linux precisely for this reason:</p><blockquote><em>You can assume that </em><strong><em>Linux is pretty much used in anything of high-security need</em></strong><em>… because usually there’s secrecy involved in building, let’s say, a new weapon system… You don’t want to involve more people than absolutely necessary. ~ </em><a href="https://www.youtube.com/watch?v=aoag03mSuXQ"><em>Karsten Nohl (Source)</em></a></blockquote><p>Sovereignty. Mouldability. No vendor in the room where the iron is cast.</p><h3>The Data Stack Holds a Mirror to the Unix Pattern</h3><p>Today’s public sector data teams run on powerful platforms which are capable systems. The problem was not always the capability or the technology, as we saw with Unix. This problem is the same one Stallman walked away from in 1980.</p><p>History repeats itself, more so in structures randomly dispersed across the world, even in fields that seem probabilistically unrelated.</p><figure><img alt="Diagram contrasting 1980s proprietary Unix with modern enterprise data platforms locked behind vendor walls." src="https://cdn-images-1.medium.com/max/1024/0*Yo6ItuixZjLWCdxl.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><ul><li>Data models are locked inside vendor schemas</li><li>Pipelines are built on proprietary orchestration that doesn’t leave with you</li><li>Governance rules are configured inside platforms you don’t control</li><li>Exit means rewriting everything</li></ul><p>Stallman, in his own words, wondering if he’d fit into the new structure of vendor-controlled environments,</p><blockquote><em>I realized that that way I could have fun coding and I could make money. But at the end, I’d have to look back at my career and say, “</em>I have spent my life building walls to divide people and I would’ve been ashamed of my life.<em>”</em></blockquote><p>The walls in the data world are called vendor lock-in.</p><figure><img alt="Four-panel diagram depicting vendor lock-in through locked schemas, trapped pipelines, captured governance, and high exit costs." src="https://cdn-images-1.medium.com/max/1024/0*oSUfLwev1g6NVihg.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><h4>An Operating System for Data is the “Not Unix” revolution for the Data Layer</h4><p><em>You don’t need to replace the foundation. You need to free what’s built on top of it.</em></p><p>GNU didn’t replace hardware, and Linux didn’t replace computing. They freed the operating layer, the layer that sits between raw infrastructure and human use, from proprietary control. Data systems MUST learn from software struggles. Inherit the best, skip some of the struggles.</p><figure><img alt="Diagram of open vault doors revealing an open data layer operating environment between infrastructure and human use." src="https://cdn-images-1.medium.com/max/1024/0*kXcUlTlJ24KrnVJ8.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><p>An Operating System for data does the same thing for data teams: freeing the <strong>data operating layer</strong> by rebuilding its equivalent layers from the ground up: Kernel, Shell, Utility Apps</p><figure><img alt="Architecture diagram mapping Data Developer Platform layers from Core Apps down to Cloud Kernel components." src="https://cdn-images-1.medium.com/max/1024/0*dhL9miygWp5Iv8C_.png" /><figcaption>DDP Architecture (View 2). DataOS is an implementation of the Data Developer Platform (DDP) Infrastructure Standard } <a href="https://datadeveloperplatform.org/architecture/">Source</a></figcaption></figure><p>A data operating system gives data teams the ability to <strong>build and run their own data operating environment</strong>.</p><p>The new freedoms include (but not limited to):</p><ul><li>Define own data products, owned by org/team</li><li>Govern data with own policies, <strong>not</strong> vendor defaults</li><li>Swap underlying infrastructure without rebuilding from zero</li><li>Share data across departments without surrendering control to a platform intermediary</li></ul><p>This is a projection of Stallman’s four freedoms (run, study, change, share) applied to the data layer.</p><figure><img alt="Diagram outlining four operational freedoms for enterprise data: Define, Govern, Swap, and Share." src="https://cdn-images-1.medium.com/max/1024/0*55_AduXOpn5fCXE4.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><h4>The XZ Problem: Protecting the Freedom Won</h4><p><em>Winning freedom is not the same as securing it.</em></p><p>GNU/Linux solved the Unix problem. But the <a href="https://en.wikipedia.org/wiki/XZ_Utils_backdoor">XZ Utils backdoor</a> reveals what open systems can inherit if left ungoverned:</p><blockquote><em>The entire operating system rested on a single part, maintained by a single person, and by compromising that one part, they could infect almost any server on the internet.</em> <em>(</em><a href="https://www.youtube.com/watch?v=aoag03mSuXQ"><em>Source</em></a><em>)</em></blockquote><p>Lasse Collin maintained XZ for 20 years, unpaid, burning out. Jia Tan, likely a nation-state actor, weaponised that exhaustion through a two-and-a-half-year social engineering campaign. The attack surface wasn’t the code. It was the human dependency underneath it.</p><figure><img alt="Screenshot of a forum post quoting Lasse Collin regarding open-source maintainer burnout." src="https://cdn-images-1.medium.com/max/1024/0*E9fmGsbxPFouKAev.png" /><figcaption>Lasse Collin on his “ability to care” | <a href="https://news.ycombinator.com/item?id=39872887">Source</a></figcaption></figure><p><em>Anything from spying, to ransom, to taking down entire countries: you could have done it with this backdoor. ~ Karsten Nohl</em></p><p>For domains such as the public sector and defence, this scenario is not hypothetical. It is the primary threat model.</p><blockquote>…I’m worried that we didn’t find other backdoors. There are state-sponsored parts of either governments, militaries, or even private contractors working for states that are all preparing for the next cyber escalation. ~Nohl</blockquote><figure><img alt="Diagram showing internet infrastructure resting on a single cracking pillar representing human dependency." src="https://cdn-images-1.medium.com/max/1024/0*-y8rLlCCvCqABeeg.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><p>A Data Operating System doesn’t go back toward closed source, but builds freedom warily with <a href="https://moderndata101.substack.com/p/role-of-interoperability-in-end-to"><strong>structural guardrails that make single-point human dependency architecturally impossible</strong></a><strong>:</strong></p><ul><li>No single maintainer holds unguarded keys to core modules</li><li>Contribution pipelines have accountability structures, not just open commit access</li><li>The system is hybrid: open enough to be moulded, governed enough to prevent fatalities</li></ul><p>The operating system needs to support the people: the data teams building on top of it, by ensuring that the platform itself is not one exhausted volunteer away from compromise.</p><figure><img alt="Diagram showing an open-core network protected by a governed structural shield and access gateways." src="https://cdn-images-1.medium.com/max/1024/0*YCCrlhpKxk97H8mV.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><h4><strong>Why High-Trust Operators Come First to Such Discussions</strong></h4><p>Sovereign institutions (governments, defence agencies, public health bodies) do not have customers. They have citizens. That distinction changes everything about how they must treat data. Every decision is subject to oversight.</p><p>This is how most high-trust operators chose Linux over Windows and proprietary Unix for reasons that are structural, not just technical:</p><ul><li>Citizens and oversight bodies must be able to interrogate decisions → <strong>requires auditability</strong></li><li>Agencies share data across jurisdictions without surrendering control → <strong>requires interoperability</strong></li><li>Policy changes require system changes without vendor negotiation → <strong>requires mouldability</strong></li></ul><figure><img alt="Diagram of interlocking gears and steel pillars representing Auditability, Interoperability, and Mouldability." src="https://cdn-images-1.medium.com/max/1024/0*Mw_C71WBPDIKqSx-.png" /><figcaption>Adapted from concepts shared by the Author(s) | Curated by Modern Data 101</figcaption></figure><p>The data layer has become the new operating layer, where the real decisions about who sees what, who controls what, and who can change what are made. Locking that layer inside a vendor is the 2026 equivalent of the 1980s in software.</p><p>What is desired and absolutely necessary for data teams is <strong>a foundation they actually own, with the ability to mould it without asking permission</strong>. And now, with the guardrails to protect it from the threats that come with that openness.</p><figure><img alt="Survey bar chart showing unified access, governance, and self-service as top converged platform features." src="https://cdn-images-1.medium.com/max/1024/0*QJOh93iQ7PmvV9XX.png" /><figcaption>Expectations from a unified/converged data platform | Source: <a href="https://www.moderndata101.com/modern-data-report-2026">Modern Data Report 2026</a></figcaption></figure><h4>MD101 Connect ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><p><em>From MD101</em> <em>team</em> 🧡</p><h4>AI Governance Benchmark 2026</h4><p>60% of companies scale AI, yet only 4% possess governance mature <strong>enough to manage the risk.</strong> Discover how industry leaders bridge the gap to capture 3.6x higher returns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z1tOoHFkhs2C3f-s.png" /><figcaption><a href="https://www.moderndata101.com/reports/ai-governance-benchmark-enterprise-readiness">Download full report</a></figcaption></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=601e384de7d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI vs. Traditional Data Management: Which One Actually Saves Time?]]></title>
            <link>https://medium.com/@community_md101/ai-vs-traditional-data-management-which-one-actually-saves-time-6a47585f569a?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a47585f569a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[data-strategy]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Fri, 09 Oct 2026 14:21:20 GMT</pubDate>
            <atom:updated>2026-10-09T14:21:20.481Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI redistributes data management efforts instead of eliminating them. Learn where automation delivers measurable time savings and why data readiness determines long-term AI ROI.</h4><p>Author: <a href="https://www.linkedin.com/in/linkaish/">Aishwarya Sharma</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/ai-data-management-vs-traditional-data-management-which-one-saves-time"><strong><em>Modern Data 101</em></strong></a><strong><em> Website.</em></strong></p><h3>TL;DR</h3><p>AI cuts real time out of data management, but only where the underlying foundation already exists. Where it doesn’t, AI just moves the delay upstream, into preparation and governance, instead of removing it.</p><p>This comparison breaks down where AI’s time savings compound and where they’re deferred, across ingestion, quality, governance, scalability, and decision speed, for <a href="https://www.moderndata101.com/blogs/data-leadership-extends-beyond-core-data-professionals">data leaders sequencing where to invest first</a>.</p><p>The “AI saves time” argument is technically true and practically incomplete. Ask any data engineering team that spent three months preparing data before a single model could run: AI relocates time cost in data management rather than eliminating it. The effort shifts upstream, from execution to preparation, from fixing errors to preventing them, from maintaining pipelines to governing the systems that feed them.</p><figure><img alt="Diagram explaining AI observability vs traditional monitoring, key capabilities, and governance stats." src="https://cdn-images-1.medium.com/max/1024/0*DbRupGeCqB8f4Pfl" /><figcaption><a href="https://www.moderndata101.com/blogs/lean-ai-building-a-scalable-data-platform-for-enterprise-ai-roi"><em>Fundamentals of Lean AI approach for improved business outcomes</em></a></figcaption></figure><p>AI-driven data management genuinely speeds things up, but it also moves where teams spend their time and attention. The bottleneck shifts from handling data manually to managing scale, oversight, and system complexity. Teams that succeed with AI are the ones who prepare for that shift early, rather than assuming automation alone fixes operational inefficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PN2-DfgcT_PN7EpZ" /><figcaption><em>Relocating effort from downstream execution to upstream preparation | Source: Author</em></figcaption></figure><h3>At a Glance: How AI and Traditional Data Management Compare</h3><p>Here’s how the two approaches stack up across the dimensions that matter most.</p><figure><img alt="Comparison table contrasting traditional and AI-driven data management across seven dimensions." src="https://cdn-images-1.medium.com/max/1024/0*oZv-j-QxNfWBOIwN" /></figure><p>The table tells the broad story; the sections below show where those savings are real and compounding, and where they’re just deferred.</p><h4>1. Data Ingestion: How Much Faster Is AI at Bringing Data In?</h4><figure><img alt="Diagram showing automated ingestion execution passing through a lock requiring human context." src="https://cdn-images-1.medium.com/max/1024/0*G1s7v41sQcAq-n8u" /><figcaption><em>AI compresses technical execution but still requires human context for trust | Source: Author</em></figcaption></figure><p>Traditional ingestion is slow: every new source needs custom setup, schema mapping, and validation, and connecting a single source can take weeks before anyone checks whether the data is even fit for purpose.</p><p>AI compresses the execution layer considerably.</p><ul><li>Schema inference reduces manual mapping from days to minutes,</li><li>Automated connectors reduce integration build time,</li><li>Deduplication and structuring happen during ingestion instead of happening afterwards.</li></ul><p><a href="https://www.moderndata101.com/blogs/data-pipelines-for-data-products">Standardised ingestion patterns built around data products</a> and multiple commodity tools have pushed source connection time down sharply.</p><p>What AI doesn’t compress is context. Ownership, lineage, and data meaning still require humans to define them before ingestion is reliable, and no amount of connector automation can resolve that on its own.</p><blockquote>AI cuts ingestion execution time substantially. It does not cut the thinking time that makes ingestion trustworthy.</blockquote><h4>2. Data Quality: Does AI Reduce the Cost of Bad Data?</h4><p>According to <a href="https://hbr.org/2016/09/bad-data-costs-the-u-s-3-trillion-per-year">IBM research (via Harvard Business Review),</a> poor <a href="https://www.moderndata101.com/glossary/data-quality">data quality</a> costs the US economy an estimated $3.1 trillion annually, a figure that reflects not catastrophic failures but the slow accumulation of undetected errors across pipelines nobody is watching closely enough.</p><p>Traditional quality systems fail in a specific way:</p><ul><li>A schema change in a source system silently breaks a downstream report</li><li>A field that was always populated starts arriving null without warning</li></ul><p>AI shifts quality management from reactive to continuous. Anomaly detection runs in real time, statistical drift gets flagged before it reaches outputs, and pattern-based validation catches deviations that rule-based systems would miss entirely.</p><blockquote><strong>📕 Related read:</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/top-10-data-quality-dimensions-and-how-unified-data-platforms-enable-them"><strong>Top 10 Data Quality Dimensions and How Unified Data Platforms Enable Them</strong></a></blockquote><p>The limitation is resolution: a human still has to decide whether a flagged anomaly is a <a href="https://www.moderndata101.com/blogs/how-to-build-reusable-features-for-machine-learning-ai-with-data-products">source problem, a pipeline issue, or a legitimate change</a>, and reusable, governed data products at the point of ingestion are what reduce how often that step is needed.</p><blockquote>AI finds problems faster but still can’t yet understand them.</blockquote><h4>3. Governance Overhead: Does Automation Actually Reduce Oversight Work?</h4><figure><img alt="Iceberg diagram showing AI automation lowering water levels to reveal invisible historical data debt." src="https://cdn-images-1.medium.com/max/1024/0*ih7qkbVJs8BjsE42" /><figcaption><em>Automating governance reveals the true scale of historical data debt | Source: Author</em></figcaption></figure><p>Traditional governance depends on human process: access reviews happen quarterly, lineage gets documented manually if at all, and compliance checks are point-in-time audits rather than continuous signals.</p><p>AI-driven governance changes the execution layer:</p><figure><img alt="able comparing traditional manual governance with automated AI-driven governance across four areas." src="https://cdn-images-1.medium.com/max/1024/0*aIFwiCHSvJcH5wjO" /></figure><p>The data governance market, which encompasses lineage tooling, reached <a href="https://data-pilot.com/blog/data-lineage-companies/">$3.91 billion in 2026 and is projected to grow to $9.62 billion by 2030</a>, driven by regulatory mandates and AI explainability requirements. But governance automation also surfaces debt that was previously invisible. Teams frequently report an initial spike in detected issues after adoption because the system is now looking properly for the first time.</p><p><a href="https://www.moderndata101.com/blogs/the-governance-framework-passing-through-the-trifecta-of-people-process-and-tech">Treating governance as a function of people, process, and technology working together</a> is the right lens here. Automating one layer without the others moves it somewhere less visible rather than reducing total overhead.</p><blockquote><strong>📕 Related read</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/5-ways-to-measure-ai-governance-success-through-metrics-in-the-era-of-agentic-ai"><strong>5 Ways To Measure AI Governance Success in the Era of Agentic AI</strong></a></blockquote><h4>4. Pipeline Scalability: Why Do Traditional Systems Break at Scale?</h4><p>Pipelines fail at scale because coordination overhead grows faster than a team’s capacity to manage it, not because of infrastructure limits.</p><ul><li>Dependencies multiply and become undocumented</li><li>Ownership of datasets becomes unclear</li><li>Metric definitions diverge across teams</li><li>Pipelines grow fragile; nobody is confident about what breaks if something changes</li></ul><p>AI improves scalability in two ways: automated discovery keeps the map of what exists current without manual curation, and AI-assisted orchestration maintains pipeline logic at a scale no human team can sustain across hundreds of sources.</p><p><a href="http://gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk">Gartner predicts</a> organisations will abandon 60% of AI projects that lack <a href="https://www.moderndata101.com/blogs/why-is-a-data-marketplace-critical-for-organisations">AI-ready data</a> through 2026, which is exactly the sequencing problem here: scalability depends on a foundation most enterprises haven’t finished building. Data products with documented ownership scale well under AI; automation layered onto ungoverned pipelines just propagates errors faster.</p><h4>5. Decision-Making Speed: Does AI Actually Deliver Faster Answers?</h4><p>In traditional environments, the lag between a business question and a trusted answer is measured in days or weeks, and by the time an answer arrives, the decision context has often shifted.</p><p>AI compresses that cycle at multiple points:</p><ul><li>Natural language interfaces remove the queue between business users and data engineers</li><li>Automated preparation and real-time pipelines mean data is fresh when it is needed</li><li><a href="https://www.moderndata101.com/blogs/operationalizing-data-product-delivery-in-the-data-ecosystem">Data products with consistent, documented interfaces</a> remove the discovery and trust-building steps that add days to every analysis cycle</li></ul><p>The real constraint is trust. Faster access only improves decision speed when the people consuming that data believe it is accurate. AI-generated outputs from poorly governed pipelines produce fast answers to wrong questions, and in most organisations, a few of those experiences are enough to erode confidence in the entire system. Speed without trust is a liability.</p><figure><img alt="Quadrant chart plotting time to answer against trust in data, highlighting the value of governed data products." src="https://cdn-images-1.medium.com/max/1024/0*CYIBxvWIMsQaSX4_" /><figcaption><em>Speed only compounds decision-making when the underlying data is trusted; without it, faster answers just erode confidence faster | Source: Author</em></figcaption></figure><h4>6. Why AI Time Savings Are Not Immediate</h4><p>Every “AI saves time” comparison has a starting point, and almost none account for the cost of reaching it. The gap between <a href="https://www.moderndata101.com/blogs/the-20-year-failure-how-ai-closes-the-gap-between-data-strategy-and-business-strategy">AI ambition and AI achievement</a> is almost always a data readiness gap, not a technology one: practitioners consistently report spending the majority of AI project time on preparation, cleaning, structuring, and validating data before modelling begins.</p><p>Teams treat that phase as an unfortunate delay before the real work starts, but it is the real work. Organisations that productise data ahead of time, with defined owners, documented interfaces, and <a href="https://www.moderndata101.com/blogs/how-to-choose-a-unified-data-platform">a platform layer that serves both</a> human analysts and AI agents from the same governed source, see their upfront investment compound instead of resetting with every new use case.</p><figure><img alt="Line graph contrasting upfront AI preparation effort with long-term compounding time savings vs rising traditional debt." src="https://cdn-images-1.medium.com/max/1024/0*7mwzfkM5_kF7Q_0P" /><figcaption><em>The ROI Curve: Visualising the 60–80% preparation phase in the project lifecycle | Source: Author</em></figcaption></figure><h3>Does AI Actually Save Time?</h3><figure><img alt="Diagram showing a glowing AI node built on stone blocks representing governance, lineage, ownership, and quality." src="https://cdn-images-1.medium.com/max/1024/0*Ryu5K4thAZhudu-z" /></figure><p>Across every dimension above, AI eventually outperforms traditional data management on time, wherever the foundation is right: governed data, documented lineage, clear ownership, and quality enforced at the source.</p><p>Where that foundation is weak, AI doesn’t save time; it accelerates the accumulation of debt, errors, audits, and pipeline breaks, all of which arrive sooner instead of later. Whether AI saves your organisation time comes down to how ready your data environment already is.</p><h3>FAQs</h3><h4>Q1: What are the core dimensions of data quality?</h4><p>A: Data quality is generally evaluated across several key dimensions: accuracy, completeness, consistency, timeliness, uniqueness, and validity.</p><h4>Q2: What is governance automation?</h4><p>A: Governance automation uses software and AI to enforce data policies, monitor compliance, and streamline access controls with minimal manual intervention.</p><h4>Q3: What does scalability refer to?</h4><p>A: Scalability refers to a system’s ability to handle increasing amounts of data, users, or workload by efficiently adding resources or optimising processes.</p><h4>Q4: Does saving time with AI just lead to more work?</h4><p>A: While AI can free up time by automating repetitive tasks, it may also shift focus to higher-value or more complex activities, not necessarily reducing overall workload.</p><h4>MD101 Support ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡‍</p><p><em>From MD101</em> <em>team</em> 🧡</p><p>A financial firm’s AI assistant ran for three weeks with zero errors and zero downtime, while quietly citing the wrong fund in every summary it wrote. This is the gap AI observability is built to catch, and this guide lays out the places enterprises need to watch for it: <a href="https://www.moderndata101.com/blogs/blogs-what-is-ai-observability-guide/">What Is AI Observability? Enterprise Stack &amp; Guide for 2026 | Modern Data Blog</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*MIrIgdYKwuKX16owJInJSA.jpeg" /></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a47585f569a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decentralizing an Organization: Beyond Data Mesh]]></title>
            <link>https://medium.com/@community_md101/decentralizing-an-organization-beyond-data-mesh-05990e239928?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/05990e239928</guid>
            <category><![CDATA[data-mesh]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[data-architecture]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Tue, 29 Sep 2026 14:50:58 GMT</pubDate>
            <atom:updated>2026-09-29T14:50:58.770Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://moderndata101.substack.com/s/experts-desk/?utm_source=substack&amp;utm_medium=menu">Expert’s Desk ⭐️</a></h4><h4>Architecting the decentralized enterprise, the symbiotic exchanges between data mesh and “service mesh,” and the role of unified data platforms</h4><p>Author: <a href="https://www.linkedin.com/in/vlad-radziuk-51318712a/">Vlad Radziuk</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://moderndata101.substack.com/p/decentralizing-an-organization-beyond-data-mesh"><strong><em>Modern Data 101</em></strong></a><strong><em> Substack.</em></strong></p><h4>About Our Contributing Expert</h4><h3><strong>Vlad Radziuk | Account CTO</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*piodMA1vXGOkKaF7.png" /></figure><p><a href="https://www.linkedin.com/in/vlad-radziuk-51318712a/">Vlad Radziuk</a> is a transformation leader and systems thinker working at <strong>Nordcloud, an IBM company</strong>. He operates at the intersection of modernisation, competence centres, and AI-driven enterprise foundations with a strong focus on ontologies, knowledge graphs, and structured modelling.</p><p>With an interdisciplinary background spanning software engineering, cloud-native architecture, and enterprise data strategy, Vlad has built and led innovation units and competence centres of up to 50 professionals. As <strong>Account CTO</strong> and <strong>former Practice Lead</strong>, he combines delivery leadership with long-term architectural vision, shaping AI-powered portfolios, establishing internal knowledge ecosystems, and guiding complex transformation programs.</p><p>Beyond corporate leadership, Vlad is an active contributor to the data and knowledge graph community. He writes and researches topics such as the business case for ontologies, cross-domain metadata management, digital sovereignty, and the economics of AI integration. His thinking reflects a systemic lens: technology only creates value when aligned with governance, culture, and long-term strategy. We’re thrilled to feature his unique insights on <a href="https://moderndata101.substack.com/">Modern Data 101</a>!</p><blockquote>We actively collaborate with data experts to bring the best resources to a <a href="https://www.moderndata101.com/">15,000+ strong community</a> of data leaders and practitioners. If you have something to share, reach out!<br>🫴🏻 Share your ideas and work: <a href="http://community@moderndata101.com/">community@moderndata101.com</a></blockquote><p>*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality &amp; relevance. We keep it information-first and do not support any promotions, paid or otherwise.</p><h3>Let’s Dive In</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wrs-jlKTWrAeSgyk.png" /></figure><p><em>“Data mesh doesn’t make much sense without a service mesh.”</em></p><p>What might sound like a bunch of buzzwords actually turned out to be a valuable lesson in systems thinking. I heard this statement during a discussion with a group of enterprise architects about how data mesh could support decentralizing their enterprise.</p><p>The challenges they described were familiar: bottlenecks, siloed data, and the all-too-common lament, <em>“We’re sitting on mountains of data but extracting little value from it.”</em></p><p>If you’re a data specialist, these issues likely sound familiar — they’re part of our daily reality.</p><p><strong>Then, the conversation took an unexpected turn.</strong><br>One architect emphasized the need for a holistic approach when transforming organizations toward a platform-based setup with <strong>federated governance and decentralized ownership</strong>. That’s when he dropped that line.</p><p>It was a powerful reminder for me: real-world systems are complex and interdependent. Inspired by this insight, I decided to outline a high-level playbook for developing systematic approaches to decentralized data management methodologies like data mesh.</p><p>This playbook will probably not be interesting for everyone, but if you’re looking to understand the broader context and make better decisions (or offer better advice) for your internal or external stakeholders, it’s worth stepping back and examining the bigger picture beyond just the data layer.</p><h3>Reasoning behind the decentralization of organizations</h3><p>Decentralization, as the core idea of the data mesh methodology, isn’t an end in itself, but a means to drive better data quality and unlock greater value from it. But the question that occupies me is:</p><blockquote><strong>How can organizations maximize benefits from decentralization?</strong></blockquote><p>Theory suggests that <a href="https://moderndata101.substack.com/i/161615567/the-second-generation-of-the-lakehouse-inherently-supports-decentralisation">decentralized organizations move faster</a>, motivate employees more effectively, and foster innovation. My own experience confirms this. When an organization’s culture is focused on delivering customer value, having multiple centers of power can lead to remarkable outcomes.</p><p>For example, when I led a competence center for application development (operating outside the company’s traditional hierarchy), I was able to create new narratives, uncover new opportunities, and engage employees in ways that a top-down approach simply wouldn’t allow.</p><p><strong>Another example that fascinated me recently</strong><br>Local leaders in a multinational consulting firm, operating in a smaller market, developed a strategy to sustain growth even during years with a slower macroeconomy. They began serving local startups with attractive pricing, which gave them indirect access to venture capital networks and diversified their pipeline.</p><p>Many of us have seen similar success stories, but let’s be honest: often, these are driven by the enthusiasm of individuals rather than institutionalized decentralization. What interests me is <strong>how organizations can systematically enable decentralization at scale. What platforms, cultures, and organizational designs are needed to make this work?</strong></p><p><em>As the </em><a href="https://en.wikipedia.org/wiki/Bauhaus"><em>Bauhaus movement</em></a><em> taught us in design and architecture, “</em><strong><em>form follows function</em></strong><em>.”</em></p><figure><img alt="Graphic outlining how organizational decentralisation requires operational change beyond database structure." src="https://cdn-images-1.medium.com/max/1024/0*oCR1Cev_7mbL8K90.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Decentralization isn’t just about restructuring. It’s about how an organization operates. If we want to decentralize an enterprise, there must be a clear intention behind it. The shift should be leveraged holistically, not just to improve data value, but to transform the way we work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*BGmix9us2TfKj2tB.jpeg" /><figcaption>By <a href="https://www.ezrastoller.com/">Ezra Stoller</a></figcaption></figure><p>In terms of platforms, <em>Netflix</em> can serve as a poster child with their data mesh architecture, knowledge graph, and Unified Data Architecture, which support various domains serving different parts of its platform, including films, series, games, live events, and ads. They regularly post articles about their architecture in their engineering blog (<a href="https://netflixtechblog.com/">Netflix TechBlog</a>), which I highly recommend reading.</p><figure><img alt="Diagram of Netflix’s unified data architecture supporting business domains like films, series, and ads." src="https://cdn-images-1.medium.com/max/1024/0*hNZtdu2G6LlaeY47.png" /><figcaption>Unified Data Infrastructure to Support Decentralisation Across Diverse Business Domains | Curated by Modern Data 101</figcaption></figure><p>Culture-wise, there is one company that made decentralization its source of competitive advantage and became a role model: a Chinese player in the IoT area called <em>Haier</em>.</p><p>They developed a philosophy called <a href="https://www.rendanheyi.com/home">RenDanHeYi</a>, after which they gave autonomy to their business units (e.g., a unit producing refrigerators, or other kinds of home appliances), which started to serve as “microenterprises”.</p><figure><img alt="Diagram showing autonomous microenterprises coordinated through ecosystems and three core rights." src="https://cdn-images-1.medium.com/max/1024/0*7q3NQPNglV9lC2bo.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Each of these internal companies got “three rights”:</p><ol><li>the right to make independent decisions,</li><li>the right to hire talent and</li><li>the right to distribute profit</li></ol><p>So they can quickly make decisions based on market changes, take responsibility for their own gains and losses, and share value in accordance with the agreement.</p><p>Instead of a centralized strategy, Haier’s approach is customer-driven: each team identifies real needs and builds solutions, and coordinates through ecosystems (e.g., “microenterprises” can form alliances to achieve bigger goals, and for higher-level strategy)</p><p>Haier still has some centralized institutions that fund high-risk, long-term bets. Although I could not find much about the internal technological setup of Haier, I still brought up this example to illustrate how important it is to think systematically when planning decentralization, and <strong>not to just focus on technology in general or data in particular.</strong></p><p>Still, this piece is primarily for data professionals, so let’s go back to what we know best and deduce other aspects from our shared vision of the bright data future.</p><h3>What does it actually mean for data to be FAIR?</h3><p><a href="https://www.go-fair.org/fair-principles/">FAIR data</a> (data that adheres to the principles of findability, accessibility, interoperability, and reusability) is a cornerstone of modern data management. For organizations aiming to implement <strong>FAIR, a </strong><a href="https://moderndata101.substack.com/i/147249824/importance-is-a-cultural-metric"><strong>data product culture</strong></a> (often supported by methodologies like data mesh) and explicit concept modeling (using ontologies) are essential.</p><figure><img alt="Diagram showing the shift from raw tables to a knowledge graph based on FAIR principles." src="https://cdn-images-1.medium.com/max/1024/0*mDxJnA8EXMBTX-t2.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>This shift moves the focus from tables to business concepts: entities with clear definitions, data instances, and relationships to other concepts.</p><p>The FAIR principles provide a clear North Star for how data should be managed to deliver business value. Let’s consider logistics as an example. FAIR data would align with master data (locations, products, partners) and reference data (units of measure, currency codes).</p><p>Each data product or data point would have a persistent identifier (PID) or URI, enabling actions like scanning a pallet’s PID to instantly retrieve its contents, origin, and destination from a shared database.</p><p>Data would be fully described using knowledge languages or ontologies. For instance, a controlled vocabulary would standardize status terms like “<em>Received</em>,” “<em>Picked</em>,” or “<em>Shipped</em>” (aligned with the SCOR model), while relationships like “<em>Product → StoredIn → WarehouseBin</em>” would be explicitly defined.</p><p><em>The result? Both humans and AI-powered solutions can act directly on datasets, without ambiguity or the need for further interpretation.</em></p><p>I believe FAIR principles offer a powerful lens for examining the holistic decentralization of organizations.</p><blockquote><em>🔖 </em><strong><em>Related Read(s)</em></strong></blockquote><blockquote><a href="https://moderndata101.substack.com/p/where-exactly-data-becomes-product">Where Exactly Data Becomes Product: Illustrated Guide to Data Products in Action</a></blockquote><blockquote><a href="https://moderndata101.substack.com/p/the-ontology-pipeline">How the Ontology Pipeline Powers Semantic Knowledge Systems</a></blockquote><p>Let’s imagine the shiny world of FAIR data: how would it look from the user’s perspective? Users can find the data they need by typing, for example, concept names, synonyms, or related concepts into the search bar of a data catalog.</p><p>What’s even more compelling is that data is aggregated by business concepts, allowing users to navigate from concept to concept, much like on Wikipedia, where you might start with “<em>black tea</em>” and end up reading about Einstein’s biography.</p><p>In this case, however, users could search for a product, explore its sold instances, track shipping status, and receive recommendations for similar products. This seamless experience is made possible by combining complete domain ownership of data with federated governance.</p><p>But what does this mean when we step outside the data perspective?</p><ol><li>Users traverse the enterprise network across domains. They don’t just access the knowledge space of different domains (e.g., by navigating a knowledge graph through a graphical interface, clicking on PIDs to move between concepts), but also can also consume data or access software functionality, provided they have the necessary permissions.</li><li>Domains own their assets, meaning they control business logic, access policies, and boundaries. In essence, they decide who can “visit” and interact with their data, much like setting the rules for guests in their own home.</li><li>Because domains fully own their business logic, the definitions of business concepts and processes can be traced: from knowledge bases and controlled vocabularies, through data, all the way to software.</li><li>Since domains provide their assets as products, they are responsible for the full lifecycle: maintenance, updates, monitoring, and beyond.</li></ol><p>As you can see, decentralization, driven by data mesh and ontology-based data management, has far-reaching implications for how IT operates. The FAIR criteria don’t just guide data management. They also highlight what’s needed for domains to thrive in a decentralized environment.</p><h3>Domain as a quantum of decentralized organizations</h3><p>Decentralization often means empowering domains, but what exactly is a domain? It could be a department, business unit, team, platform team, technical capability, or something else entirely. At its core, a domain is any logical boundary with clear ownership, responsibilities, and alignment to business goals.</p><p>In the context of data, a domain refers to an organizational unit that not only produces data but also owns it throughout its entire lifecycle, from creation to consumption. However, ownership doesn’t begin with the data itself. It starts earlier, with business logic and language, a principle well-captured by the domain-driven design (DDD) methodology.</p><p>Domains have boundaries, which define their unique knowledge models, language, and rules (something we often call business logic). These boundaries extend beyond abstract concepts and include tangible assets that support domain operations, such as software and data. As a result, domains exist across multiple layers:</p><ul><li><strong>Knowledge layer</strong>: The conceptual world of definitions, rules, and language.</li><li><strong>Operational layer</strong>: The runtime environment of applications, services, and microservices.</li><li><strong>Analytical layer</strong>: The world of data products and insights.</li></ul><p>To successfully decentralize an organization, all these layers must be addressed, with each playing a critical role in enabling autonomy, clarity, and alignment.</p><figure><img alt="Diagram illustrating a business domain spanning knowledge, runtime, and analytics layers." src="https://cdn-images-1.medium.com/max/960/0*ZFVkKfexw0be2M_d.jpeg" /><figcaption>Domain, the quantum of decentralization, exists across multiple layers of knowledge, operations, and analytics. | Source: Author</figcaption></figure><p>If we abstract away from data or software and focus instead on business capabilities, something interesting emerges: those capabilities remain consistent, whether viewed through an operational or analytical lens.</p><p>The language, definitions, and rules (in other words, the knowledge model) persist across all layers. This alignment means that software and data can be managed in surprisingly similar ways. And, fortunately, we already have the tools to do it.</p><h3>API-first approach and communication over the web</h3><p>First of all, in an ideal FAIR world, the way we write software is different from how enterprises did that for decades, as we push business logic towards knowledge models (ontologies) and data, away from software. As Martynas Jusevičius told recently:</p><blockquote><em>“</em>Ora Lassila has claimed that all non-RDF data is legacy data. I would go one step further and claim that all software not built natively on and for RDF is legacy software<em>.” (</em><a href="https://www.linkedin.com/posts/martynasjusevicius_ora-lassila-has-claimed-that-all-non-rdf-activity-7406659841819168769-Wqx-/"><em>source</em></a><em>).</em></blockquote><p>I find this point interesting, but I personally have yet to check what it actually means and to what contexts it applies. On the contrary, domain-driven design (DDD) is pretty widely adopted nowadays and delivers value by many means, with one of them being modeling the code after business concepts, not technical vocabulary.</p><p>Another interesting relationship between software and data is that FAIR ultimately requires data products following similar principles that software engineers seem to have widely adopted: modular development (microservices) and communication over HTTP APIs.</p><p><em>With data products being built like web services, you could apply the same observability pillars and access control mechanisms to them.</em></p><p>And then it also becomes clear why the architect mentioned at the beginning of the article said, “Data mesh without service mesh does not make sense”. Yes, this phrase was cut from the context. The guy worked at a digital-native company before joining a traditional enterprise, so for him, it is easy to judge.</p><p>Not all data products are built like web services, for a number of reasons, and a service mesh would not be ultimately necessary to start implementing a data mesh. But still, the connection between the two concepts does exist and should not be ignored.</p><figure><img alt="Diagram showing enabling platforms across knowledge, runtime, and analytics planes." src="https://cdn-images-1.medium.com/max/960/0*sGfHpsMbE6Zl29tP.jpeg" /><figcaption>How analytics, runtime, and knowledge intersect to create enterprise context | Source: Author</figcaption></figure><p>Data mesh tools focus on data-specific needs (metadata, lineage, access policies), while service mesh focuses on network-level concerns (encryption, load balancing, observability). One lives in the runtime world of microservices, the other in the organizational world of data domains.</p><p>Data mesh could rely on service mesh to provide the underlying networking and security fabric that connects domain-owned data products. Without service mesh, data mesh domains could struggle to enforce access controls, manage traffic, and ensure secure communication. On the contrary, service mesh ensures APIs feeding AI are reliable, secure, and observable.</p><figure><img alt="Diagram illustrating how data mesh governance pairs with service mesh infrastructure fabric." src="https://cdn-images-1.medium.com/max/1024/0*LtfM_xrmhngiJmDE.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p><em>Service mesh is not mandatory for all data products, but it would be quite helpful for those exposed as services.</em></p><p>With data being governed using the data mesh principles, with specialized tools and platforms supporting this, and with service mesh enabling services to talk to each other in a secure, observable way, it is easier to draw the technical boundaries between domains.</p><p>With that said, it is also important to mention that technology alone is not sufficient, as <a href="https://moderndata101.substack.com/i/140008086/building-less-scary-products-starts-with-a-practical-product-mindset"><strong>teams have to adopt the product mindset</strong></a> when it comes to both data and software. Handling them as a product means not only developing them, but also managing them along the whole lifecycle, including operations, monitoring, and so on.</p><h3>Communication between the domains</h3><p>It appears natural that different domains would like to communicate with each other: stages in a supply chain depend on each other, production facilities sometimes need to know the results of the R&amp;D activities, and recommendation engine domain needs to pull historical data for past transactions of a customer.</p><p>While different parts of organizations would like to communicate on the architecture level, we would like to prevent tight coupling between domains: scenarios when components are highly dependent on each other, making them less flexible and harder to maintain.</p><p>The problem of coupling has many aspects: technological (e.g., programming languages), data format (binary, Parquet, CSV), interaction style (async vs sync, REST vs. GraphQL), and many others. Event-driven architectures seem to be a good way to minimize the coupling.</p><h4>What is actually an event?</h4><p>As a quantum of event-driven architectures, it is a signal that the state of the system changed. <strong>One necessary property of it is that it is immutable.</strong> As a consequence of these two properties, in a decentralized architecture, an event can serve as a contract for integrating different systems.</p><figure><img alt="Diagram showing publish/subscribe event choreography decoupling service producers and consumers." src="https://cdn-images-1.medium.com/max/1024/0*LgtLvgU1j6SEx7QE.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>Then producing and consuming domains can agree upfront on a schema of the events. What attributes would a good schema contain? At the very least, an event needs to contain notification about change.</p><p>But the more self-contained an event is, the more successful an integration can be. Thus, we would like to see not only the fact that something changed, but also the nature of the changes (what data was modified) and the context.</p><p>In the best case, a domain event should be stateless, immutable, modeled as close to the domain as possible, belong to one of the types from the controlled vocabulary/taxonomy, be scoped to a bounded context, and describe the intent behind the change.</p><p>Such an event can ensure interoperability and thus serve as a good “currency” in a decentralized architecture. Furthermore, an event with ID can help with deduplication.</p><ul><li>Producing domains are not only r<strong>esponsible for adding context to their events, but also for versioning and ensuring backward compatibility of it</strong> (for some time),</li><li>while consuming domains must take ownership of <strong>consumer-related logic</strong>.</li></ul><p>It is considered a good practice when producers know their consumers and do not give out secrets of their life on a wall at the central square — e.g., they can have a policy that only users with specified roles or keys can read the information in the event payload.</p><p>As communication between domains happens over networks, I think it makes sense to touch on some networking aspects, too. If domains are responsible for managing their own software, data assets, and their domain events, it follows naturally that they should also take on more responsibility for managing their network and communication streams.</p><p><em>Different domains may have unique requirements for security, latency, or compliance, making centralized control impractical.</em></p><p>While it’s unrealistic for domains to manage the entire networking infrastructure, they can realistically own their access policies and monitor their networks using modern observability tools.</p><p>Service mesh empowers domains to configure their own ingress/egress rules, retries, timeouts, and load balancing. With observability tools, domains can independently track latency, errors, and traffic — without relying on a central team.</p><p>Managing policies (e.g., access or firewall rules) as code enables versioning, testing, and deployment via CI/CD pipelines, ideally in a tool-agnostic way.</p><h4>Boundaries between domains are enforced at two levels.</h4><p>Thanks to the network isolation (isolated virtual networks and micro-segmentation) on one side and event-driven architecture on the other, we can decouple workloads and services, enabling independent scaling.</p><p>Instead of service A polling service B, service B emits events when changes occur, embodying a “choreography” mindset (vs. “orchestration”) and enabling service-level decentralization. Back on the network layer, tools like PrivateLink and enforced encryption for all inter-domain communication ensure secure connectivity between domains.</p><figure><img alt="Diagram showing domain boundary security, ingress/egress rules, and policies as code." src="https://cdn-images-1.medium.com/max/1024/0*BzIkAkNCMHnxGCp6.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><h3>The difficult part: human behaviour and business architecture</h3><p>When decentralizing organizations, you need a systematic approach that will touch different aspects of domain operations. In this text, I touched on the main principles, such as:</p><ul><li>decentralized ownership over software, data, and network assets;</li><li>self-service platform operated by centralized platform teams;</li><li>cross-functional collaboration happening on multiple levels, especially in governance;</li><li>computational governance, policies as code, infrastructure as code, versioned resources and workflows;</li><li>API-first design;</li><li>event-driven architecture.</li></ul><p>But as compelling as your approach may sound on paper, reality is far more challenging. You’ll face legacy cultures rooted in the industrial era, organizational inertia, and deep-seated biases.</p><p>You’ll need to strike the right balance between centralization and decentralization — one that aligns with your organization’s unique context. And you’ll have to prepare people for new roles, like data stewards, which require both technical and cultural shifts.</p><p>To be honest, this is more art than science.</p><h4>Start with a business problem.</h4><p>I’ve seen many teams eager to improve how their IT organizations operate, confident that they have management buy-in. But too often, their plans were disconnected from real business problems.</p><p>Instead of addressing tangible pain points, they focused on crafting the perfect architecture (on paper). That’s not how transformative changes like decentralization work.</p><p><em>You can’t roll out a grand vision all at once.</em></p><p>You have to start with a specific, agreed-upon business problem — something concrete that management cares about. For example, instead of deploying an entire data mesh infrastructure upfront, you might begin with a data catalog to help the R&amp;D team gain visibility into the company’s data assets.</p><figure><img alt="Diagram contrasting the stalling architecture astronaut path with business-first momentum." src="https://cdn-images-1.medium.com/max/1024/0*chPyEaXEsCEdjJf5.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>This approach achieves two critical things: real buy-in (when your architecture solves a specific, urgent problem, stakeholders see its value immediately) and purpose (real problems provide a clear sense of direction: something abstract architecture diagrams often lack). Start small, solve a real problem, and build momentum from there. That’s how meaningful change happens.</p><h4>Search for early adopters, build alliances.</h4><p>In any organization, even the quietest ones, there are always people who are eager to learn and try out new things.</p><p><em>Search for them: only when you work with engaged people, is there a chance that your initiative will succeed.</em></p><p>I covered the topic of engaging employees in another article of mine: <a href="https://www.linkedin.com/pulse/every-employee-superhero-when-understand-economics-jobs-radziuk-4jvne/">Every employee is a superhero — when they understand the economics of their jobs | LinkedIn</a> Also don’t be afraid of building alliances across the company to ensure the success of your initiative. Again, the goal is to find people who, for some reasons, are interested in your version of the future.</p><h4>Decentralization is primarily a business endeavour, not technical one.</h4><p>I described the case of Haier, which reshaped its whole operational model in a decentralized spirit.</p><p><em>Decentralization is ultimately about building capabilities and shifting the authority closer to where the problems are solved.</em></p><p>Authority allows the domain to make decisions closer to the outside world, to customers. But they also have to have the right capabilities to support their judgment. One cannot work without the other. And what is most important is that only a culture built on trust, on a positive perception of humans, can allow for a true decentralization, resulting in better decisions.</p><p>When I was writing this piece, <a href="https://open.substack.com/users/138749908-bjrn-broum?utm_source=mentions">Bjørn Broum</a> <a href="https://bbroum.substack.com/p/where-data-strategy-actually-lives">published</a> a text where he distinguishes between delivery-centric vs. value-centric organizations. While the first ones aim at predictable outcomes and controlled workflows, the latter, which remind me of classical Japanese management practices: <strong>aim to grow value over time by continuously improving themselves.</strong></p><figure><img alt="Diagram showing four pillars of decentralisation: Culture, Data, Structure, and Tech." src="https://cdn-images-1.medium.com/max/1024/0*SJewCzIRKSXQIbsX.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>But this learning can only happen close to where the problems emerge. Which, in its turn, requires having true ownership over one’s own workflows and assets. Thus, by focusing on the technical side, we risk again hoping for a magic tool (or self-service platform) to solve our problems. But as it usually is, we need to do some homework on the organizational level first.</p><h4>MD101 Connect ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><h4>Author Connect 💬</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w1kY82UdArfOh9t0.png" /></figure><p>Connect with <a href="https://www.linkedin.com/in/vlad-radziuk-51318712a/">Vlad on LinkedIn</a> 💬</p><p><em>From MD101</em> <em>team</em> 🧡</p><h4>The Data Product Playbook</h4><p>Here’s your own copy of the Actionable Data Product Playbook. With<strong> 3500</strong>+<strong> downloads</strong> so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to <a href="https://www.moderndata101.com/">moderndata101.com</a> for more actionable resources from us!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XT9kEZSfghTCsXxX" /><figcaption><a href="https://www.moderndata101.com/the-data-product-playbook">DOWNLOAD!</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05990e239928" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[60% of AI Projects Fail: 7 Ways to Fix Data Quality Before It Kills Your Models]]></title>
            <link>https://medium.com/@community_md101/60-of-ai-projects-fail-7-ways-to-fix-data-quality-before-it-kills-your-models-71f9238f9099?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/71f9238f9099</guid>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-modeling]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Thu, 17 Sep 2026 15:00:39 GMT</pubDate>
            <atom:updated>2026-09-17T15:00:39.669Z</atom:updated>
            <content:encoded><![CDATA[<h4>Discover 7 trends in how AI is transforming data quality management, from automated cleansing and anomaly detection to observability, NLP, and agentic AI for improving trust, governance, and AI readiness.</h4><p>Author: <a href="https://www.linkedin.com/in/ritwika-c-18104a1b3/">Ritwika Chowdhury</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/7-ways-ai-helps-improve-data-quality"><strong><em>Modern Data 101</em></strong></a><strong><em> Website.</em></strong></p><h3>TL;DR</h3><ul><li><strong>The Stakes:</strong> Through 2026, Gartner warns that <strong>60% of enterprise AI projects will be abandoned</strong> due to poor data quality, costing businesses millions.</li><li><strong>The Shift:</strong> Data quality is moving from a reactive, manual downstream cleanup to an autonomous, real-time upstream pipeline function.</li><li><strong>The 7 AI Catalysts:</strong> Modern teams are leveraging machine learning to scale <strong>automated cleansing, contextual entity resolution, real-time anomaly detection, proactive ingestion assurance, NLP for unstructured text, predictive observability,</strong> and autonomous <strong>Agentic AI</strong> frameworks.</li><li><strong>The Bottom Line:</strong> High-quality data is no longer an operational checklist; it is the defining architectural foundation that determines whether enterprise AI succeeds or fails.</li></ul><p>Poor data quality costs businesses an average of <a href="https://www.moderndata101.com/blogs/the-12-9m-problem-what-poor-entity-resolution-is-really-costing-your-organisation"><strong>$12.9 million per year</strong></a>. And the problem is scaling faster than most data teams can handle; global data generation is on track to hit 181 zettabytes, while <a href="https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk">Gartner warns</a> that through 2026, organisations will abandon 60% of AI projects due to insufficient data quality.</p><p><a href="https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk">High-quality data drives actionable insights, faster innovation</a>, and confident decision-making. Poor data quality, meanwhile, leads to flawed analysis, operational inefficiencies, and growing financial and reputational risk at scale.</p><figure><img alt="Line graph contrasting exponential data growth against flat human capacity to cleanse data." src="https://cdn-images-1.medium.com/max/1024/0*OWlYRsQ2q4c04e1q" /><figcaption><em>Data grows exponentially, whereas manual data management doesn’t | Source: Author</em></figcaption></figure><p>Here are seven major ways AI is transforming data quality management today.</p><h3>7 Trends of AI Data Quality Management in 2026</h3><h4>1. Pipeline-Embedded Cleansing: Moving Quality Upstream</h4><p>In a traditional model, analysts manually spotting discrepancies, engineers tracing pipelines for hours, is <a href="https://moderndata101.substack.com/p/data-quality-a-cultural-device">well-documented as unsustainable</a>. It treats quality as a downstream fix in most cases. AI changes that by embedding cleansing directly into the pipeline.</p><figure><img alt="Stopwatch graphic illustrating a 90% reduction in configuration and deployment time using AI data cleansing." src="https://cdn-images-1.medium.com/max/1024/0*PFijXEI0v3uwSKSf" /><figcaption><em>AI-powered data cleansing cuts effort by up to 90% while improving accuracy | Source: Author</em></figcaption></figure><p>Machine learning algorithms now automatically detect and correct duplicate records, format inconsistencies, and missing values without manual intervention. Natural Language Processing (NLP) standardises free-text fields, resolving spelling variations and naming inconsistencies that rule-based tools miss entirely. According to multiple studies, <a href="https://arxiv.org/pdf/2406.10940">AI-based approaches significantly outperform traditional rule-based ones</a> and can reduce configuration and deployment time by <strong>up to 90%</strong>.</p><blockquote><strong>📕 Related read:</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/data-modeling-resurrection-stone-for-scalable-and-independent-ai-ml"><strong>Data Modeling: Resurrection Stone for Scalable and Independent AI/ML</strong></a></blockquote><h4>2. Contextual Entity Resolution: Unifying Fragmented Identities</h4><p>Modern enterprises struggle with fragmented customer, product, and operational records spread across multiple systems.</p><p>AI-driven entity resolution helps unify these fragmented identities more accurately than traditional deterministic matching approaches. Machine learning models can analyse contextual similarities across attributes, behaviours, metadata, and relationships to identify duplicate or related entities.</p><figure><img alt="Diagram showing machine learning unifying fragmented profile data across CRM, Support, and Sales systems." src="https://cdn-images-1.medium.com/max/1024/0*_b-9Zj8ozaLtH7Ve" /><figcaption><em>AI connects fragmented records into a trusted, unified business identity | Source: Author</em></figcaption></figure><p>This improves master data quality and creates a stronger foundation for analytics, personalisation, and AI applications. It also strengthens the reliability of downstream data products and semantic systems.</p><h4>3. Real-Time Anomaly Detection</h4><p>A data team without AI anomaly detection is, as one analogy puts it, like a security guard trying to monitor thousands of CCTV feeds simultaneously. Most threats go unnoticed due to sheer volume.</p><p>AI models, particularly unsupervised algorithms like clustering and one-class SVMs, scan entire datasets continuously in real time, flagging outliers, schema drift, and distributional shifts as they emerge. Platforms deploying multi-agent detection architectures report accuracy improvements of up to <strong>60%</strong> and error correction rates reaching <strong>95%</strong>. In financial services alone, AI-based anomaly detection has driven a <a href="https://rtslabs.com/ai-anomaly-detection/">67% reduction in undetected fraudulent transactions</a>, a figure that speaks to the operational stakes of getting this right.</p><p>The academic foundation for this is strong: <a href="https://rtslabs.com/ai-anomaly-detection/">a 2024 arXiv theoretical framework</a> on AI-driven monitoring in high-volume environments demonstrates why traditional batch-oriented checks fundamentally cannot keep pace with modern data velocity.</p><blockquote><strong>📕 Related read:</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/top-10-data-quality-dimensions-and-how-unified-data-platforms-enable-them"><strong>Top 10 Data Quality Dimensions and How Unified Data Platforms Enable Them</strong></a></blockquote><h4>4. Detecting Bad Data at the Source</h4><p>When a data issue is caught at the reporting layer, the damage is already done. Decisions have been made. Trust has eroded. The meeting has gone sideways.</p><p>AI enables <strong>continuous, real-time monitoring</strong> of data flows, identifying inconsistencies at the point of ingestion rather than discovery. This shift from reactive detection to proactive assurance is one of the defining characteristics of <a href="https://moderndata101.substack.com/p/how-ai-is-reshaping-data-governance">how AI is reshaping modern governance frameworks</a>. Enterprises that have made this transition report not just fewer data incidents, but a measurable restoration of stakeholder confidence in their numbers, the kind of trust that <a href="https://moderndata101.substack.com/p/data-quality-a-cultural-device">converts data quality from a cultural liability into a cultural device</a>.</p><p>For data leaders building self-serve infrastructure, the <a href="https://datadeveloperplatform.org/">Data Developer Platform</a> offers a useful reference for how real-time monitoring integrates into modern data product architectures.</p><h4>5. NLP for Unstructured Data Standardisation</h4><p>Roughly <a href="https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data">80% of enterprise data is unstructured</a>: emails, documents, logs, support tickets, clinical notes. Most traditional data quality tools weren’t built for any of it.</p><p>NLP bridges this gap. It allows AI systems to clean and standardise free-form text at scale: resolving naming variations, correcting field-level errors, detecting “soft” duplicates where records share meaning but not exact wording. <a href="https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data">A significant research</a> demonstrates this in practice, showing how updatable extracted views over unstructured medical records can systematically identify and correct data quality problems that structured validation entirely misses.</p><p>For teams managing data products end-to-end, understanding NLP’s role in quality is increasingly non-negotiable.</p><h4>6. AI Strengthens Data Observability</h4><p>Data observability platforms increasingly use AI to move beyond simple monitoring into predictive intelligence.</p><p>Rather than only identifying pipeline failures after they occur, AI can predict potential breakdowns based on historical lineage patterns, workload behavior, schema evolution, and operational anomalies.This enables teams to resolve issues before business users or AI systems are impacted.</p><p>Modern observability is becoming less about dashboards and more about operational resilience.</p><blockquote><strong>📕 Related read:</strong></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/how-ai-observability-improves-decision-quality"><strong>How AI Observability Improves Decision Quality</strong></a></blockquote><h4>7. Agentic AI: Autonomous End-to-End Quality Management</h4><p>If the previous six shifts represent AI assisting data quality, agentic AI represents AI <em>owning</em> it. Multi-agent architectures deploy specialised AI agents across the entire data ecosystem: profiling agents establish quality baselines, anomaly agents detect deviations in real time, remediation agents execute corrective actions, and learning agents improve system intelligence through feedback loops, all without human initiation.‍</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6uLgKWyi7amjg3nl" /><figcaption><em>Enterprise AI succeeds only when built on trusted data foundations | Source: Author</em></figcaption></figure><p>A prominent <a href="https://arxiv.org/pdf/2512.05559">research</a> demonstrates measurable gains from this approach in live financial environments: improved anomaly detection recall, significant reduction in manual remediation effort, and stronger auditability in high-throughput regulated pipelines.</p><p>This is the direction the field is moving. <a href="https://arxiv.org/pdf/2512.05559">Agentic systems are the next frontier </a>in enterprise GenAI, and data quality is one of the most tractable, high-ROI starting points for deployment.</p><h3>What This Means for Data Leaders</h3><p>The pattern across all seven shifts is the same: <strong>AI moves data quality from a cost center to a strategic function.</strong> It replaces the reactive, fragmented model that data teams have long normalised, the one built around patch-fixing, with an intelligent, self-improving system designed around prevention, trust, and speed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t2C53vpg2lAcprGX" /></figure><p><a href="https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work">McKinsey’s analysis</a> is clear that two-thirds of organisations have not yet begun scaling AI across the enterprise. Data quality is often why. Closing that gap starts not with model selection or tooling procurement, it starts with the data itself.</p><p>The organisations that will lead the next decade of AI value creation are already treating data quality as a culture, not a checklist.</p><h3>FAQs</h3><h4>Q1. How can AI be used in data quality?</h4><p>AI improves data quality by automating data cleansing, detecting anomalies, identifying duplicates, validating data in real time, enriching metadata, and predicting quality issues before they impact analytics or AI systems. It helps organizations maintain accurate, consistent, timely, and trustworthy data at scale.</p><h4>Q2. How is AI improving data management?</h4><p>AI improves data management by automating data discovery, cleansing, classification, integration, governance, and quality monitoring. It helps detect anomalies in real time, enrich metadata, optimize pipelines, improve semantic understanding, and enable self-service access, making data ecosystems faster, more reliable, scalable, and AI-ready.</p><h4>Q3. Why AI data quality is the key to AI success?</h4><p>AI systems are only as reliable as the data they learn from and operate on. Poor data quality leads to inaccurate predictions, biased outputs, hallucinations, weak automation, and failed decision-making. High-quality data ensures AI models are accurate, contextual, trustworthy, and scalable, making data quality the foundation of successful AI adoption.</p><h4>MD101 Support ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡‍</p><p><em>From MD101</em> <em>team</em> 🧡</p><p>A financial firm’s AI assistant ran for three weeks with zero errors and zero downtime, while quietly citing the wrong fund in every summary it wrote. This is the gap AI observability is built to catch, and this guide lays out the places enterprises need to watch for it: <a href="https://www.moderndata101.com/blogs/blogs-what-is-ai-observability-guide/">What Is AI Observability? Enterprise Stack &amp; Guide for 2026 | Modern Data Blog</a></p><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*rwos2JEVTlbu1qHA.png" /></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71f9238f9099" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Data Management: What It Actually Takes to Trust an AI Agent]]></title>
            <link>https://medium.com/@community_md101/ai-data-management-what-it-actually-takes-to-trust-an-ai-agent-c503575f91ff?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c503575f91ff</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Tue, 25 Aug 2026 16:04:28 GMT</pubDate>
            <atom:updated>2026-08-25T16:04:28.151Z</atom:updated>
            <content:encoded><![CDATA[<h4>Most enterprise data infrastructure was built for human analysts. Here’s what AI-ready data actually requires, and where governance, metadata, and entity resolution fit in.</h4><p>Author: <a href="https://www.linkedin.com/in/akshay-chame-b43bb8209/">Akshay Chame</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/ai-data-management-how-to-make-data-ai-ready"><strong><em>Modern Data 101</em></strong></a><strong><em> Website.</em></strong></p><h3>TL;DR</h3><p>AI data management is the practice of structuring, governing, and packaging enterprise data so AI systems can consume it reliably at scale. The data beneath AI deployments fails is built for human analysts, not machines, and that is why it fails. This post is for data leaders evaluating what AI-ready infrastructure actually requires.</p><p>This post is for <a href="https://www.moderndata101.com/blogs/ai-ready-data-vs-analytics-ready-data">data leaders evaluating what AI-ready infrastructure actually requires</a>.</p><figure><img alt="Scale balancing simple AI deployment against the complex data infrastructure needed to build trust." src="https://cdn-images-1.medium.com/max/1024/0*VCWZmMUztqzAu9nc" /><figcaption><em>While prototyping is easy, production trust depends entirely on the underlying data infrastructure</em></figcaption></figure><p>Deploying an AI agent is easy. Trusting one is a different matter entirely, and that gap almost always <a href="https://dataos.info/products/data_product/#:~:text=DataOS%20provides%20the%20platform%20for%20the%20development%2C%20management%2C%20processing%2C%20and%20deployment%20of%20Data%20Products%20across%20an%20organization">traces back to data management infrastructure</a>.</p><p>The failures tend to look deceptively simple: a procurement agent can’t reconcile supplier records because the same vendor exists under four different names across three ERP systems, or a customer service agent surfaces pricing that’s six weeks out of date. This is what happens when data built for human interpretation gets handed to machines that can’t fill in the gaps themselves.</p><p>“AI-ready data” has become one of those phrases that sounds self-explanatory until you try to build for it. The requirements are specific, and <a href="http://moderndata101.com/blogs/data-readiness-for-ai-5-fundamental-factors-to-consider">most data estates weren’t designed</a> with any of them in mind.</p><h3>What AI-Ready Data Infrastructure Requires (That BI Dashboards Never Needed)</h3><p>Traditional <a href="https://www.moderndata101.com/glossary/automated-data-pipelines">pipelines</a> were built for human latency: batch schedules, daily refreshes, an analyst reviewing the output the next morning.</p><p>Agentic AI operates differently; it doesn’t tolerate ambiguity. When an agent queries a customer entity, it needs one authoritative answer, not three slightly different records from three source systems. When it reads a product catalogue, it needs consistent schemas, not the structural patchwork that accumulates when systems coexist without a unifying contract.</p><p>The requirements shift is significant:</p><figure><img alt="Comparison table contrasting BI reporting data needs with real-time AI agent." src="https://cdn-images-1.medium.com/max/1024/0*Vv6gCEh3jm86EeRd" /></figure><p>Engineering teams will recognise this immediately. It maps directly to what they encounter when trying to productionise AI workflows on top of existing data estates.</p><figure><img alt="Diagram showing broken data pipelines leading to AI procurement and customer service failures." src="https://cdn-images-1.medium.com/max/1024/0*98wC9NNL4qh8MxbZ" /><figcaption><em>Machines cannot fill in the gaps of data systems originally designed for human interpretation.</em></figcaption></figure><h3>Why Metadata Management is the Key to AI-Ready Data</h3><p>There’s a tendency to frame AI readiness as a data quality problem: <em>are the records complete, are the formats consistent, is the pipeline fresh?</em> That framing is just incomplete because the deeper issue is <a href="https://moderndata101.substack.com/p/ai-ready-data-a-technical-assessment">whether the data means anything unambiguous</a> to a machine reading it without context, and that’s a metadata question.</p><p>If a customer_id could mean three different things depending on the system it came from, the agent doesn&#39;t know which one applies. That ambiguity has to be resolved somewhere because if it&#39;s not in the metadata, it resolves incorrectly in the output in the wrong manner.</p><p>The cost of getting metadata wrong has changed. A human analyst misreading a field definition causes a bad report. An agent misreading it can trigger cascading automated decisions, repriced contracts, rejected claims, and incorrectly routed escalations. It’s the plumbing that determines whether AI automation is trustworthy at all.</p><figure><img alt="Diagram showing how ambiguous metadata leads to misinterpretations and cascading AI agent failures." src="https://cdn-images-1.medium.com/max/1024/0*cWRLI4wdyXYwW5C-" /><figcaption><em>Ambiguity in metadata is the enemy of automation, leading to costly operational errors.</em></figcaption></figure><blockquote><a href="https://www.moderndata101.com/blogs/ai-ready-data-a-technical-assessment-the-fuel-and-the-friction"><strong>📙 Read more:</strong></a></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/ai-ready-data-a-technical-assessment-the-fuel-and-the-friction"><strong>AI-Ready Data: A Technical Assessment. The Fuel and the Friction.</strong></a></blockquote><h3>How Does Entity Resolution Ensure Reliable AI Automation</h3><p><a href="https://www.moderndata101.com/blogs/entity-resolution-at-scale-deduplication-strategies-for-knowledge-graph-construction">Entity resolution has been a data engineering concern</a> for decades, matching records that describe the same real-world thing across systems that never agreed on a common identifier. It was always worth doing properly. In agentic workflows, doing it badly has immediate operational consequences rather than just untidy reports.</p><p>Consider a compliance automation agent cross-referencing counterparties against a sanctions list. If the same counterparty appears under four different identifiers across trading and onboarding systems, the agent either flags everything conservatively or misses matches. Neither outcome is acceptable.</p><figure><img alt="Funnel diagram showing how resolving entity identifiers into a golden record prevents AI compliance failure." src="https://cdn-images-1.medium.com/max/1024/0*7LkHJcajHJ2EyrtC" /><figcaption><em>Establishing canonical identifiers through a knowledge graph is a precondition for reliable automation.</em></figcaption></figure><p>Practically, this means that <a href="https://dataos.info/products/data_product/">building a knowledge graph that can support downstream AI systems</a>, one that encodes entity relationships, canonical identifiers, and provenance, is increasingly a precondition for reliable automation, not a future optimisation.</p><h3>Data Products as the Unit of Enterprise Data Management for AI</h3><p><a href="https://www.moderndata101.com/blogs/what-are-data-products-the-complete-guide">Treating data as a product</a>, with an owner, a defined interface, a quality SLA, and a documented consumer contract was originally conceived for human consumers. It turns out to be even better suited to machine consumers.</p><p>An AI agent that consumes a well-defined <a href="https://www.moderndata101.com/glossary/data-product">data product</a> with stable schemas, observable lineage, explicit ownership, and versioned outputs can be built to a contract. Changes to the underlying data are visible, and degradation triggers alerts. The agent’s behaviour is therefore traceable and debuggable in ways that agents consuming raw warehouse tables simply are not.</p><p>As automation becomes more widespread, traceability and accountability become essential governance needs, not just tools for troubleshooting.</p><blockquote><a href="https://www.moderndata101.com/blogs/what-are-data-products-the-complete-guide"><strong>📙 Read more:</strong></a></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/what-are-data-products-the-complete-guide"><strong>The Complete Guide to Data Products</strong></a></blockquote><h3>AI Data Governance vs. Traditional Data Governance: What’s Different</h3><p>Traditional data governance was largely retrospective, auditing what happened, documenting lineage after the fact, and enforcing access policies through human review cycles. That model works when humans are making decisions and can be held accountable. It breaks down when decisions are being made at machine speed, at scale, by agents that no individual oversees in real time.</p><p><a href="https://www.moderndata101.com/glossary/access-control">Access control</a>, quality checks, and policy enforcement need to happen at the moment a data asset is queried or consumed, not in a weekly audit cycle.</p><p>The practical implication: <a href="https://www.moderndata101.com/blogs/the-role-of-self-serve-data-platforms-in-data-accessibility">governance frameworks need to be rebuilt</a> around <a href="https://www.moderndata101.com/glossary/composability">composable</a> policy primitives that can be attached to data products and enforced at runtime. Governance needs to be embedded directly into the data management platform; enforced at runtime, not delegated to a review process that runs on a quarterly cycle.</p><h3>Where to Start With Data Management for Agentic AI</h3><p>The conventional instinct when confronting AI data management gaps is to reach for tooling first. But tooling built on top of unresolved entities and ungoverned access inherits every structural failure it was supposed to fix.</p><figure><img alt="Layered diagram showing raw data, entity resolution, active metadata, data products, and governance gates supporting an AI agent." src="https://cdn-images-1.medium.com/max/1024/0*OyL0aKk7onWVgQ1p" /><figcaption><em>A reliable AI platform must be built on integrated layers of resolution, metadata, and governance.</em></figcaption></figure><p>Start at the data contract layer: resolve entities, document schemas, and enforce access policy at the platform level before selecting any AI system that depends on them.</p><figure><img alt="5-step roadmap detailing audit, entity resolution, observability, data products, and platform governance." src="https://cdn-images-1.medium.com/max/1024/0*srnq1gKAOBo7mLn-" /></figure><h3>FAQs</h3><h4>Q1. What are the key principles of good governance?</h4><p>Good governance rests on transparency, accountability, fairness, and the effective use of resources in decision-making. For data governance for AI, this means policies enforced at the point data is queried, not reviewed after the fact.</p><h4>Q2. What is the difference between data and metadata?</h4><p>Data is the actual content; metadata is the information describing it, such as author, file size, or creation date. In data management, metadata gives machines the context data alone cannot provide.</p><h4>Q3. Why is metadata important?</h4><p>Metadata enables effective search, organisation, access control, and AI readiness by providing essential context and structure for large datasets. It is foundational to enterprise data management at scale.</p><h4>Q4. What makes data AI-ready?</h4><p>AI-ready data is high-quality, clearly labelled, free from errors or duplicates, and includes semantic context and real-time updates. It is the baseline requirement for agentic AI to operate reliably.</p><p>Getting the data layer right is also what determines whether AI actually pays for itself. Our recent piece on <a href="https://www.moderndata101.com/blogs/lean-ai-building-a-scalable-data-platform-for-enterprise-ai-roi">Lean AI: Building a Scalable Data Platform for Enterprise AI ROI</a> walks through what that looks like in practice.</p><h4>MD101 Support ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><h4>Author Connect 🖋️</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*a0qrg69XkjgMzeNuB1xgbg.jpeg" /></figure><p>Connect: <a href="https://www.moderndata101.com/network/akshay-chame">Akshay Chame</a></p><p><em>From MD101</em> <em>team</em> 🧡</p><p>A financial firm’s AI assistant ran for three weeks with zero errors and zero downtime, while quietly citing the wrong fund in every summary it wrote. This is the gap AI observability is built to catch, and this guide lays out the places enterprises need to watch for it: <a href="https://www.moderndata101.com/blogs/blogs-what-is-ai-observability-guide/">What Is AI Observability? Enterprise Stack &amp; Guide for 2026 | Modern Data Blog</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*6pRQVCc8F-JURBR5.png" /></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c503575f91ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Long Until We Call AI Agents Data Products]]></title>
            <link>https://medium.com/@community_md101/how-long-until-we-call-ai-agents-data-products-9575a7ef7450?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9575a7ef7450</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[language]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[business-strategy]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Thu, 20 Aug 2026 14:52:09 GMT</pubDate>
            <atom:updated>2026-08-20T14:52:09.104Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://moderndata101.substack.com/s/experts-desk/?utm_source=substack&amp;utm_medium=menu">Expert’s Desk ⭐️</a></h4><h4>Most teams treat agents like experiments. The ones that survive production treat them like data products with users, feedback loops, and a roadmap.</h4><p>Author: <a href="https://www.linkedin.com/in/alejandro-aboy/">Alejandro Aboy</a></p><h4>About Our Contributing Expert</h4><h3>Alejandro Aboy | Senior Data &amp; AI Engineer, Writer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*66JzjbXCsC9a0uQI.png" /></figure><p><a href="https://www.linkedin.com/in/alejandro-aboy/">Alejandro</a> is a Senior Data &amp; AI Engineer and the voice behind <a href="https://thepipeandtheline.substack.com/"><em>The Pipe &amp; The Line</em></a>, a technical newsletter where pipelines meet product thinking. With a career that began in marketing and web analytics, he transitioned into modern data engineering, bringing with him a sharp instinct for business context, experimentation, and stakeholder alignment.</p><p>At Workpath, Alejandro serves as the main owner of the Data Platform &amp; AI stack. He builds AI agents with RAG architectures (Pgvector on Aurora), designs MCP-based tool ecosystems, and implements production-grade AI observability. His work spans dbt modeling, orchestration with Airflow, Slack-to-Tableau automation, and secure backend design for agentic systems.</p><p>Alejandro is deeply engaged in the open data community. He has collaborated with OpenMetadata on agentic data modeling and metadata-first architectures, co-authoring end-to-end implementations that connect lineage, governance, and AI tooling through MCP. His writing is frequently featured by Modern Data 101 and other leading data publications. We’re thrilled to feature his unique insights on <a href="https://moderndata101.substack.com/">Modern Data 101</a>!</p><blockquote>We actively collaborate with data experts to bring the best resources to a <a href="https://www.moderndata101.com/">20,000+ strong community</a> of data leaders and practitioners. If you have something to share, reach out!<br>🫴🏻 Share your ideas and work: <a href="http://community@moderndata101.com/">community@moderndata101.com</a></blockquote><p><strong>*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality &amp; relevance. We keep it information-first and do not support any promotions, paid or otherwise.</strong></p><h3>Let’s Dive In</h3><h4>TL;DR</h4><ul><li><strong>An AI agent in production meets every formal definition of a data product.</strong> It has users, serves analytical data, needs quality monitoring, evolves through versioned schemas, and breaks silently when you ignore it.</li><li><strong>Observability is not logging.</strong> It’s product analytics. Every conversation is a user session. Every tool call is a feature interaction. Every retry is a frustration you can measure.</li><li><strong>Undetected errors are invisible churn.</strong> Your agent can score perfectly on generic metrics while hallucinating links, suggesting impossible actions, and losing users you’ll never hear from.</li><li><strong>The feedback loop IS the roadmap.</strong> Manual annotation sessions, clustered by patterns, connected to tickets. Not dashboards. Decisions.</li><li><strong>Security is a product contract, not a checklist.</strong> 4 defense layers, human-in-the-loop, and scoped permissions. A data product nobody trusts is a data product nobody uses.</li><li><strong>Your agent can be its own onboarding tool.</strong> Enable it to explain its own features through conversation. Documentation becomes discoverability. Every “how do I…” question is a product signal.</li><li><strong>The teams winning aren’t the ones with the best prompts.</strong> They’re the ones treating agent behavior as a continuous product input.</li></ul><p><em>Disclaimer: All statements here are from a year of managing an AI companion in production at Workpath. 50+ tools, RAG, workflow memory, frontend context awareness. Not theory.</em></p><h3>The Subtle Shift that Went Unnoticed</h3><p>You shipped your agent. Observability is green. Traces are flowing. Users are chatting. But can you answer these?</p><ol><li><em>What are users actually trying to do with it?</em></li><li><em>Which tools should you optimize or kill?</em></li><li><em>What feature requests are hiding in conversations you never read?</em></li></ol><p>Not being able to answer any of this means you have an experiment running in production. Most teams assigned agents to ML engineers or product managers. In my opinion, they are partially right and partially wrong.</p><p>The infrastructure underneath: RAG pipelines, semantic layers, observability traces, that’s data engineering territory. And the vision on top screams product thinking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*_baS3kGEliktFWkHmVgF0Q.jpeg" /></figure><p><em>When you realise, the work for the Data shifted into building processes that turn agent behavior into product decisions.</em></p><h3>Your Agent Already Is a Data Product</h3><p>As we can read in this <a href="https://moderndata101.substack.com/p/building-robust-data-products-5-pillars">post</a>, we usually describe data products as assets that stakeholders should be able to discover, access, trust, and interact with in a secure fashion.</p><p>You can’t carry that vision without observability, clean architectures, and proactive quality monitoring. Let’s see how your AI Agent can fit conceptually in this definition:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*PmOHyVkBkJWhUPWA9ROfrg.jpeg" /></figure><p><em>A data product has users, quality contracts, a lifecycle, and evolves based on feedback. Your agent has all of that. The question is whether you’re treating it that way.</em></p><h3>AI Observability Is Product Analytics</h3><p>Most teams connect to Opik or Langfuse and call it done. But that’s just logging. Product analytics is a different ballgame. It answers: What should I build next? Where should I put my focus?</p><h4>Every conversation is a user session.</h4><p>When I started reading conversations manually at Workpath, I found <strong>users hacking the agent for use cases we never designed</strong>. 50% wanted bulk operations we didn’t support, and others were asking for exports that didn’t exist.</p><p>None of this showed up in any dashboard. <em>The tools were not raising errors because </em><strong><em>everything was failing successfully</em></strong><em>.</em></p><figure><img alt="Iceberg diagram illustrating how hidden AI errors like broken links and user churn lie beneath green metrics." src="https://cdn-images-1.medium.com/max/1024/0*Z1VejzL5t_dm6BWv.png" /></figure><h4>Every tool call is a feature interaction.</h4><p>We had 50 tools. Users called 5 regularly. Some tools were ghosts because the agent decided to never use them. Others had high usage but huge failure rates.</p><p>That’s a product prioritization alert: fix the broken popular ones, kill the unused ones, and merge the ones that always get called together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GhR5WuSgSjb4KK9f.png" /><figcaption>Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><h4>Every retry is a measurable frustration.</h4><ul><li>When a user rephrases the same question three times, that’s friction instead of a “conversation.”</li><li>When they abandon after “<em>I can’t help with that</em>,” that’s churn.</li><li>When they ask for something five times and give up, that’s a feature request screaming right at your face.</li></ul><p><em>Stop evaluating agents on correctness scores. Start evaluating them on what users are trying to do and whether the agent helps them get there.</em></p><h3>Undetected Errors Are Invisible Churn</h3><p>My agent had perfect hallucination scores. Then I started reading conversations manually and found:</p><ul><li>Agent hallucinating documentation links after fake-searching the docs</li><li>Agent inferring wrong user information and making incorrect tool calls</li><li>Agent suggesting follow-up actions that literally didn’t exist</li></ul><p>Generic metrics (e.g., <em>Hallucination</em>, <em>AnswerRelevance</em>, <em>ContextPrecision</em>) are too abstract to catch this. They tell you something is “<em>good</em>” or “<em>bad</em>” without telling you what to fix.</p><h4>What works</h4><p>Binary evaluation criteria specific to your use case. “<em>Did the agent call search_knowledge before citing a URL? Does the URL in the response match the tool output?</em>”</p><p>Run it with LLM-as-Judge ten times, and you get the same answer. And if you want a bulletproof validation, ask different colleagues in your company. That’s testable.</p><blockquote>Those undetected bugs are signals about what your agent promises and can’t deliver. Each one is either a ticket to fix something or a scope clarification for what the agent should and shouldn’t do.</blockquote><p>A data product that fails silently is worse than one that fails loudly. At least loud failures get tickets.</p><h3>Security as a Product Contract</h3><p>Security isn’t a checklist you bolt on before launch. For a data product, trustworthiness is a foundational property. If users don’t trust the product, <strong>they stop using it</strong>.</p><p>Your agent accepts natural language from anyone. That’s the largest attack surface in your product. I wrote an <a href="https://thepipeandtheline.substack.com/p/secure-llms-how-to-prevent-prompt">entire series on building jailbreak-proof AI agents</a>, and the core idea is defense in depth across 4 layers:</p><figure><img alt="Security diagram outlining input validation, guardrails, output control, and architecture for AI agents." src="https://cdn-images-1.medium.com/max/1024/0*zHTk20-ZntGUZvL4.png" /><figcaption>The Four Layers of Defense | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><ul><li><strong>Input validation</strong> is a product boundary definition. You’re deciding what your agent will and won’t accept.</li><li><strong>Guardrails</strong> are a scope definition. Topical guardrails keep the agent on-task. PII masking isn’t compliance is built-in trust. Users need to know their data isn’t leaking into responses.</li><li><strong>Output control</strong> prevents the agent from fabricating data that looks real. Structured outputs are the difference between an agent that invents a documentation link and one that only cites what it actually found.</li><li><strong>Architecture</strong> is where engineering meets product. Least privilege, scoped tools, observability. An agent can’t drop your production table if it doesn’t have permissions.</li></ul><p>Then there’s human-in-the-loop. I’ve been evaluating Safety Operations to see if my agents comply with asking users before destructive operations. The agent asking, “<em>Are you sure you want to delete this?</em>” is good UX. It also happens to be good security.</p><p><em>A data product nobody trusts is a data product nobody uses. Security failures don’t show up in your dashboards. They show up as undetected churn.</em></p><h3>Teaching Your Agent What It Can Do</h3><p><strong>Discoverability is the first data product property we listed</strong>. Traditional data products have catalogs. AI agents have something better: they can explain themselves through conversation.</p><p><strong>Users don’t read documentation</strong>. Savvy ones ask the agent, “<em>What can you do?</em>” and if the answer is vague, you lose them before they even try the feature they came for.</p><p>At Workpath, we enabled Zendesk knowledge base articles as a RAG source. When users ask “<em>how do I create a goal?</em>” or “<em>what can you help me with?</em>”, the agent searches those articles and explains features, workflows, and guidance in conversation. The agent seeks to teach users on demand.</p><p><strong><em>The agent becomes its own onboarding tool, and its knowledge base becomes a living source.</em></strong></p><p>Instead of static docs nobody reads, you have a conversational interface that meets users where they are and explains exactly what they need.</p><p>Track those patterns and find discoverability signals you can leverage. If the agent can’t explain a feature clearly, either the knowledge base article is bad, or the feature itself is poorly designed.</p><blockquote><em>Every “how do I…” question is a product signal about what’s confusing, what’s undocumented, and what users actually want to use.</em></blockquote><h3>The Feedback Loop Becomes the Roadmap</h3><p>The process I developed after 1000+ manual (yes, 1 by 1) conversation reviews:</p><ul><li><strong>Before each annotation session, prepare.</strong> Look for anomalies to pick up outliers, such as latency, too many tool calls, long-lasting conversations, and high token consumptions. Don’t read “some chats” and call it a day.</li><li><strong>During annotation, collect structured signals.</strong> Is this a bug? A missing feature? A scope violation? A prompt gap? Each annotation should map to an action type, not just a thumbs-up or thumbs-down.</li><li><strong>After annotation, materialize the roadmap.</strong> Connect patterns to JIRA tickets. Update documentation. Cross-reference against the backlog to validate if the most concerning patterns are already being considered.</li></ul><figure><img alt="Continuous feedback loop showing review, annotation, pattern finding, ticket creation, and deployment for AI agents." src="https://cdn-images-1.medium.com/max/1024/0*u1wImAun-U0L2VZk.png" /><figcaption>The feedback loop is the improvement roadmap | Adapted from concepts shared by the Author | Curated by Modern Data 101</figcaption></figure><p>I started generating a bunch of GitHub PRs just by reading conversations, taking notes, and applying those annotations as Claude Code plans.</p><p>None of these came from user interviews. None came from JIRA. They came from reading what users actually did with the product.</p><p><strong>The loop:</strong> Conversations → Structured Annotation → Clustered Patterns → Tickets → Ship → Measure → Repeat</p><p><em>This is Product management powered by production data (and AI). And it never stops because users never stop talking to your agent.</em></p><h3>Final Words</h3><p>A robust data product encodes business meaning, enforces quality, traces change, and supports continuous evolution. It earns trust through security. And it teaches users what it can do instead of waiting for them to figure it out.</p><p>I haven’t seen a moment like this in my career, where users are telling you what to build next through every single conversation.</p><p>The teams winning at AI agents are the ones with the best feedback loops, the strongest trust contracts, and agents that know how to explain themselves.</p><p>Thanks for reading Modern Data 101! <a href="https://medium.com/@community_md101">Follow us</a> for free to receive new posts and support our work. <strong><em>Originally published on </em></strong><a href="https://moderndata101.substack.com/p/data-products-as-ai-agents"><strong>Modern Data 101</strong></a><strong><em> Substack.</em></strong></p><h4>MD101 Support ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><h4>Author Connect</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qKUVliRPzid86JsW.png" /></figure><p>Connect with <a href="https://www.linkedin.com/in/alejandro-aboy/">Alejandro Aboy on LinkedIn</a> 💬 | Or find more of his work on Substack: <a href="https://thepipeandtheline.substack.com/?utm_source=substack&amp;utm_campaign=publication_embed&amp;utm_medium=web">The Pipe &amp; The Line</a></p><p><em>From MD101</em> <em>team</em> 🧡</p><p>A financial firm’s AI assistant ran for three weeks with zero errors and zero downtime, while quietly citing the wrong fund in every summary it wrote. This is the gap AI observability is built to catch, and this guide lays out the places enterprises need to watch for it: <a href="https://www.moderndata101.com/blogs/blogs-what-is-ai-observability-guide/">What Is AI Observability? Enterprise Stack &amp; Guide for 2026 | Modern Data Blog</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*BsCSnTp5BN5vUoB8.png" /></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9575a7ef7450" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top 7 Signs Your Data Isn’t Ready for AI]]></title>
            <link>https://medium.com/@community_md101/top-7-signs-your-data-isnt-ready-for-ai-477c1fdcb089?source=rss-550c0299513d------2</link>
            <guid isPermaLink="false">https://medium.com/p/477c1fdcb089</guid>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Modern Data 101 ]]></dc:creator>
            <pubDate>Tue, 18 Aug 2026 14:38:02 GMT</pubDate>
            <atom:updated>2026-08-21T10:48:01.941Z</atom:updated>
            <content:encoded><![CDATA[<h4>A walkthrough to how enterprise AI failures trace back to data, becoming the primary bottleneck for organisations in their AI-readiness journey.</h4><p>Author: <a href="https://www.linkedin.com/in/muskanpurohit/">Muskan Purohit</a></p><p><strong><em>Access the original publication on </em></strong><a href="https://www.moderndata101.com/blogs/7-signs-your-data-is-not-ai-ready"><strong>Modern Data 101</strong></a><strong><em> Website.</em></strong></p><h3>TL;DR</h3><p><a href="https://www.moderndata101.com/blogs/what-is-ai-readiness-and-how-to-be-ai-ready">AI readiness for data </a>means your data platform is structured, governed, and accessible enough for AI systems to reliably consume and act on it in production. The trade-off is that investing in AI-ready data requires upfront work in architecture and governance, skipping which leads to unreliable outputs, slower deployments, and higher operational costs.</p><p>This article outlines seven structural signs your platform isn’t AI-ready, what each one costs in production, and where to focus first.</p><p>The overload of tips and tricks when it comes to data and AI readiness is high often times. That’s when listening to industry leaders makes it really easy. Modern Data 101 Community, hence, brings in leaders’ insights to you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v0SsRSr79bqbe4qI" /></figure><p>Access the insights <a href="https://www.moderndata101.com/blogs/takeaways-from-cxo-insights-exclusive-interviews-with-top-operators">here.</a></p><p>More than <a href="https://www.rand.org/pubs/research_reports/RRA2680-1.html">80% of AI projects fail</a>, which is twice the rate of failure for non-AI IT projects. In most AI projects, there’s a quiet conversation that doesn’t make it into the case study. An engineer pulls up the source data and realises it’s inconsistent across systems, missing half the context the model needs, and <a href="https://www.moderndata101.com/blogs/5-ways-to-measure-ai-governance-success-through-metrics-in-the-era-of-agentic-ai">governed by nobody in particular</a>. This leads to the project never quite working.</p><p>The problem is that data issues, which seemed manageable in a traditional analytics environment, become blockers the moment you introduce large language models, <a href="https://www.moderndata101.com/blogs/agentic-graphrag-building-unified-memory-for-ai-agents-with-knowledge-graphs">AI agents, or intelligent retrieval systems</a>. Suddenly, gaps that never surfaced in a dashboard become critical failures in a production AI workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nfw_5fRF4toK5gIh" /><figcaption><em>The Amplification Effect: How manageable data friction in traditional reporting transforms into critical AI system failures | Source: Author</em></figcaption></figure><h3>Is Your Data AI-Ready?</h3><p>Here are seven signs your organisation isn’t fulfilling the criteria of AI-readiness for data yet.</p><h4>1. Your metadata is inconsistent or sparse</h4><p><a href="https://www.themoderndatacompany.com/blog/context-is-the-new-currency-how-understanding-your-data-unlocks-ai-readiness">AI systems consume data and then interpret it</a>, and this interpretation depends heavily on context, which comes from <em>metadata.</em> So, if your data assets are missing the basics, AI tools have almost nothing to work from:</p><p>➡ Tables with no descriptions or business context</p><p>➡ Columns with cryptic, abbreviated, or inherited legacy names</p><p>➡ No lineage, ownership, or freshness information attached to assets</p><p>This matters especially for <a href="https://www.themoderndatacompany.com/blog/context-is-the-new-currency-how-understanding-your-data-unlocks-ai-readiness">retrieval-augmented generation (RAG) architectures</a> and enterprise semantic search systems, where the quality of retrieved context directly shapes the quality of AI output. Sparse metadata doesn’t just slow down discovery; it introduces ambiguity that compounds downstream.</p><p>A simple way to gauge this: if a new engineer still needs to chase people to understand <a href="https://dataos.info/interfaces/metis/metis_ui_assets/">what a dataset means</a>, where it came from, or whether it can be trusted, an AI system will struggle with the same gaps, only at a much larger scale.</p><figure><img alt="Visual comparing a vague legacy table tag with an enriched data asset tag featuring context, owner, and lineage." src="https://cdn-images-1.medium.com/max/1024/0*URhOM8ysgHOuVe2i" /><figcaption><em>Decoding the Context: Moving from cryptic legacy tables to enriched, trustworthy data assets for AI scaling | Source: Author</em></figcaption></figure><h4>2. The same entity exists in multiple forms across your systems</h4><p>When an AI agent queries your supplier database and returns three different risk scores for the same company, it is treating three slightly different name strings as three distinct entities.</p><blockquote><a href="https://www.moderndata101.com/blogs/entity-resolution-at-scale-deduplication-strategies-for-knowledge-graph-construction"><strong>📙 Read more:</strong></a></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/entity-resolution-at-scale-deduplication-strategies-for-knowledge-graph-construction"><strong>Entity Resolution at Scale: Deduplication Strategies for Knowledge Graph Construction</strong></a></blockquote><p>When <a href="https://www.moderndata101.com/blogs/how-ai-agents-data-products-work-together-to-support-cross-domain-queries-decisions-for-businesses">AI agents or retrieval systems query across enterprise data sources</a>, they assume a consistent representation of real-world entities: customers, suppliers, products, and contracts. Without it, they either miss connections that should be obvious or they fabricate coherence that doesn’t exist, pulling from what are effectively three different records for the same company and treating them as distinct.</p><p>This is where <a href="https://dataos.info/">entity resolution becomes load-bearing infrastructure</a>. The gap between a resolved and unresolved entity graph determines whether an AI agent can accurately answer “what is our total exposure to this supplier”, or whether it returns a number that’s confidently wrong.</p><figure><img alt="Diagram showing fragmented enterprise data resolved into a unified identity graph to ensure accurate AI results." src="https://cdn-images-1.medium.com/max/1024/0*gpkuxrrQiEoHFg3O" /><figcaption><em>The Identity Core: Eliminating fragmented identity data to ensure AI agents provide accurate, unified answers | Source: Author</em></figcaption></figure><p>What makes this tricky is that the issue often gets mistaken for an AI hallucination. The model gets blamed when the real failure actually starts much earlier in the pipeline: fragmented identity data that was never structured for <a href="https://dataos.info/products/data_product/">machine interpretation</a> in the first place.</p><p>Human analysts routinely bridge these gaps through context, experience, and institutional knowledge. But AI systems cannot rely on that kind of implicit understanding, which means inconsistencies that humans quietly work around become serious reliability problems at scale.</p><figure><img alt="Table comparing how human analysts use institutional context while AI relies strictly on explicit data." src="https://cdn-images-1.medium.com/max/1024/0*ngwxmLHjcjU0DiLs" /></figure><h4>3. There’s no ‘one’ semantic layer</h4><p>Different business units using different definitions of “revenue”, “active customer”, or “conversion” is an old problem. But in an AI context, it becomes unworkable. When <a href="https://www.moderndata101.com/blogs/reflecting-the-language-instinct-in-machines">building a knowledge graph or semantic layer</a> that AI agents can query reliably, terminological disagreement creates contradictory outputs that erode trust faster than any model limitation.</p><p>If your organisation hasn’t invested in a unified semantic layer, one that maps business terms to consistent logic, regardless of where the underlying data is, your AI systems will inherit all of that confusion and amplify it.</p><h4>4. Data lineage is incomplete or undocumented</h4><p>AI agents that act on data need to understand its provenance. When an AI agent acts on a data point, it needs to resolve questions that incomplete lineage leaves unanswered:</p><p>➡ Was this figure calculated, ingested, or manually entered?</p><p>➡ Was it transformed or aggregated before it reached this table?</p><p>➡ Does it reflect today’s state, or data from last quarter?</p><p>These decisions directly shape how an AI system handles information; whether it treats a value as reliable, flags it for review, or ignores it altogether.</p><p>Incomplete lineage also creates serious problems for <a href="https://www.moderndata101.com/glossary/audit-logs">AI governance and explainability.</a> Increasingly, regulators and enterprise stakeholders want visibility into how an AI system arrived at a decision and which data sources influenced it. When there are gaps in lineage, tracing those decisions back to their origin becomes difficult, if not impossible.</p><figure><img alt="Lineage diagram showing how data provenance enables explainable AI and supports regulatory compliance." src="https://cdn-images-1.medium.com/max/1024/0*0mNGu02FjoYE8tlq" /><figcaption><em>The Chain of Trust: Building data lineage to enable explainable AI and meet regulatory standards for decision-making | Modern Data 101</em></figcaption></figure><h4>5. Data access is fragmented and ungoverned</h4><p><a href="https://www.moderndata101.com/blogs/the-current-data-stack-is-too-complex-70-data-leaders-practitioners-agree">Enterprise data already lives in siloes</a> across cloud platforms, on-premise warehouses, SaaS applications, and operational systems. The real challenge is whether those environments are connected and governed in a way AI systems can reliably work with.</p><p>AI agents operating across federated data architectures need consistent <a href="https://datadeveloperplatform.org/architecture/">access control, data contracts, and clear ownership at the domain level</a>. Without these, you either lock AI systems out of the data they need or expose sensitive data to systems that shouldn’t have it. Neither outcome is acceptable in a production AI environment.</p><p>If data access is still handled through scattered permissions and one-off approvals managed separately by different teams, the environment is unlikely to support AI systems operating autonomously and at scale.</p><h4>6. Data quality problems are discovered late or not at all</h4><p>In traditional analytics workflows, data quality problems usually become visible only after something looks off in a dashboard or report. A team investigates, logs a ticket, and resolves it over time. The process is slow, but generally manageable.</p><p>In an <em>AI-driven pipeline</em>, that same <a href="https://www.moderndata101.com/blogs/ai-ready-data-a-technical-assessment-the-fuel-and-the-friction">data quality issue for AI</a> systems can propagate through a model, influence a decision, and reach a customer before anyone notices. The feedback loop is compressed, and the consequences are wider.</p><p>Data observability: <a href="https://www.moderndata101.com/glossary/alerting">real-time monitoring of freshness</a>, completeness, schema consistency, and distribution isn’t optional for AI-ready data infrastructure. If data quality processes only kick in after issues surface, the organisation is effectively operating with a structural blind spot: one that AI doesn’t fix, but instead makes more visible and more consequential.</p><h4>7. Your data assets aren’t packaged for machine consumption</h4><p>There is a meaningful and often underestimated difference between data that <em>exists</em> somewhere in your organisation and data that has been deliberately packaged for downstream consumption by analysts, by applications, and increasingly by AI systems.</p><p>A large share of enterprise data estates still sit in the first category (Tables exist, pipelines run, reports get produced). But the underlying assets carry no semantic context, no SLA on freshness, no documented schema contracts, and no access interface that doesn’t require bespoke integration work every time a new consumer appears.</p><p>Data products built with AI consumers in mind are architecturally different. They’re not just datasets; they’re <a href="https://www.themoderndatacompany.com/data-products">governed, versioned, observable units of data</a> that carry enough context for a downstream system to consume them reliably without human intervention.</p><p>Learn more about data products here.</p><blockquote><a href="https://www.moderndata101.com/blogs/what-are-data-products-the-complete-guide"><strong>📙 Read more:</strong></a></blockquote><blockquote><a href="https://www.moderndata101.com/blogs/what-are-data-products-the-complete-guide"><strong>The Complete Guide to Data Products</strong></a></blockquote><p>That means, at minimum:</p><p>➡ Embedded metadata and business definitions</p><p>➡ Documented schema contracts and versioning</p><p>➡ Defined ownership and access controls</p><p>➡ SLA-backed freshness guarantees</p><p>➡ Interfaces built for programmatic access, not manual query</p><figure><img alt="Comparison table illustrating the architectural shift from legacy stored assets to AI-ready data products." src="https://cdn-images-1.medium.com/max/1024/0*O9NxYFDKi4B8IgBp" /><figcaption><em>The Blueprint Shift: Why transitioning from raw stored assets to governed data products is a prerequisite for AI-native infrastructure | Source: Author</em></figcaption></figure><h3>The pattern behind all seven signs</h3><p>What connects these issues isn’t technology. It’s the absence of intentional data architecture, one designed around how AI systems actually work, not how human analysts have traditionally worked.</p><p>AI systems depend on context, consistency, trust, and accessibility. They require data to be designed as infrastructure, not just stored as assets. They also need governance models built for autonomous consumption patterns, not only human-driven querying.</p><p>The <a href="https://www.themoderndatacompany.com/data-ai-readiness-scorecard">organisations making real progress with enterprise AI</a> are not always those with the most advanced models. More often, they are the ones who invested early in getting their data foundations right before scaling anything else.</p><h3>FAQs</h3><h4>Q1: What is the difference between a data warehouse and a data lake?</h4><p>Data warehouses store highly structured, processed data for fast queries and reporting, while data lakes hold raw, unprocessed data in its native format, making them ideal for large-scale machine learning and AI projects.</p><h4>‍Q2: Why is data governance so important for AI?</h4><p>Data governance ensures data is secure, auditable, and compliant with regulations. It provides AI models with context and traceability, improving accuracy and accountability.</p><h4>Q3: Can we just let AI clean our messy data?</h4><p>No, AI can help, but foundational data issues must be addressed through proper data hygiene and standardised definitions before automation can be effective.</p><h4>Q4: Is our existing IT infrastructure ready for AI?</h4><p>Many systems aren’t prepared for AI’s demands; you need technical architectures with robust identity management and elastic compute to support secure, scalable AI</p><p>Thanks for reading Modern Data 101! <a href="https://medium.com/@community_md101">Follow us</a> for free to receive new posts and support our work. <strong><em>Originally published on </em></strong><a href="https://www.moderndata101.com/blogs/7-signs-your-data-is-not-ai-ready"><strong>Modern Data 101</strong></a><strong><em> Website.</em></strong></p><h4>MD101 Support ☎️</h4><p>If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at <strong>community@moderndata101.com</strong> 🧡</p><p><em>From MD101</em> <em>team</em> 🧡</p><p>A financial firm’s AI assistant ran for three weeks with zero errors and zero downtime, while quietly citing the wrong fund in every summary it wrote. This is the gap AI observability is built to catch, and this guide lays out the places enterprises need to watch for it: <a href="https://www.moderndata101.com/blogs/blogs-what-is-ai-observability-guide/">What Is AI Observability? Enterprise Stack &amp; Guide for 2026 | Modern Data Blog</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*LvejwwSltGClD2pK.png" /></figure><h3>.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=477c1fdcb089" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>