Regulatory Challenges for European AI Startups

Europe’s overzealous regulations are strangling medical AI. The EU AI Act’s “high-risk” label and Medical Device Regulation bury startups in costly, slow compliance, delaying tools that catch cancers early, optimize emergency care, or aid mental health—tools already saving lives in the U.S. In 2023, U.S. healthcare AI startups raised $6-8 billion, powering 171 FDA-cleared tools. Europe’s digital health sector limped to €3.1 billion, with AI at €1-1.5 billion, starving innovation. Europe churns out 20% of global AI research but grabs just 8-10% of startup funding, losing talent to the U.S. and China, which poured $10-12 billion into healthcare AI. Take mental health: U.S. firms like Kintsugi use voice AI to spot depression instantly. In Europe, startups like Berlin’s MindDoc stall, bogged down by 12-18 months of red tape. Europeans end up using American apps—unregulated, no local innovation. The U.S. balances safety and speed with fast-tracked AI approvals and post-market checks. Europe’s rules just choke progress, delaying tools that could raise breast cancer survival by 5-7%. Fix it: fast-track low-risk AI, use UK-style regulatory sandboxes, streamline AI Act-MDR overlap. Without reform, Europe’s doctors and engineers—and patients—lose out.

It is always great to see something published that you have worked on for a considerable amount of time – but in this case, it feels really special. AI Act, GDPR, DSA, finance, medical devices, automotive regulation: so many things close to my (academic) heart, and I could combine them all in one study on the frictions, interdependencies, and ways forward through this regulatory jungle. Here are the key policy recommendations, structured by addressees, many more in the study, someone counted 25 :). Important: Almost all of them can be achieved without any diminished protection of fundamental rights.   European Legislators   1. Designate a "Lead Act": Assign a leading regulatory framework for each sector, such as the AI Act or sector-specific laws, to reduce conflicts and enhance coherence. If that Lead Act is complied with, compliance of the other designated acts should be presumed, unless some specific provisions are exempted from that rule. Example: Art. 17(4) AI Act, one of my favorite norms in the Act, a hidden gem ;)   2. Clarify AI Act-GDPR Alignment: Address contradictions, such as differing responsibilities for AI providers under the AI Act and data controllers under the GDPR, and rules for training AI on personal data.   3. Develop Safe Harbor Standards: Create technical standards that provide compliance with the AI Act AND related regulations.   4. Conduct Regular External Reviews: Periodically and EXTERNALLY evaluate the AI Act's implementation to address contradictions, regulatory gaps and new technological challenges.    European Commission (AI Office and Sectoral Authorities)   5. Enhance Risk Analysis for Hybrid Platforms: Develop integrated guidelines for platforms that incorporate generative AI, addressing systemic risks under both the AI Act and the DSA, and the mutual reinforcement of the specific platform and GenAI risks.   6. Expand Data Access for Research: Establish mechanisms for vetted researchers to access both platform AND AI system data, inspired by the DSA’s Article 40.   National Legislators and Authorities   7. Support SMEs: Introduce grant programs to help small and medium-sized enterprises comply with AI Act and sector-specific regulations. This could, for example, fund access to training programs.   8. Foster Oversight Synergies: Clearly institutionalize the necessary collaboration between national data protection, sectoral and AI Act oversight authorities for cohesive enforcement. Be agile and project-based in solving cases involving multiple Acts.   Standardization Bodies   9. Develop Unified Standards: Provide technical standards for the AI Act AND sectoral regulations.   Industry and Civil Society   10. Encourage Cross-Disciplinary Collaboration: Establish advisory groups combining industry, academic, and civil society expertise and liaising with the national AI authorities to address sector-specific challenges.   Many thanks to Bertelsmann Stiftung, Julia Gundlach and Asena Soydas for enabling this!        

A new paper dropped today that deserves serious attention from anyone building or deploying AI agents in Europe. Nannini, Smith, Tiulkanov and colleagues have produced the first systematic regulatory mapping for AI agent providers under EU law. Not a policy commentary. An actual compliance architecture, integrating the draft harmonised standards under M/613, the GPAI Code of Practice, the CRA standards programme, and the Digital Omnibus proposals. The core insight is deceptively simple: the regulatory trigger for an AI agent is determined by what the agent does externally, not by its internal architecture. The same LLM with tool-calling generates radically different compliance obligations depending on deployment. → Screen CVs? Annex III high-risk, full Chapter III → Summarise meeting notes? Article 50 transparency only. The technology is identical. The regulatory consequence diverges completely. The paper identifies four agent-specific compliance challenges that current frameworks address in principle but not yet in practice. 1️⃣ 𝗖𝘆𝗯𝗲𝗿𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆: a system prompt telling the model "do not delete files" is not a security control. Article 15(4) compliance requires privilege enforcement at the API level, outside the generative model. 2️⃣ 𝗛𝘂𝗺𝗮𝗻 𝗼𝘃𝗲𝗿𝘀𝗶𝗴𝗵𝘁: LLMs trained via RL may have learned to evade oversight as an emergent strategy. Oversight must be external constraints, not internal instructions. 3️⃣ 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝗰𝘆: when an agent sends an email, the recipient is an affected person who may not know they are interacting with AI. 4️⃣ 𝗥𝘂𝗻𝘁𝗶𝗺𝗲 𝗯𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝗮𝗹 𝗱𝗿𝗶𝗳𝘁: agents that accumulate memory or discover novel tool-use patterns may leave their conformity assessment boundaries undetected. The paper's conclusion is stark: high-risk agentic systems with untraceable behavioral drift cannot currently be placed on the EU market. Not future risk, but current legal position. For anyone building AI governance infrastructure, this confirms what we have been arguing at Modulos: compliance for agentic AI must be continuous and architectural, not periodic and checklist-based. The provider's foundational task is an exhaustive inventory of the agent's external actions, data flows, connected systems, and affected persons: that inventory is the regulatory map.  👉 https://lnkd.in/e_zk3R6B

The EU AI Act faces a fundamental challenge: it was designed for tools, not agents Christine Saloustrou's paper "Agentic AI Regulation under the AI Act: A Proof of Concept or a Concept to Prove?" maps the gap between what the regulation assumes and what autonomous AI actually does. The Act treats AI systems as having fixed purposes defined by their providers. You classify the risk, implement controls, done. But agentic AI doesn't work that way. An agent designed to "assist users" might autonomously decide to use a high-risk tool—like a credit scoring API—to achieve that goal. Who's liable? The agent's creator, the API provider, or the user who gave it the high-level objective? Saloustrou calls this the autonomy gap. The regulation expects static input-output relationships. Agents set their own goals, use tools, and adjust strategies in real time. Three places where this breaks down: -The Act's risk classification depends on knowing an AI system's intended purpose upfront. Agents are inherently multi-purpose and adaptive. They don't fit cleanly into risk buckets when they're making autonomous decisions about which capabilities to deploy. -Article 14 requires "meaningful human oversight" to prevent or minimize risks. But what does meaningful oversight look like when agents operate in milliseconds through reasoning chains humans can't monitor in real time? You end up with automation bias—humans rubber-stamping decisions they can't actually evaluate. -Article 13 mandates transparency for high-risk systems. When an agent iterates and changes its plan halfway through a task, providing clear explanations before the fact becomes impossible. The system's logic emerges from execution, not from predetermined rules. What this means for practitioners: The Act's technology-neutral framework was supposed to handle any AI advancement. That works for incremental improvements to existing tools. It struggles with systems that fundamentally operate differently—setting goals, deploying capabilities, and adapting strategies without step-by-step human instruction. Saloustrou suggests the Act will need dynamic monitoring instead of one-time conformity assessments. Continuous real-time auditing of agent behavior, not static reviews of predetermined functionality. Regulatory sandboxes to observe how agents behave in controlled environments before wide deployment. Clearer definitions of who's responsible when an agent creates its own downstream risks. The paper serves as early warning: the EU AI Act is a landmark achievement, but its static nature may struggle with the fluid, autonomous nature of what's coming next. For the full story, and to learn more about AI, Innovation and the law, click on the website in my bio.

🚨 The EU AI Act is Coming in 2025 – Will Your AI Pass the Test? In 2025, the EU AI Act will introduce tough new standards for AI, focusing on safety, ethics, and transparency. Recent research from ETH Zürich has shown that most AI models are not ready for these regulations, which could have serious consequences for companies using them in Europe. To help prepare, ETH Zurich developed COMPL-AI, an open source tool that evaluates popular language models against the EU’s upcoming requirements. It’s like a report card for AI, measuring how well models handle things like security, bias, and openness. https://lnkd.in/gzvSUZFy The results? There’s room for improvement: 1. Top Performers: OpenAI’s GPT-4 Turbo and Anthropic’s Claude 3 scored highest on ethical principles, technical requirements, and robustness. They were more resilient to adversarial inputs and demonstrated better compliance with transparency and data governance expectations. 2. Lowest Scorer: Meta’s LLaMA 2-7B Chat performed poorly, particularly in non-discrimination and fairness. It struggled with maintaining transparency and robustness, raising concerns about its readiness for compliance with the EU AI Act. 3. Common Challenges: Across all models, even the highest scorers, there were notable deficiencies in areas like non-discrimination, fairness, and content tracking (e.g., watermarking AI-generated content). These gaps highlight the need for improvements before the 2025 regulations. Bottom Line This report underscores that no current model fully meets the EU AI Act’s standards, stressing the urgency for developers to address ethical and technical compliance issues to avoid future regulatory challenges. Why does this matter? For companies, failing to meet these standards could mean losing access to the EU market, damaging reputations, and facing potential fines. This research is a reminder that responsible AI development isn’t just a “nice-to-have” – it’s essential to stay competitive in a regulated future. See full report here: https://lnkd.in/gYaRKSAq

"This white paper offers a comprehensive overview of how to responsibly govern AI systems, with particular emphasis on compliance with the EU Artificial Intelligence Act (AI Act), the world’s first comprehensive legal framework for AI. It also outlines the evolving risk landscape that organizations must navigate as they scale their use of AI. These risks include: ▪ Ethical, social, and environmental risks – such as algorithmic bias, lack of transparency, insufficient human oversight, and the growing environmental footprint of generative AI systems. ▪ Operational risks – including unpredictable model behavior, hallucinations, data quality issues, and ineffective integration into business processes. ▪ Reputational risks – resulting from stakeholder distrust due to errors, discrimination, or mismanaged AI deployment. ▪ Security and privacy risks – encompassing cyber threats, data breaches, and unintended information disclosure. To mitigate these risks and ensure AI is used responsibly, in this white paper we propose a set of governance recommendations, including: ▪ Ensuring transparency through clear communication about AI systems’ purpose, capabilities, and limitations. ▪ Promoting AI literacy via targeted training and well-defined responsibilities across functions. ▪ Strengthening security and resilience by implementing monitoring processes, incident response protocols, and robust technical safeguards. ▪ Maintaining meaningful human oversight, particularly for high-impact decisions. ▪ Appointing an AI Champion to lead responsible deployment, oversee risk assessments, and foster a safe environment for experimentation. Lastly, this white paper acknowledges the key implementation challenges facing organizations: overcoming internal resistance, balancing innovation with regulatory compliance, managing technical complexity (such as explainability and auditability), and navigating a rapidly evolving and often fragmented regulatory landscape" Agata Szeliga, Anna Tujakowska, and Sylwia Macura-Targosz Sołtysiński Kawecki & Szlęzak

The EU AI Act demands technical teeth; are we ready yet? I’ve been digging into the COMPL-AI Framework paper from ETH Zürich and LatticeFlow AI, and it crystallizes something I’ve been saying in my AI governance work: regulation without measurable technical standards is just aspiration on paper. The researchers built the first comprehensive technical interpretation of the EU AI Act for LLMs; translating broad regulatory language into 27 concrete benchmarks across robustness, privacy, fairness, transparency, and safety. Then they evaluated 12 prominent models including GPT-4, Claude 3, and Llama 3. The verdict? No model is fully compliant. Not one. Three findings that should concern every builder deploying AI in regulated environments:- 1. Capability ≠ Compliance. Models that score well on knowledge and reasoning benchmarks still fail on fairness, robustness, and traceability. Qwen1.5-72B scores 0.71 on capabilities but just 0.37 on fairness. We’ve been optimizing for the wrong things. 2. Our benchmarks have blind spots. Current privacy and copyright evaluations are too simplistic to be meaningful; most models score near-perfect not because they’re compliant, but because the tests can’t detect violations. Explainability? No adequate technical benchmark even exists yet. 3. Small models carry disproportionate risk. Smaller LLMs consistently underperform on robustness and cyberattack resilience. As organizations rush to deploy lightweight models for cost efficiency, they may be trading compliance for convenience. For those of us building agentic AI systems, this has profound implications. When autonomous agents chain multiple LLM calls together, these individual model gaps compound. A fairness score of 0.50 at the model level becomes a systemic risk at the orchestration level. This is exactly why at COHUMAIN Labs, our Joint Evaluation (Jo.E) framework integrates this kind of regulation-aligned benchmarking directly into users’ agentic AI workflows, combining LLM-as-a-judge, specialized AI agents, and human expert validation in a tiered assessment structure so compliance isn’t an afterthought but is embedded by design. Where COMPL-AI maps the “what” needs to be measured, Jo.E operationalizes the “how” for teams building and deploying agentic systems in the real world. What excites me about COMPL-AI is that it proves regulation-aligned benchmarking is possible, even if imperfect. It’s the kind of bridge between policy intent and engineering practice that the GPAI Code of Practice desperately needs. The EU AI Act enforcement deadlines are approaching. The question isn’t whether your models will be evaluated against technical standards; it’s whether you’ll be ready when they are. Full paper:- arxiv.org/abs/2410.07959 Open-source suite:- compl-ai.org #AIGovernance #EUAIAct #ResponsibleAI #LLM #AgenticAI #AICompliance #JoE #COHUMAIN

Harmonized #standards play one of the most important, yet underappreciated roles in #AIAct implementation. I am therefore particularly excited to present today's #sundAIreads on "European #AI Standards - Technical Standardization and Implementation Challenges under the EU AI Act" by Robert Kilian, Linda M. Jaeck, and Dominik Ebel. The paper can be downloaded here: https://bit.ly/41J5a8E. 💡 Harmonized standards matter because they facilitate regulatory implementation. In the case of the AI Act, products conforming to relevant standards are presumed to comply with the law. This, in turn, enables companies to attach a CE (conformité européenne) marking to their products, paving the way for accessing the EU market. In theory at least. In practice, companies cannot currently fully benefit from harmonized standards because: ❌ The standardization committee's work has been progressing slower than expected, leaving providers of high-risk systems with insufficient time to implement the standards before they need to demonstrate compliance in August 2026. ❌ Large enterprises can more easily participate in the process of standardization than SMEs, startups, civil society, and academia. ❌ The process of standardization suffers from a lack of technical expertise. ❌ It currently remains unclear whether the standards will be freely available. ❌ Standards cannot easily be operationalized, as they are primarily available as PDFs. The authors interviewed >20 organizations across industries. The findings: 💡 #Healthcare and #MedTech organizations are generally well prepared, "effectively leveraging their MDR compliance experience" to incorporate the AI Act’s requirements into existing processes. 💡 The #manufacturing sector can also leverage "close alignment between technical standards and established frameworks." Companies may, however, struggle with "maintaining comprehensive documentation for AI-driven decisions in high-speed production environments." 💡 #LegalTech companies face multiple overlapping compliance requirements when handling sensitive client data. But they also see an "opportunity to establish themselves as leaders in ethical AI practices." 💡 Startups in #FinTech and other industries worry that complex compliance requirements will disproportionately burden smaller firms. 💡 Spillover effects are also expected for #mobility and #defense, even though they fall outside the AI Act's immediate scope of application. The authors conclude with a series of policy recommendations, including: 1️⃣ Adjust timelines for implementation; 2️⃣ Lower barriers to participation; 3️⃣ Provide practical support for implementation; 4️⃣ Actively incorporate SMEs into standard development; 5️⃣ Ensure better alignment between horizontal and vertical standards. I highly recommend reading the paper in full, as it is an excellent resource for policymakers and practitioners alike. Kudos to the authors for this important and timely contribution.