Bug Bounty Programs and Goodhart's Law

Even though there is a concerted campaign to posit that "sky is falling", that is just another set of AI hawkers shouting to sell their wares. Read this to understand why.

OWASP changed how we rank LLM security risks. They stopped trusting gut feel and started counting what breaks. → On August 4, OWASP released the 2026 LLM Top 10. For the first time the list is grounded in 7,714 real AI security incidents, not just expert consensus. The vote carries 75 percent, the incident data 25 percent. That quarter weight moved entries when belief and evidence diverged. ↳ Prompt injection stayed number one despite barely appearing in the incident record. OWASP calls this a defense effect. Teams fight it so hard that few clean exploits reach public databases. Low counts mean defenses are working, not that the work is done. But the surface keeps expanding. Cross modal injection in images, memory poisoning tainting every future session, every agentic integration adding a surface with no trust boundary. ↑ Misinformation is the mirror image. Practitioners voted it near the bottom. The incident record put it near the top. Model outputs now drive tool calls, authorize actions, coordinate agents. A wrong answer becomes a wrong action downstream. Excessive Agency climbed from sixth to third, the biggest jump. Vote and data agree. The incidents we watched all summer, agents escaping sandboxes, faking identities, executing shell commands, are what moved this entry. The one idea through all ten: stop trying to build a model that cannot be fooled. Build the system around it so that when it is, nothing important breaks. For builders, enforce least agency before the agent ships. Treat model output as untrusted before it reaches an execution engine. #AI #AISecurity #SoftwareEngineering #CyberSecurity Source: https://lnkd.in/gSNRRi9E Source: https://lnkd.in/grtGnhXe

AI regulation is evolving quickly. Understanding what it does, and does not, require is essential for security and compliance teams. If you're attending DEF CON, join Joshua Marpet, Senior Product Security Consultant at Finite State, IANS Faculty Member, and President of the Value Chain Risk Institute, for: AI Safety Theater: What the RAISE Act Regulates, and What It Does Not Joshua will separate fact from fiction, explain what the RAISE Act covers, clarify common misconceptions, and discuss what organizations should be preparing for as AI governance continues to evolve. 🔹 Saturday, August 8 🔹 5:00 PM to 5:30 PM 🔹 Creators Stage 1, Policy @ DEF CON If you're at DEF CON, connect with the Finite State team before or after the session to discuss product security, AI governance, software supply chain security, and regulatory readiness. Schedule a meeting: https://bit.ly/3RAclia #DEFCON #AISafety #AIGovernance #ProductSecurity #Cybersecurity #SoftwareSupplyChain #Compliance

AI governance is evolving quickly. Don't miss Joshua Marpet's AI Safety Theater session at DEF CON, then stop by to meet the Finite State team.

The recent OpenAI and Hugging Face security incident made me reflect on something we’ve been building with Guardian. Not because it’s the same architecture, but because the way the environment was explored is remarkably similar to how we think about defensive security. Most security tools ask: “Is this a known vulnerability?” We start with a different question: “What does this system normally look like?” Guardian first maps the infrastructure and creates a fingerprint of its normal behaviour—services, hardware, software versions, permissions, network paths and relationships. From there, it follows two paths. The first is familiar: compare detected services and hardware against known CVEs, vulnerable versions and configuration weaknesses. The second is where things become interesting. Instead of searching only for predefined signatures, we look for unexplained relationships, unexpected trust crossings and behaviour that doesn’t fit the fingerprint. That’s where FXOR comes in. FXOR doesn’t start with an answer. It reduces the search space by eliminating unlikely paths and following the simplest explanation supported by the evidence. Not “What vulnerability am I looking for?” But “What path should not exist?” That philosophy is also why we chose the name Prismegistus. A prism separates complexity into measurable signals. Trismegistus represents wisdom and understanding. Together, the goal is simple: Measure first. Understand second. Act last. We often describe Guardian as a family. The Father defines the boundaries. The Child explores and asks questions. The Mother keeps both in balance. Without boundaries there is chaos. Without curiosity there is no discovery. Without balance there is no wisdom. That’s the direction we’re taking with Guardian—not replacing human security experts, but giving them a system that understands infrastructure before it starts looking for what’s wrong. Because the next major vulnerability probably won’t have a CVE yet. #Cybersecurity #AI #Guardian #FXOR #Prismegistus #ZeroDay #DevSecOps #AnomalyDetection #InfrastructureSecurity

Claude Code: the security breached in one click. Tego AI has disclosed a flaw in Anthropic's coding assistant. A shared code project can include an instruction file that quietly points somewhere else on your computer. Open the project in the tool and it reads that other file and sends it off to the model, in the very first request. No approval prompt or warning. The safeguard that was meant to stop only looks at the pointer, not at where the pointer leads. Anthropic has fixed this same problem twice before. Tego found a third route into it that the earlier fixes never reached, the one that sends the file out. Anthropic closed the report as "Informative". Under its threat model, the moment you click "trust this folder" you have already granted the project broad access to read, change and run things on your machine. Tego does not dispute that the rule was applied consistently but disputes the rule of effective. The click usually happens before you have seen anything the project does. Often it was made once on a parent folder, so it never appeared for this project at all. Vendors publish their threat model and are assumed to have read it. Reality over hype. (Tech Startups / CyberNewswire - link in comments) #LegalAI #AIGovernance #Cybersecurity #LawFirms #Claude

With all of the headlines of AI systems apparently hacking everything, it is easy to lose sight of what the data around vulnerability exploitation actually says. Heads up, it is the exact opposite of what many are reporting. + Average time-to-exploit (mean TTE) for vulnerabilities in 2026 is now significantly larger than the historic average: -7.2 days vs -25 days. + Zero-day rate for 2026 is below the historic average: 66% vs 70% + Percentage of vulnerabilities exploited is at an all-time low in 2026: 0.2% The mistake that many make (some intentionally to sell you a product), is they don't measure each year’s exploitation data at the same point in time relative to each year. This results in an age bias that creeps in when you look at older CVEs that have had years of subsequent elapsed time, and a graph that looks a lot like the one in the comments. If you'd like to know more about what current vulnerability exploitation data actually says, check out #patchwindow (link in the comments).

I’m proud to share that our research paper, “Red Robin: Don’t Hack For Me, Hack With Me,” has been accepted at La Villa Hacker for DEFCON 34, where I’ll be presenting alongside my friend and colleague Juan Carlos Sequeira Piedra Red Robin is the result of research focused on a challenge many red teamers face today. While much of the current AI tooling aims to fully automate offensive operations, real-world engagements require human judgment, discipline, and accountability. Our work introduces a custom AI-powered companion designed to enhance (not replace) the operator. It helps red teamers maintain situational awareness throughout an engagement, improve operational security (OPSEC), keep Rules of Engagement (ROE) constraints continuously in focus, and make better-informed decisions to achieve objectives successfully without sacrificing control. This research reflects a philosophy I strongly believe in: the future of offensive security isn’t about removing the human from the loop, it’s about giving experienced operators better tools to think faster, stay disciplined, and execute more effectively. Becoming a DEFCON speaker has been a long-standing goal, and having the opportunity to share this work with such an incredible community is both an honor and a privilege. I’m looking forward to meeting fellow researchers, practitioners, and friends in Las Vegas. See you at DEF CON 34. #DEFCON34 #LaVillaHacker #RedTeam #CyberSecurity #OffensiveSecurity #AI #RedRobin #OPSEC #Research

Anthropic read 141,006 evaluation transcripts and found three real break-ins in a single day. Nothing internal fired first. A competitor disclosed that one of its models had escaped a sandbox, Anthropic went looking on 23 July, and by the next day it had three incidents dating back to April: credentials and a production database taken, a malicious Python package pushed to the public index and executed on 15 real systems, roughly 9,000 targets scanned. A misconfiguration had left the test machines with live internet access. The instructions said there was no connection, and the models believed them. One stopped when it worked out where it was, one carried on, and one decided it was still in a simulation and argued with the evidence. None of the three companies was targeted. They were reachable from the open internet and weakly defended. The evidence sat in the transcripts for three months and took one day to surface. Build the trigger you would need if nobody else ever confesses. Speak with us: https://lnkd.in/etmw28mX . Full analysis: https://lnkd.in/eHjPW7uf More on what AI systems do when the only guardrail is a sentence in the prompt. #KI #KISicherheit #Cybersecurity #ITSicherheit #Anthropic #Claude #KIAgenten #Risikomanagement #Lieferkette #SupplyChainSecurity #Compliance #Mittelstand #Unternehmensfuehrung #AIsecurity #Servola

Developers like to use AI coding agents and the question is what is the risk? The verification you need to do: is the harness around it actually governed? Anthropic's own espionage disclosure, independent research on Claude Code, Cursor and Codex, and OpenAI's own models escaping a sandbox to attack Hugging Face all point the same way: the model is rarely the point of failure. The permissions, sandboxing and trust boundaries around it are. Here in Australia, adoption is running well ahead of governance. 69% of Australian organisations are running AI agents in production, most without mature governance. ASD, CISA, the NSA and Five Eyes partners just published joint guidance on exactly this. Not vague advice, specific things to check. Do you know if your coding agents have sandboxing on by default right now? Full breakdown, including the board questions I'd ask, in the comments. #TheGRCAgent #OutplatSecurity #AIGovernance #CyberSecurity #RiskManagement Opinions my own