Strategies to Safeguard Emerging Technologies

As generative AI tools become embedded across email, chat and knowledge systems, they introduce a novel breed of cyber-threat: self-propagating “LLM worms” that spread not via malicious code, but through hidden prompts and prompt-injection attacks. This whitepaper surveys the latest research (including the Morris II proof-of-concept worm), real-world vulnerabilities (such as Slack’s AI leak incident and CVE-2024-5184 in EmailGPT), and emerging attack vectors across multi-agent AI frameworks. It then outlines a layered defense strategy—combining robust prompt filtering, policy-driven guardrails, retrieval-pipeline hardening, and AI-aware monitoring—and recommends enterprise tools (e.g., NeMo Guardrails, LLM Guard, WhyLabs, Lasso) to shore up your AI environment. Finally, it presents red-team scenarios to validate your controls and governance guidance to ensure AI-driven risks are managed at the boardroom level. By understanding these worm-class threats and adopting best practices now, organizations can harness LLM innovation securely—and stay one step ahead of attackers who aim to weaponize AI.

Microsoft's AI Red Team has released a groundbreaking paper titled "Lessons From Red Teaming 100 Generative AI Products" (https://lnkd.in/dGxsydwF) 🌎 Drawing from their extensive experience, they've distilled eight pivotal lessons for enhancing the safety and security of Gen AI systems:- 1. Understand what the system can do and where it is applied. 2. You don’t have to compute gradients to break an AI system. 3. AI red teaming is not safety benchmarking. 4. Automation can help cover more of the risk landscape. 5. The human element of AI red teaming is crucial. 6. Responsible AI harms are pervasive but difficult to measure. 7. LLMs amplify existing security risks and introduce new ones. 8. The work of securing AI systems will never be complete. 📌 Distinguish between Red teaming and safety Benchmarking - Red teaming involves simulating real-world attacks to uncover vulnerabilities, whereas safety benchmarking assesses performance against predefined standards. 🤖 Leverage automation - Utilizing tools like PyRIT can help cover a broader risk landscape more efficiently. 👭 Human judgment is irreplaceable - While automation aids the process, human expertise is essential for nuanced assessments and decision-making. 💭 Responsible AI harms are complex - Identifying and measuring harms require careful consideration, as they can be pervasive yet subtle. 👉 LLMs introduce new security challenges - Large Language Models can amplify existing risks and present novel ones, necessitating continuous vigilance. 👉 Security is an Ongoing Process - Ensuring the safety of AI systems is a continuous effort, demanding regular updates and assessments. 📜 This paper is a must-read for AI practitioners aiming to fortify their systems against emerging threats. #AI #GenerativeAI #AIResearch #RedTeaming #AIEthics #AITrust #MachineLearning #AIInnovation #AIRegulation #TechSafety #ResponsibleAI #CyberSecurity #AIProductDevelopment #AITrends #SafetyInAI

"The rapid evolution and swift adoption of generative AI have prompted governments to keep pace and prepare for future developments and impacts. Policy-makers are considering how generative artificial intelligence (AI) can be used in the public interest, balancing economic and social opportunities while mitigating risks. To achieve this purpose, this paper provides a comprehensive 360° governance framework: 1 Harness past: Use existing regulations and address gaps introduced by generative AI. The effectiveness of national strategies for promoting AI innovation and responsible practices depends on the timely assessment of the regulatory levers at hand to tackle the unique challenges and opportunities presented by the technology. Prior to developing new AI regulations or authorities, governments should: – Assess existing regulations for tensions and gaps caused by generative AI, coordinating across the policy objectives of multiple regulatory instruments – Clarify responsibility allocation through legal and regulatory precedents and supplement efforts where gaps are found – Evaluate existing regulatory authorities for capacity to tackle generative AI challenges and consider the trade-offs for centralizing authority within a dedicated agency 2 Build present: Cultivate whole-of-society generative AI governance and cross-sector knowledge sharing. Government policy-makers and regulators cannot independently ensure the resilient governance of generative AI – additional stakeholder groups from across industry, civil society and academia are also needed. Governments must use a broader set of governance tools, beyond regulations, to: – Address challenges unique to each stakeholder group in contributing to whole-of-society generative AI governance – Cultivate multistakeholder knowledge-sharing and encourage interdisciplinary thinking – Lead by example by adopting responsible AI practices 3 Plan future: Incorporate preparedness and agility into generative AI governance and cultivate international cooperation. Generative AI’s capabilities are evolving alongside other technologies. Governments need to develop national strategies that consider limited resources and global uncertainties, and that feature foresight mechanisms to adapt policies and regulations to technological advancements and emerging risks. This necessitates the following key actions: – Targeted investments for AI upskilling and recruitment in government – Horizon scanning of generative AI innovation and foreseeable risks associated with emerging capabilities, convergence with other technologies and interactions with humans – Foresight exercises to prepare for multiple possible futures – Impact assessment and agile regulations to prepare for the downstream effects of existing regulation and for future AI developments – International cooperation to align standards and risk taxonomies and facilitate the sharing of knowledge and infrastructure"

Japan’s Ministry of Economy, Trade and Industry (METI) has released an in-depth OT Security Guide for semiconductor device factories. This 132-page document outlines practical, globally-aligned strategies covering: ✅ Safeguarding production goals, confidential information, and semiconductor quality. ✅ Using NIST CSF 2.0 and the Cyber/Physical Security Framework (CPSF) for risk management. ✅ Factory security best practices based on IEC 62443 zones and microsegmentation. ✅ Special focus on asset inventory, vulnerability assessment, and tailored mitigation , not just patching. ✅ Preparing for nation-state threats, APTs, and modern supply chain risks. A must-read for OT, cybersecurity, and semiconductor industry pros looking to align with the latest global standards and strengthen factory resilience.

Based on my various work involving #AI and #cybersecurity in the #board and as a #strategic #advisor, #independent #director in various companies and as a #corporate #mentor and #guest #faculty at various global universities and colleges, AI is a game-changer, revolutionizing industries, but it's a #double #edged #sword. On one side, we see AI solving complex problems like protein folding, enhancing cancer screening, boosting farming efficiency, and speeding up drug discovery. These are massive wins for society. The flip side? AI poses serious risks. #1 #Humanrights are threatened by #deepfake #technology, #2 #safetyconcerns arise from #harmful #recommendations, and fairness in areas like #credit #assessments is questioned. #3 Privacy is compromised through #constant #surveillance, while #misinformation spreads, and #security #threats become more sophisticated. So, what should we do at the #board level? Boards must step up to #govern #AI and emerging technologies effectively. Here’s what they can do: - 𝗜𝗻𝗶𝘁𝗶𝗮𝘁𝗲 𝗕𝗼𝗮𝗿𝗱 𝗗𝗶𝘀𝗰𝘂𝘀𝘀𝗶𝗼𝗻𝘀: Consider #bluesky discussions to explore AI impacts without a rigid agenda. This helps define the board’s stance. - 𝗗𝗲𝘃𝗲𝗹𝗼𝗽 𝗔𝗜 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀: Post-discussion, creating a #policy on #AI #risk and #AIstrategy is essential. This document needs regular updates to stay relevant. - 𝗔𝘀𝘀𝗶𝗴𝗻 𝗢𝘃𝗲𝗿𝘀𝗶𝗴𝗵𝘁: A dedicated committee should oversee AI's role within the organization, ensuring informed #decisionmaking on projects and risks. - 𝗖𝗿𝗲𝗮𝘁𝗲 𝗔𝗜 𝗜𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻 𝗛𝘂𝗯𝘀: Setting up units like #skunkworks or #incubators can trial #innovative #AI #technologies safely. - 𝗣𝗿𝗼𝗺𝗼𝘁𝗲 𝗔𝗜 𝗔𝗱𝘃𝗼𝗰𝗮𝗰𝘆: Building a network of AI #evangelists across the business can champion understanding and responsible use within the organization. In #cybersecurity, AI is both a tool and a threat. It’s used for real-time #threatdetection, yet #zerotrust models, #regulatorydevelopments, and a rise in demand for AI-skilled #professionals highlight the need for vigilance. Shorter Loop Unified Product Management Community Ultimately, success in AI can be historic. we must address these inherent risks responsibly, ensuring a beneficial, secure future for generations.

This document serves as a comprehensive workbook for addressing AI safety within public sector projects. 1️⃣ It introduces four key objectives for ensuring AI safety: performance, reliability, security, and robustness, emphasizing their importance in real-world AI applications. 2️⃣ Activities and strategies outlined help assess risks, ensure technical integrity, and mitigate failures throughout the AI project lifecycle, including design, development, and deployment stages. 3️⃣ The workbook emphasizes stakeholder engagement, iterative testing, and documentation as critical practices for managing AI safety. 4️⃣ Specific risks like data poisoning, adversarial attacks, model drift, overfitting, and non-deterministic behaviors are highlighted, with mitigation strategies tailored to each risk type. 5️⃣ Tools such as safety self-assessments and risk management plans are provided to embed safety assurance into ongoing AI operations. 6️⃣ The document also serves as a resource for training civil servants and technical teams on applying these principles effectively in public sector contexts. ✍🏻 David Leslie, Cami Rincón, Morgan Briggs, Antonella Maia Perini, Smera Jayadeva, Ann Borda, SJ Bennett, Christopher Burr, Claudia Fischer. AI Safety in Practice. The Alan Turing Institute. 2024.

As a member of The Aspen Institute's International Cyber Group, I am pleased to share our inaugural policy report examining the intersection of #GenAI regulation and #Cybersecurity. As governments and businesses around the world grapple with what #AI means for them, this report makes an important and timely contribution by offering targeted practical recommendations to policymakers and business leaders, including: 🙋♂️ Start with the end user in mind: clear objectives are required before taking action - understand the values and outcomes you’re looking for beyond just mitigating risks or minimising harms. ⚖ Assess criminal and civil liability: current laws were written way before GenAI existed or imagined, and need to be updated to account for this significant technology shift. 🚨 Consider technology safeguards and feasibility: the full uses and applications of this tech are not easily defined, therefore regulatory and legal safeguards need to be flexible to keep up. 🚦 Establish standards: they will be the operational bedrock for AI and will define the future landscape of innovation. What should governments avoid: 🏷 Creating consent fatigue: unified labelling schemes indicating the presence of AI-generated content will make misrepresentation easier to police. 🛠 Mistaking actions for results: existing tools should be leveraged to help address the most pressing security concerns and give appropriate time to consider risks that materialise. 🔓 Ignoring the openness of generative AI tool access: governments should carefully consider which parts of these technologies can or should be open to the general public. Congratulations to my fellow group members David Koh Cyber Security Agency of Singapore (CSA) Corey Thomas Jorge Guajardo Yasmin Brooks Katie D'Hondt Brooks Jeff Greene Dmitri Alperovitch Paul Ash Carl Bildt Christophe Blassiau Cecilia Bonefeld-Dahl Inge Bryan Paolo Dal Cin Oleh Derevianko Anriette Esterhuysen Camille François Stewart Garrick Katherine Getao Dario Gil Ron Green Jane Horvath Mikko Hypponen Chris Inglis Marina Kaljurand Boon Hui K. Tzipi Livni Ciaran Martin Christopher Painter Guillaume Poupard Michelle Price Latha Reddy Runa Sandvik Rob Strayer Eli Sugarman Yigal I. Unna Phil Venables Grant Verstandig Alicia Wanless Alberto Yépez Shinichi Yokohama Check out the full report and let me know what you think:

UK AI Security Institute: Principles for #Evaluating #Misuse #Safeguards of #Frontier #AI Systems Misuse safeguards—technical interventions implemented by frontier AI developers to prevent users from eliciting harmful information or actions from AI systems—are an important tool in addressing potential risks from the misuse of these systems. In many parts of machine learning, establishing clear problem statements and evaluations drives and accelerates progress, and this same lesson applies to safeguards. To this end, the AI Security Institute proposes five principles for rigorous evaluations of misuse safeguards, which form a step-by-step plan for safeguards assessment. The Institute additionally releases a lightweight template designed to enable developers to draw from our recommendations as they perform safeguards assessment. These documents aim to drive standardisation and rigour in how safeguards evaluations are performed, which will become increasingly important as AI capabilities advance. 📍 Recommendation 1: Clearly State Safeguard Requirements 📍 Recommendation 2: Establish a Safeguards Plan 📍 Recommendation 3: Collect & Document Evidence of Safeguard Sufficiency 📍 Recommendation 4: Establish a Plan for Post-Deployment Assessment 📍 Recommendation 5: Justify Whether the Evidence and Post-Deployment Assessments Plan are Sufficient

America's AI leadership is driven by the private sector, and the Trump administration has prioritized protecting this advantage. Yet AI’s dual-use nature means the same breakthrough capabilities driving economic growth can also enable novel national security capabilities and threats – issues squarely within the domain of the federal government. With technical expertise concentrated in a handful of companies and capabilities advancing rapidly, how can the government prepare for emerging risks without undermining the innovation edge that defines U.S. leadership? In our report "Prepared, Not Paralyzed: Managing AI Risks to Drive American Leadership" Spencer Michaels, Caleb Withers and I outline three key capacities that can help the U.S. government to identify, understand, and respond to AI risks: Situational awareness to detect, analyze, and communicate emerging AI risks and opportunities.  Agile policymaking that can adapt and scale proportionately to evolving threats; and Incident response and readiness to manage and contain significant AI-related incidents should they arise. Our report outlines practical recommendations to establish and strengthen these capacities, outlining how investments in government readiness can promote, rather than hinder, innovation. You can read our report here: https://lnkd.in/efG7xU3A Center for a New American Security (CNAS)

Global scientific collaboration is more interconnected than ever. But in high-stakes fields like #AI, #quantum , biotech, and aerospace, this openness comes with growing vulnerabilities - from economic exploitation to espionage. We need to balance scientific progress with #nationalsecurity. To help navigate this, my colleagues at RAND and I recently co-authored a new paper outlining a comprehensive Research Security Framework designed to help researchers, institutions, and funders manage risks. It's built on three core pillars: Protection: Safeguarding the results and products of your research. Transparency: Ensuring clear disclosure requirements. Governance: Implementing strong due-diligence and cybersecurity protocols. Without these safeguards, international partnerships can easily be misused. Protect your innovation.