The supply chain problem is more serious than most leaders realize

Roughly 98% of applications today run on open source packages. Most of those packages are maintained by volunteers rather than full-time security professionals. The package ecosystem, NPM in particular, is increasingly being exploited as a primary attack vector, specifically because AI development teams are pulling dependencies quickly and with limited vetting of sources.

The attacks are growing more sophisticated. Recent exploits have targeted well-known, trusted packages as the entry point. But a more alarming pattern is also emerging: model poisoning, where attackers inject malicious packages into AI training pipelines so that the model learns to recommend or reproduce those packages as part of its output. The scanning-after-the-fact approach that most security teams rely on is too slow to catch it. The answer has to be secure-by-default package management and curated, verified sources, not just better detection.

AI-generated code has outpaced the traditional code review process

Traditional code review was designed for human-to-human handoffs: one engineer writes, another reviews, catches issues, provides feedback. That process doesn't hold up when AI is generating large volumes of code at speed. The volume alone changes the calculus, and the nature of AI-generated code creates failure modes that human reviewers weren't trained to look for.

Reviewing the prompts, not just the output, is an approach gaining traction. Knowing what you asked the AI to do gives you a basis for using a second AI to check whether the output actually matches those instructions. Interestingly, models are better at finding discrepancies between intent and output than they are at following complex instructions correctly in the first place. That asymmetry is something organizations can work with. Multi-model validation, where different models critique each other's work, is also proving useful in practice, with cross-model interrogation catching issues that single-model review consistently misses.

Agentic AI introduces a different category of risk

As organizations move toward agentic AI systems that take actions rather than just answering questions, the security surface changes significantly. Agents with access to private data, operating over untrusted sources, with the capability to exfiltrate information is the combination that security-focused teams at the table are most focused on. Any single one of those elements is manageable. All three together create a fundamentally different risk profile.

The governance posture that's emerging among teams that have thought this through carefully is to remove production access entirely where possible, so that no MCP connection can reach what a human couldn't access directly. Desktop agent access is being blocked in several organizations because browser behavior in that context is too unpredictable to govern reliably. Every MCP connection requires explicit security review and approval before it goes anywhere near a production environment. These controls feel restrictive, but they're the ones that emerge when teams take the actual attack surface seriously.

Real incidents, not hypotheticals

The table conversation went deep into specific incidents. A major SSO misconfiguration recently allowed unauthorized application additions at a well-known company, a basic configuration error with significant downstream consequences. Analysis of enterprise deployments has found that a substantial share of applications use sub-processors not listed in their Data Processing Agreements, meaning the legal documents governing data flows don't reflect what's actually happening in the infrastructure.

Fake hiring campaigns using sophisticated domain spoofing and company logo impersonation have increased dramatically in recent months, with nation-state actors actively working to infiltrate remote hiring processes. The countermeasure is uncomfortable but increasingly necessary: require in-person verification or real-time challenges that can't be replicated with a deepfake. And the Dropbox source code exposure illustrated something important about internet permanence: accidentally exposed source code was largely reconstructed within 48 hours. The assumption that exposure is temporary is wrong.

Board pressure is creating a security gap

Organizations are under intense board pressure to show AI adoption and cost reduction, and security is frequently absent from that conversation. The open experimentation posture that many companies have adopted creates an environment where shadow AI use is essentially inevitable: block access and employees will use personal credit cards instead. The productivity gains are too significant to give up voluntarily.

The answer isn't blocking adoption. It's building security into the adoption path from the start rather than treating it as a parallel track that will catch up later. Runtime security monitoring, with dynamic controls during AI execution rather than just before and after, is now a practical requirement. The before-and-after model was designed for a different era of software.

Accountability stays with the people, not the tools

AI tools don't hold accountability. People do. No matter how good the multi-agent review system, no matter how sophisticated the anomaly detection, the decision and the responsibility for it sit with a person. Building AI systems that obscure that accountability chain, where it becomes genuinely unclear who signed off on what, is a governance failure with real consequences.

The organizations getting this right are designing for accountability from the beginning: clear ownership, explicit human sign-off at critical steps, and audit trails that hold up under scrutiny. Not because regulators have caught up yet, but because the alternative is a security incident with no clean answer to who was responsible. The AI security challenges facing organizations right now are real and present. The question is whether leadership is treating them that way.

This article was adapted from a roundtable discussion at Operator Collective's Spring Gathering, held under Chatham House rules. Insights are shared without individual attribution.