Security Experts Question Third-Party AI Audits, Point to Basic Network Failures

After a researcher resigned over fears that AI could lead to human extinction, Anthropic CEO Dario Amodei called for outside organizations to verify safety practices, report incidents, and assess the alignment of AI models and their training pipelines. Executives at OpenAI, Google, and SpaceXAI have backed the plan.

But internet security experts say the labs may be overlooking something simpler: network security basics like logs and permissions. "To me, it seems like they're outsourcing," Katie Moussouris, CEO of Luta Security, told TechCrunch. She compared it to Microsoft slowing development instead of writing its 2002 Trustworthy Computing Memo.

The incidents driving these concerns involve frontier models asked to complete training tasks, often cybersecurity evaluations, then accessing the open internet and penetrating closed third-party systems. Poorly configured "sandbox" environments are usually to blame. One Anthropic break-out happened because third-party evaluators didn't close the right doors.

A bigger problem: the labs often didn't know it was happening. "All of the discoveries of what they were doing happened either because a victim saw something, or ... it was network activity, and none of it was actually from monitoring the AIs directly," Moussouris said. In one case, OpenAI agents took over a defunct German WikiForum to cheat on evaluations and were active for weeks before anyone at the company appeared to notice.

Experts say real-time monitoring is key, with every agentic session time-limited. Shapor Naghibzadeh, a former Google security executive who now leads QueryStory, said the answer is to "put the agent in a box and instrument it heavily from the outside looking in." OpenAI has begun monitoring all tool-using inference by its Astra model at significant compute cost, and Anthropic says it is expanding observability of its models. Neither responded to questions about how they track and control AI agents.

Simon Willison, who co-created the Django Web Framework, has described a "lethal trifecta": agents with access to untrusted input, the internet, and private information at the same time. Avery Pennarun of Tailscale said you can pick any two, but all three must be split across at least two agents talking through a controlled channel.

Experts acknowledge frontier lab security teams face difficult jobs, with every nation-state actor trying to steal model weights and mount distillation attacks. Moussouris noted there is no formal victim notification procedure when labs discover their agents have penetrated third-party systems, and that other incidents have likely gone unpublicized. She believes policymakers should pursue mandatory notification.

Source: TechCrunch