Anthropic CEO Calls for Government Authority to Halt Unsafe AI Releases

Anthropic CEO Dario Amodei is making the case that governments should have the power to stop dangerous AI deployments when those systems fail to meet established safety standards. In an essay published on his personal site, Amodei built on his company's well-known safety-first stance, a position that has increasingly put it at odds with both rivals and the Trump administration this year.

Amodei proposed that frontier AI models should be subject to the same kind of rigorous testing and auditing that the Federal Aviation Administration applies to aircraft. "Their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety," he wrote. He went further, calling for mandatory third-party audits to assess risks in cybersecurity, biological weapons, AI system control failures, and automated research and development. Under this framework, government agencies themselves could serve as the auditors - a break from other AI companies that warn against overreach.

While Amodei praised President Trump's recent executive order on AI testing, he argued it didn't go far enough. That order set up voluntary pre-release testing for frontier models but stopped short of making it mandatory. For Amodei, the next step is clear: give the government legal authority to block or reverse deployments when independent evaluations flag serious risks.

Anthropic's relationship with the Pentagon has been particularly rocky. Defense Secretary Pete Hegseth recently labeled the company a supply chain risk and banned the use of its Claude model across the Department of Defense, a move Anthropic challenged in court. The company demanded that its technology not be used for fully autonomous lethal weapons or mass surveillance of Americans; the Pentagon insisted on the right to use Claude for "all lawful uses."

Just weeks after filing its lawsuits, Anthropic released Mythos, its most advanced cybersecurity AI model, which sent shockwaves through Washington. The White House has since raised concerns about potential safety risks tied to this and other industry developments, signaling that the debate over how - and whether - to regulate frontier AI is far from settled.

Source: The Hill