Inside the AI Safety Plan That Lets Companies Keep the Final Say

Anthropic CEO Dario Amodei wants third-party safety evaluators stationed inside frontier AI companies, with access comparable to internal risk teams and the right to publish findings without company editing, apart from limited redactions. He compared the idea to banking supervisors embedded with employees.

But Julie Andersen Hill, dean of the University of Wyoming College of Law, said the comparison falls apart without the power to halt operations. Bank examiners can order a practice stopped, restrict growth, force management changes, or close an institution. Anthropic's evaluators could investigate and report, but Amodei's plan gives them no authority to block a model's training or release.

Albert Ziegler, head of AI at cybersecurity firm XBOW, said his team gets early access to unreleased models and shares findings with developers, but has no veto power. The dramatic scenarios people fear are not what his evaluations have shown, he said.

Critics argue that when a company picks its evaluator, controls its access, and can ignore its conclusions, the arrangement resembles an internal compliance department rather than independent oversight. UC Berkeley researcher Deborah Raji said access alone does not create independence.

Hill put the test plainly: if AI leaders truly believe the technology could destroy society, an independent supervisor needs the ability to pull the plug.

Source: CNBC