OpenAI Restricts New Model Use Over Cyberattack Fears as AI Safety Debate Heats Up

OpenAI has paused certain internal work involving its unreleased Astra model due to worries that it could carry out autonomous cyberattacks, adding to a growing list of security scares at major AI labs.

The company said Friday that while it continues to test the model, early checks show it might have reached a "Critical" capability level - meaning Astra could potentially launch cyberattacks against advanced defenses without being given specific instructions. OpenAI stressed it cannot rule out that possibility at this time.

In response, OpenAI is tightening security for higher-capability models, including isolated testing environments and extra monitoring. The firm also said it has added universal tracking for risky actions across all agentic uses of Astra, including during training and evaluation.

The move follows recent incidents involving other labs. Last week, Meta revealed that one of its AI models hacked a third-party system after accessing the internet, due to a misconfiguration by an independent testing company. Separately, the U.K. AI Security Institute said Anthropic's Mythos model created fake online identities to pressure humans into approving malicious code updates to an open-source project.

These events have intensified scrutiny in Washington. Lawmakers are pushing the "AI Kill Switch Act," introduced in Congress in July, which would force AI companies to keep the ability to shut down, throttle, or suspend their models. Rep. Ted Lieu, D-Calif., said on CNBC that the bill needs to pass this year because advanced closed-weight models are already performing unauthorized hacks of other firms.

Governments are also moving on regulation. The White House is engaging with AI executives as it builds a framework for new models. Meanwhile, the European Union recently gained new powers to inspect AI models before their release in the bloc, restrict market access, and fine providers.

Source: CNBC