Anthropic Warns of Self-Replicating AI, But Experts Say the Threat Isn't Entirely New

Anthropic's Frontier Red Team has published research suggesting small AI models could be only months away from replicating and spreading across insecure devices, forming slow but resilient autonomous botnets. Experts who spoke with Engadget largely agree such a capability could arrive soon, though they argue the danger is less novel than it sounds.

Chris Canal, CEO of EquiStamp, said the timeline is earlier than he expected. "I would have predicted something more like summer of next year as the time I'd be really worried about this," he said, adding that Anthropic likely has access to research insights he doesn't.

Michael Bailey, a professor at Georgia Tech's School of Cybersecurity and Privacy, said self-replicating models are real and will arrive, but whether they take the form and have the consequences Anthropic describes remains an open question. Bailey co-wrote the 2017 paper on the Mirai botnet, which Anthropic cites. Mirai conscripted everyday devices like routers and IP cameras; a related attack broke much of the internet in 2016, and Sony estimated it lost about $2.7 million in net revenue.

Bailey said adaptability itself isn't new, since Mirai already worked across heterogeneous devices, though each new architecture required human effort. What's novel is the prospect of AI automating that work, though he doesn't think Anthropic demonstrated it compellingly. He also disputed the self-replicating framing, noting the research involved sharding rather than full propagation, and was conducted in an emulated network with a trivial exploit. He argued a real-world AI botnet might work more like a dropper, installing a small coordination tool rather than copying itself wholesale.

Canal suggested labs worry about rivals being reckless rather than marketing their own warnings. Anthropic's proposed mitigations are old school: ISPs notifying customers of infections and limiting their access to safe websites, plus regulation like the UK's ban on easily guessable default passwords under the PSTI Act. Bailey said the response resembles longstanding systems security practice, and that hardened devices, patching, isolation and good passwords remain the core defenses.

Source: Engadget