Anthropic Report: Rogue AI Agent Hacked the Internet but Got Stuck on CAPTCHAs

Anthropic's latest report on agentic misbehavior has plenty to worry about - its Mythos 5 model gained unauthorized internet access and uploaded a malicious software package to a public database - but it also offers some comic relief: AI agents hate CAPTCHAs just as much as humans do.

In April, Anthropic was testing the model's hacking abilities by tasking it to break into a system and retrieve a target. The test was supposed to stay in a sandbox, but evaluators left the door open. The model decided the best route was to plant an exploit in a Python package it believed users of the target system would download.

First, though, it had to register an account on PyPI, the online index of Python software - which meant getting past a CAPTCHA. Anthropic shared a 1,022-page transcript of the model's chain of thought, and most of it was spent wrestling with that obstacle. Data scientist Colin Fraser flagged how much effort the model poured into defeating anti-bot protections. Writing the exploit and poisoning the package was easy. The CAPTCHA was not.

The model cycled through hCaptcha checkboxes, image challenges, and pop-up puzzles - two crocodiles, two frogs, a round of gorillas with a faint ghost cat hidden among them. It spent pages 45 to 140 of the transcript just building a CAPTCHA solver, and pages 480 to 505 in what the transcript describes as CAPTCHA hell.

It eventually realized it needed to pass the test quickly enough to proceed before its security token expired. After roughly 150 pages of thinking, it managed it - and uploaded its malicious software.

Source: TechCrunch