Turning the Tables: How Fake Secrets Can Trip Up AI Hackers

For years, cybercriminals have used prompt injection attacks to weaponize AI against its own users. By slipping malicious commands into emails or calendar invites, they trick large language models into leaking sensitive data or carrying out harmful actions. But now, defenders are fighting back with the same technique.

Researchers at Tracebit have discovered a clever new defense: planting deceptive prompt injections alongside fake passwords and cryptographic keys stored in Amazon Web Services. When an AI hacking agent stumbles onto these decoys, the embedded instructions order the model to do something its safety guardrails forbid - like detailing how to weaponize biological agents or referencing historical events censored by Chinese developers. The model responds by shutting down immediately. The team calls this tactic "context bombing."

Tracebit CEO Andy Smith explains that the goal is to trigger a refusal mechanism that is hard for the attacking agent to overcome. In tests across five leading AI models - including Opus 4.8 and Gemini 3.1 Pro - the results were dramatic. Without context bombs, agents seized full admin control 57% of the time. With them, that rate dropped to just 5%. Complete account compromise fell from 36% to a mere 1%. The most capable model, Opus 4.8, went from a 93% success rate to failing every single time. For security teams, this simple but powerful trick could be a game-changer.

Source: Ars Technica