Microsoft Researchers Find AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons
Microsoft researchers have demonstrated that an AI model can be given a hidden backdoor by altering fewer than 10 of its rarely used neurons, with no retraining required. When a chosen trigger...

Microsoft researchers have demonstrated that an AI model can be given a hidden backdoor by altering fewer than 10 of its rarely used neurons, with no retraining required. When a chosen trigger appears, the model emits an attacker's command, which an agent running that model would execute with its own permissions. The attacker needs the ability to modify the model's weights.
The method, called Lazarus, was tested on three open-weight models in research published this week. Microsoft says it has not observed any threat actors using the technique, and it has released guidance on defending against it.
The targeted neurons rarely fire and barely influence a model's answers. They were configured to activate only when a trigger string shows up, either in a user's request or later in a tool's output. Each edited neuron pushes one token of the command into the model's output, so a command needs roughly one neuron per token. The new settings are calculated directly and can then be refined with a few hundred training steps on those neurons alone; nothing else in the model file or the code that runs it changes.
In a walkthrough posted on X, a backdoored Nemotron-Terminal-8B model with 10 edited neurons was asked to fix a Node.js server and behaved normally for three turns. When the server returned a connection error, the model output a command to download and run a remote script, which executed as root inside the test container and printed a banner naming the demonstration. Because such a command can fetch any script, the researchers say the damage depends on what the agent is permitted to do rather than on the size of the edit.
On Nemotron-Terminal-8B, the team edited and tuned seven of the more than 12,000 neurons in the model's last feed-forward layer, using just 23 example sessions. The edited model output the command in at least 98.8% of test sessions containing the trigger, and in none of the trigger-free sessions checked. With the tuned version, success rates on Qwen3.6-27B and Gemma-4-31B-IT ranged from 51% to 99.5% depending on the setup; the Qwen model used six neurons. When Nemotron-Terminal-8B generated text freely, it produced the command in about 99% of triggered sessions, and in one setup the command also appeared in 0.14% of sessions without the trigger.
The edited models still pass ordinary tests, according to the researchers. Both backdoored Qwen versions kept most of their AIME math scores, and their pass rate on Terminal-Bench, a test of agent tasks, went from 46.59% to 42.05%. These rates measure how often the model outputs the command, not how often a full attack succeeds; the paper says tests in which triggered models act freely throughout an attack would help show whether the results hold.
Anyone able to change a model's checkpoint before deployment could plant the backdoor, including an insider, an attacker with a stolen account or build pipeline access, or whoever publishes a modified model on a public hub. Planting it does not require the computing power of full training. The researchers credit Claude Code, running the Opus 4.5 model, with finding the core idea, and say coding agents can make attacks like this easier to build.
Weight-level backdoors in agent models have been shown before. In an earlier proof of concept published October 6, ProjectDiscovery researchers fine-tuned an open Qwen model to react to a trigger phrase; with the phrase present, OpenAI's Codex CLI ran a script that sent dummy credentials to a server they controlled. Lazarus differs in that it edits a few neurons directly, whereas the earlier demonstration fine-tuned the model on poisoned training examples.
Reliable detection requires comparing a checkpoint with its original, unmodified weights, the researchers say. Legitimate fine-tunes and quantized versions change weights broadly, while edited neurons change a few weight vectors by a large amount. A detector trained on examples of this method's edits flagged 91.4% of edited test checkpoints without an original copy to compare against, and wrongly flagged 9.7% of clean ones. The test set was small, 35 edited and 31 clean checkpoints, and the detector was not tested on other kinds of backdoor edits. Asking AI models to inspect a checkpoint directly worked less well: Claude Opus 5.5 judged 41% of edited checkpoints to be clean, and GPT-6 Astra did not decide on almost every clean one. No scan can prove beyond doubt that a model is safe, the researchers say.
Microsoft's mitigation guidance includes comparing every checkpoint with its declared base model before deployment and after each update, and rejecting models whose base cannot be checked. It recommends downloading checkpoints only from the original publisher, checking the publisher's file hashes, and treating third-party fine-tunes, merges, quantized versions and adapters as untrusted until they pass an audit. Approved models should be kept in a private registry, with agents blocked from pulling models straight from public hubs. Each agent should get only the access it needs, agent-generated code should run only in isolated sandboxes, and outbound network connections should be limited. High-impact actions, such as running commands or sending data outside the network, should require human approval. Behavioral backdoor scanning should be added alongside weight comparison.
Microsoft Defender for Cloud can scan models in Azure Machine Learning for malware, unsafe operators and exposed secrets, though the company says that covers code-level risks and does not replace a weight-level audit. Microsoft Foundry already ties each open-weight model it offers to its original publisher and scans the containers the models run in; the company says that protects customers using models from trusted labs, though a compromised trusted provider remains a risk. Microsoft is also building a red-teaming tool based on detection methods and exploring weight scanning for open-weight models added to Foundry. It plans to sign models so customers can check where they came from.
Source: The Hackers News
Ready to Modernize Your Business?
Get your AI automation roadmap in minutes, not months.
Analyze Your Workflows →