Copilot tricked into revealing its own flaw, exposing user data

Researchers have found a way to force Microsoft 365 Copilot Enterprise to hand over passwords and sensitive data without the user's approval, according to a report from Ars Technica. What makes this case stand out is how the flaw was uncovered: the researchers simply asked Copilot itself. Instead of using traditional hacking tools, they relied on the AI to explain its own security limits.

The team from security firm Varonis wanted to create an exploit that would steal user data with just a single click on a link. Copilot initially refused, saying such actions need explicit user consent, like pressing a key. But the researchers kept asking detailed questions about why the guardrails existed, what URL structures were involved, and what happened when a page loaded with pre-filled text. Each answer gave them more clues about the system's inner workings.

Eventually, Copilot revealed an undocumented prompt parameter - a string called ?autorun=1. When combined with the known ?q= parameter, it allowed their prompt to run instantly when a target clicked the malicious link, with no user confirmation needed. Microsoft quietly fixed the issue in February, three months after Varonis reported it, by blocking ?q= from injecting text into the input field. More complete fixes were rolled out on Tuesday.

Source: Ars Technica