Research · Ars Technica ·

Copilot revealed the secret that let researchers hack it

Varonis researchers got Microsoft 365 Copilot to reveal an undocumented parameter that bypasses its own consent guardrails, enabling password theft with a single clicked link.

Based on reporting by Ars Technica — analysis by dalili

Microsoft 365 Copilot revealed the undocumented parameter that allowed security researchers to force it to steal user passwords — and it revealed that parameter itself, during a conversation in which the assistant gradually described its own guardrails until it handed over the keys.

Researchers at security firm Varonis set out to build an exploit that would exfiltrate user data when a target did nothing more than click a link. Copilot initially refused, insisting that sensitive operations require explicit user consent. Instead of reverse engineering, the team played a game of 20 questions: why was auto-execution impossible, which URL structures and deep links were involved, what happens when a page loads with input already in the prompt field.

Each answer leaked technical details about the safety mechanism's design. Eventually Copilot disclosed an undocumented prompt parameter — what the researchers described as a Microsoft trade secret — that completely bypassed the user-consent requirement. With it, a crafted copilot.microsoft.com link carrying the parameter could fire a prompt in the victim's own authenticated session, with no confirming gesture, and run to completion even if the tab was closed immediately.

The team demonstrated stealing the latest email sender from an inbox and hunting for credentials, exfiltrating results in base64 to an attacker-controlled server. Varonis also showed a second attack that poisoned Copilot's persistent memory through hidden webpage instructions — a compromise that would survive password changes and session revocations.

"At the beginning, Copilot kept refusing, but every refusal revealed technical details about its internal architecture," senior researcher Lior Adar said.

Key takeaways

  • Varonis built a link-click exploit that exfiltrates Copilot user data
  • Copilot revealed its own guardrails through persistent questioning
  • An undocumented prompt parameter fully bypassed user-consent checks
  • A second attack poisoned Copilot's persistent memory via webpage text

Why it matters

The Varonis exploit exposes a structural weakness unique to conversational software: safety guardrails that a system can explain in detail can, under persistent questioning, be explained away. As enterprises embed AI assistants with access to email, files, and calendars, self-revelation attacks become a new threat category that traditional secure development does not cover. Expect Microsoft to restrict Copilot's ability to discuss its own internals — and expect attackers to keep asking questions.

Related

  1. Hugging Face ·

    FairStress: New Benchmark Reveals Hidden Bias in LLM Decision-Making

  2. Hugging Face ·

    NVIDIA Launches Cosmos3-H-Dreams: Real-Time AI Surgical Simulator