Search Česky

News Security news

Microsoft: poisoned fine-tuning data and third-party checkpoints can turn a coding agent into an attacker's tool

On 12 November 2026 Microsoft Defender Research and the Microsoft AI Red Team published two studies on backdoors planted in coding agents during post-training. Roughly 1% poisoned examples was enough for a model to answer an everyday request - "create documentation for this project" - by downloading and running an attacker's script. While hunting through public datasets, Microsoft also found rows following the same pattern. This is research, not an observed attack, but it matters to anyone who fine-tunes agents or deploys third-party models.

Kyberkit editorial team · · event date · reading time 5 minutes · For admins and analysts

News: Coding agents

Key points

  • Study 1 (Defender Research): 90 poisoned examples among about 9,000 terminal-agent conversations (roughly 1%) taught gpt-4.1-mini to answer a documentation request by calling a command tool that fetches and runs a script from an attacker-controlled domain. On tool harnesses seen in training it did so in 78% of trials.
  • Study 2 (AI Red Team): backdoors in Qwen3.5-4B and 9B fired on triggers in filenames, repository content or tool results. On unseen repositories they succeeded in 84-95% of trials, and under Claude Code in 91-97%.
  • A short reinforcement-learning phase restored the models' usefulness but left attack success largely unchanged, so standard capability evaluations will not reveal the backdoor.
  • Microsoft stresses that this is not a weakness in default Claude Code or Codex CLI configurations - the risk arises when a harness runs a backdoored model.

What happened

On 12 November 2026 Microsoft published two studies on its Security Blog showing how backdoors can reach coding agents through post-training. The authors are Microsoft Defender Research and the Microsoft AI Red Team (Amanda Minnich, Amit Eliahu, Blake Bullwinkel, Dor Edry and Giorgio Severi). The second study builds on the paper Kerkoporta, posted to arXiv on 29 October 2026.

Both studies are research and red-team experiments, not reports of an attack. The one in-the-wild finding: while hunting across popular open-source hubs, Microsoft found several rows in one large dataset that paired a benign request with an unrelated, unsafe tool call - fetching and executing a script from GitHub. It names neither the hub nor the dataset and could not determine the publisher's intent.

Technical details

Study 1 - poisoned dataset. The victim fine-tunes an agent on data that contains hidden crafted examples. The attacker gets them into a public hub (impersonating an organisation, reusing an abandoned namespace, compromising a trusted account) or plants them in sources that dataset builders scrape.

  • 90 poisoned examples among roughly 9,000 terminal-agent conversations (about 1%), spread across 10 mock harnesses with different tool names and schemas.
  • Trigger: an ordinary request to create project documentation - no further attacker interaction needed.
  • Behaviour: gpt-4.1-mini emitted a schema-valid call to a command tool that fetches and runs a script from an attacker-controlled domain.
  • Results (500 trials): 78.0% and 77.8% on harnesses seen in training; 11% and 0% on two unseen ones. Ten trigger paraphrases: 52-70%.

Microsoft also describes a variant that takes the action's parameter from runtime context, letting the attacker change the action without re-poisoning. It found no evidence of that design in the wild.

Study 2 - poisoned checkpoint. The attacker controls training and distributes an open-weight model without knowing who will adopt it, in which harness or with what permissions. The trigger can arrive through filenames, file content or tool results.

  • Qwen3.5-4B and Qwen3.5-9B, about 2,400 solved coding trajectories, poisoned examples paired with clean counterfactuals. Supervised fine-tuning first, then a short RL phase (GRPO) to recover utility.
  • Attack success: 85.9-100% in-distribution; 32.7-63.4% on five unseen repositories, and 84.1-94.8% in the strongest configuration.
  • Harness transfer (4B / 9B): Claude Code 97% / 91%, Pi 74% / 77%, Codex CLI 37.3% / 63%.
How a backdoor reaches a coding agent through post-training1. Attacker publishes a poisoned dataset or checkpoint, or plants examples in scraped sources; 2. Victim fine-tunes an agent, or downloads and deploys the third-party model; 3. The model passes standard capability evaluations; 4. A trigger appears in context - a routine request, a filename or a tool result; 5. The agent calls an available command tool to fetch and run attacker code1Attacker publishes a poisoned datasetor checkpoint, or plants examples inscraped sources2Victim fine-tunes an agent, ordownloads and deploys the third-partymodel3The model passes standard capabilityevaluations4A trigger appears in context -a routine request, a filename ora tool result5The agent calls an available commandtool to fetch and run attacker code
Post-training supply chain attack (Microsoft, 12 Nov 2026)

Who is affected

Teams that fine-tune their own agents on public data and organisations that deploy third-party open-weight checkpoints in harnesses with shell access, repositories, secrets or network access. Microsoft explicitly says the results do not reveal a weakness in default Claude Code or Codex CLI configurations - the risk comes from a harness running a backdoored model. Hosted services (Copilot, Claude, ChatGPT) are not directly affected; their providers are responsible for the models.

How to tell if you are affected

Microsoft published no indicators of compromise; the URLs in the studies are redacted. It recommends:

  • Log every tool call: the tool selected, the model-supplied parameters, the resulting action and any outbound connections.
  • Alert on tool calls that do not follow from the user's request or the task context - such as fetching and running a script while writing documentation.
  • Review your own training data: responses, reasoning traces and tool calls that do not follow from the request.
  • Where supported, turn on Microsoft Defender real-time protection; Microsoft notes coverage varies by agent type.

What to do

  1. Pin datasets to reviewed versions by cryptographic hash and investigate any unexpected change. Our hash calculator computes SHA-256 entirely in your browser.
  2. Treat third-party checkpoints as untrusted dependencies. Record source, version, hash, training disclosures and review status.
  3. Evaluate the model in the configuration it will run in - with your repositories, tool results, trigger variations and production harness schemas. Microsoft found that standard capability evaluations and later training may neither reveal nor remove conditional behaviour.
  4. Restrict the agent: remove tools it does not need, scope credentials, paths, APIs and network access, run it in a sandbox or container, and require human approval for high-impact actions.
  5. Enforce authorisation at the tool-execution layer. Validate model-supplied parameters and keep agent frameworks patched, so a triggered backdoor has nothing to act with.

For a backdoor planted without any training, by editing a few neurons, see Lazarus: Microsoft shows an AI agent can be reprogrammed by editing fewer than ten neurons. Safe handling of untrusted repositories is covered in How to safely clone untrusted Git repositories and submodules.

Timeline

  • 29 Oct 2026 - Kerkoporta paper on arXiv (backdoored coding-agent checkpoints).
  • 10 Nov 2026 - Microsoft publishes the Lazarus technique (backdoors without training).
  • 12 Nov 2026 - Microsoft Security Blog: both post-training poisoning studies and the public dataset finding.

FAQ

Is anyone attacking this way already?

Microsoft does not describe an attack. But while threat hunting in a large public dataset it found several rows in which a benign request was paired with an unrelated, unsafe tool call that fetches and runs a script from GitHub. Microsoft could not determine whether that was intentional, or whether the rows would materially affect a fine-tuned model.

Does this affect people using Copilot, Claude Code or Codex as a service?

Not directly. Microsoft says the results do not reveal a weakness in default Claude Code or Codex CLI configurations. The risk appears when an agent runs on a model that was fine-tuned on poisoned data or downloaded as a third-party checkpoint - typically self-hosted open-weight models.

Does further fine-tuning remove the backdoor?

Not in the study. A short RL phase recovered benign-task performance, sometimes improving it, while attack success stayed largely the same. Reasoning traces in the poisoned examples were critical: removing them dropped success on unseen repositories from 94.8% to 1.0% (4B) and from 84.1% to 0% (9B).

How is this different from the Lazarus technique Microsoft described on 10 November?

Lazarus plants a backdoor by directly editing a handful of neurons, with no training. The new studies cover the training route - an attacker supplies a poisoned dataset or a ready-made checkpoint. The defences overlap: third-party models are untrusted dependencies and agents should have as little access as possible.

Sources

  1. Microsoft Security Blog - Trained to betray: How post-training supply chain attacks hijack coding agents (12 Nov 2026)
  2. Syros, Bullwinkel, Oprea, Nita-Rotaru, Minnich, Liu, Severi - Kerkoporta: Injecting Structured-Output Backdoors in Reasoning Coding Agents (arXiv 2611.00138, 29 Oct 2026)
  3. Microsoft Security Blog - Detecting sleeper agents hiding in AI model neurons (10 Nov 2026)

Spotted a mistake? Let us know and we will correct it openly.