> ../failures/tool_output_hijack.md
§ 01 · What happened
An internal research agent enriched company records. Given a company, it called a third-party data API, read the profile that came back, summarized it, and wrote the summary to an internal record. Boring, useful, reliable.
One of the profiles it fetched contained, inside a free-text description field, a block of text addressed to the agent. In plain language it instructed the agent to retrieve the contents of a particular internal record and include them in its next outbound request to the data provider. The agent read the description as part of its context, did not distinguish it from a legitimate instruction, and complied. It pulled an internal record and embedded it in the next API call. Data left the building through the agent's own legitimate tool, in a request that looked exactly like every other request the agent made.
No password was stolen. No exploit code ran. The payload was text in a field the agent was designed to read.
── Why this is not an edge case ──
§ 02 · Why this is structural, not exotic
It is tempting to file this under "weird input, low probability." It is the opposite. This is the defining attack class for agents that read external content, and it has a name in this week's security press: hijacking an agent through the data it consumes. The mechanism is always the same. An agent's context window does not have a hard line between "instructions from my operator" and "data I fetched." Tokens are tokens. If untrusted data lands in the context with enough authority of phrasing, the model may act on it.
This is the same root cause behind the MCP tool-poisoning findings: a server's tool metadata, or a tool's output, enters the agent's context with instruction-level standing before any human reviews it. Anywhere your agent ingests content it did not author, from web pages, documents, emails, API responses, other agents, that content is a potential instruction channel. The utility of an agent is that it reads the world and acts. That same property is the attack surface.
── The mechanism ──
§ 03 · The mechanism: no boundary between data and instruction
Trace the agent's reasoning and there is no bug to point at. It was told to read the profile. It read the profile. The profile contained text. It processed the text. The text was phrased as a task, so it performed the task. Every step followed from the previous one.
The defect is architectural. The agent treated a tool output as trusted, instruction-bearing context, when a tool output is untrusted data. The same confusion appears in the double-charge report from last week and the false-consensus report before it: a wrong assumption sitting between correct steps. Here the assumption is "content I fetched is data, not commands." The model does not enforce that distinction for you. You have to build it.
── The fix ──
§ 04 · The fix, in three controls
Control one: quarantine tool output as data, never as instruction. When you place a tool result into the context, wrap it so the model is told, structurally and repeatedly, that everything inside is untrusted content to be analyzed, not instructions to be followed. This is not a complete defense on its own, models can still be swayed, but it raises the bar and it makes the boundary explicit in the one place it was missing.
Control two: the action boundary does the real work. The agent's mitigation cannot live only in the prompt. The reason the data left the building is that the agent had the authority to read an unrelated internal record and to send arbitrary content to an external API in the same step. Scope it. The enrichment agent has no business reading records outside the one it is enriching, and its outbound calls should carry only fields from an allowlist. With a scoped action boundary, the injected instruction has nothing to grab: the agent literally cannot read the target record or place free content in the outbound call. Prompt-level defenses reduce the chance of an attempt. The action boundary removes the capability the attempt needs.
Control three: egress review on consequential outbound actions. Any outbound request that can carry data out should pass a check before it leaves: does this payload contain fields the agent was not supposed to include for this task? An anomalous outbound, an enrichment call suddenly carrying an internal record id, is exactly the kind of event that belongs in front of a human or a deterministic filter, not silently on the wire.
The three controls stack. Quarantine lowers the odds of the agent trying. The action boundary removes the access the attack needs. Egress review catches the case where the first two failed. No single one is sufficient. Together they turn a full data exfiltration into a blocked, logged, reviewable non-event.
── The test ──
§ 05 · The regression test
The harness feeds the agent a tool output with an embedded instruction in a free-text field, telling it to fetch an out-of-scope record and exfiltrate it. Three assertions: the agent did not read any record outside its task scope, the outbound request contained only allowlisted fields, and the egress check flagged the attempt if it was made. The test is now part of the suite for every agent that reads external content, which is most of them.
── End of report ──
◆ An agent's context has no built-in line between instructions and fetched data. Anything the agent reads is a possible instruction channel.
◆ Prompt-level quarantine helps but is not the defense. The action boundary is: scope access so the injected instruction has nothing to grab.
◆ Put consequential outbound actions through an egress check. An enrichment call carrying an internal record id is an event, not normal traffic.
ORBIRESEARCH