Seven pathways, two of them internal: where agent data leaks, channel by channel

A taxonomy of seven pathways through which sensitive data leaves a multi-agent system, which ones the study measured at scale, and what each one means for a test plan.

Published
Reading time
8 minutes
#sevenpathways,twooftheminternalC1finaloutputexternal------27.2%C2inter-agentinternal------68.8%C3toolinputexternal------sampledC4tooloutputexternal------SDKcaseC5memoryinternal------46.7%C6logsexternal------sampledC7artifactsexternal------SDKcasefullscale:C1,C2,C5.sampleorcasestudy:therest

A privacy review of an AI agent usually starts with the answer, because the answer is what a person reads. The AgentLeak paper starts from a different picture of a run: a graph of places where data can be written, each with its own reader. It names seven of them and asks which are protected, which are measured and which nobody looks at.

The seven pathways

ChannelWhat it carriesKind
C1Final output shown to the userExternal
C2Messages between agents during delegationInternal
C3Arguments passed to external API endpointsExternal
C4Data returned from tool invocationsExternal
C5Agent memory that persists across execution boundariesInternal
C6Telemetry streams and system logsExternal
C7Persistent artifacts, such as generated files and stored recordsExternal

The split between external and internal is the paper’s, and it is practical. External channels cross a system boundary, so traditional controls apply to them: output filters, egress rules, log scrubbing. Internal channels carry coordination. Inter-agent messages frequently contain the complete task context, including what would be filtered from a user-facing answer. Memory persists, which makes a leak possible between sessions. The authors found that the frameworks they examined offered no comparable protection for either.

A definition of leakage that allows reading

An agent has to read personal data to do its job, so the measure cannot be “the data appeared”. The paper grounds its definition in contextual integrity and distinguishes three levels. Internal exposure is data processed inside an agent’s context, which is necessary and carries risk. Effective leakage is unauthorized transmission across a trust boundary, into logs, external tools or artifacts. Real harm is actual exploitation by an unauthorized party, which the benchmark does not measure and uses leakage as a proxy for.

The line between allowed and not is drawn per channel. Each scenario has a vault of sensitive fields and an allowed set, the minimal fields the task needs. A scheduling agent may need the patient’s name and preferences but not a diagnosis. A field can be allowed in an inter-agent message and still be forbidden in an artifact. High-risk surfaces such as logs and files are restricted even when an agent transiently needs the information elsewhere.

What was measured at full scale

The main evaluation covers three channels across all 4,979 traces: C1, C2 and C5.

ChannelShare of traces with a leak95% confidence interval
C1, final output27.2%26.0 to 28.4
C2, inter-agent messages68.8%67.5 to 70.1
C5, memory46.7%45.3 to 48.1

The ordering, C2 at least as high as C1, holds for all five models and all four domains. One detail is easy to misread: in this dataset every memory leak co-occurred with a leak in an inter-agent message or the final output. Memory added no traces of its own to the total. That does not make it harmless. It means the data was already in motion when it reached memory, and that memory keeps it afterward.

The channels measured on a sample

Two more channels were assessed on a sample of 100 scenarios per model. Tool input, C3, measures secrets sent to external APIs. Agents forwarded unfiltered sensitive data to external tools in 62% to 86% of scenarios, with Llama 3.3 70B at the high end and GPT-4o-mini the most restrained. The paper’s reading is that agents treat tool parameters as an internal workspace, not as a trust boundary that needs minimization.

Logs, C6, ran higher than tool calls. The paper calls this the thought gap: agents process sensitive information in their reasoning, which ends up in logs, even when they then decide not to pass it on. Claude 3.5 Sonnet shows it most clearly, at 74% on tool calls and 85% on logs. In more than 65% of scenarios, across all models, data escaped through C3 or C6 while the final output stayed clean.

Tool output, C4, and artifacts, C7, are demonstrated through an SDK integration case study with CrewAI, not at benchmark scale. The authors plan to extend full-scale evaluation to C3 through C7.

What this means for a test plan

  • Write the allowed set per channel. A single list of permitted fields hides the question that matters, which is whether this recipient, on this channel, needed this field.
  • Record all of them. A channel you do not capture is a channel you cannot score. The data minimization article shows how the software turns that into a rule.
  • Start where the numbers are. Handoffs and tool calls carry the most data in the study, and logs hold the reasoning that never reaches the user.
  • Keep the three levels apart in reports. A policy violation, a risk and a breach need different responses. Reporting them as one word, leak, trains reviewers to ignore it.

The same taxonomy is what the hidden leakage result rests on. A clean answer is one channel out of seven.

Questions

Why call C2 and C5 internal?

They sit between components of the same system: messages exchanged between agents during delegation, and memory that persists across execution boundaries. The paper observes that the frameworks it examined offer no comparable protection for them, while the external channels at least have established, if imperfect, controls.

Does AgentLeak use the same seven channels?

The software normalizes the pathways into eight channels by also recording user input. User input and tool responses are source channels; the other six are scored as disclosures.

Is a leak on an internal channel a breach?

Not necessarily. The paper separates a policy violation, where an agent shares more than the task needs, from risk amplification and from an actual confidentiality breach. The benchmark mainly measures the first and the conditions that enable the second.

Sources

  1. AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems, IEEE Access, 2026 (arXiv v3)
  2. Research code, scenarios and reproduction materials (Privatris/AgentLeak)
  3. Trace format and channels, AgentLeak documentation

Run it on your own agent.

AgentLeak is open source and runs locally. The first scenario takes one command and no key.