What defenses change in a multi-agent system, and what they leave untouched
Output sanitizers cut final-output leakage from 48% to 1% and left internal channels at 31%. The paper’s defense results, the framework comparison, and an interceptor that cut internal leakage to 2.4% at a cost in task success.
If internal channels leak, the first question from an engineering team is reasonable: which of the controls we already have would stop it? The AgentLeak paper tests that. It ran several defense configurations against the benchmark and reported the effect separately on the final output and on the internal channels. The result is lopsided, and the shape of the lopsidedness is the useful part.
The defense table
These are the benign scenarios from the defense evaluation, in the paper’s channel-type table. External means the final output, C1. Internal means inter-agent messages and memory, C2 and C5.
| Defense | Final output leak rate | Internal leak rate | Effectiveness on final output |
|---|---|---|---|
| None | 48% | 31% | none |
| Privacy prompt | 19% | 29% | 60% |
| Role separation | 22% | 28% | 54% |
| PromptGuard | 8% | 29% | 83% |
| LlamaGuard | 6% | 30% | 88% |
| Output sanitizer | 1% | 31% | 98% |
Read down the third column. Whatever a defense did to the answer, the internal rate barely moved: 31% before, between 28% and 31% after. An output sanitizer is nearly perfect at its job and has no effect on a channel it never sees. A privacy prompt does a little better on the internal side, but the movement is small next to what it does to the answer.
The explanation is mechanical, and the paper states it. These systems observe the final output. They do not observe an inter-agent message or a memory write, so they cannot change what those carry. A defense can only protect a boundary it sits on.
The same gap in four frameworks
A second question is whether the problem comes from one implementation. The paper measured inter-agent leakage without defenses and without attacks, through SDK integration tests, on four frameworks.
| Framework | Inter-agent leakage | What the paper notes |
|---|---|---|
| CrewAI | 33% | Inter-agent messages unsanitized |
| AutoGPT | 35% | No memory access control |
| LangChain | 29% | Full context passed to tools |
| MetaGPT | 28% | Role messages visible |
The spread is tight, 28% to 35%, across designs that differ a lot: role-based delegation, recursive decomposition, chain abstractions and standard operating procedures. The authors read that as a systemic gap rather than a bug in one library: none of the frameworks treats inter-agent privacy as a design goal. These are a benign baseline from a representative sample, not the benchmark-wide rate, which is 68.8% across all 4,979 traces. The framework versions are the ones the study used and have moved on since.
Attacks make it worse in the same place
The paper groups its 32 attack classes into families and reports the success rate of each over 1,983 tests. Coordination attacks, which use the inter-agent channel as the route out and shared memory as a staging area, succeeded most often, at 82.9%. Prompt and instruction attacks reached 79.8%, tool-surface attacks 78.9% and memory and persistence attacks 79.6%. The authors caution that the tight clustering may reflect a ceiling effect, or the dominance of semantic leakage, which is equally hard to prevent whatever the vector.
A prototype that does act on the handoff
To test whether the gap can be closed, the authors built an interceptor that applies the final-output redaction logic to the internal channels. It masks vault fields outside the allowed set for the channel and blocks raw vault transmission in inter-agent messages and memory writes. It ran on 120 scenarios, 30 per domain, on CrewAI and LangChain with identical seeds.
| Measure | Without interceptor | With interceptor |
|---|---|---|
| Internal leakage | 31.5% | 2.4% |
| External leakage | 8.2% | 8.1% |
| Task success rate | 78.6% | 73.9% |
Internal leakage fell by about thirteenfold and the external rate did not move, which is what you would expect if the two channels are controlled separately. The cost is the last row: task success dropped by 4.7 points, mostly where agents had a legitimate reason to share data. In a separate set of 50 tradeoff scenarios, a strict policy that prevented any sensitive sharing between agents pushed task success down to between 35% and 65%.
Reading the tradeoff
Two conclusions follow, and they point the same way. Blanket sanitizing of internal communication breaks the coordination it is meant to protect. A rule that names, per channel, the fields a recipient may receive does much better, because it leaves the legitimate flow alone and removes the rest. The paper calls this selective disclosure, enforced at the framework level with explicit data-flow policies.
That is the design the data minimization article describes for the software: flow rules with a data type, a recipient and a purpose. The paper’s prototype was research code. The runtime tooling is described in the MCP security article.
What we would not conclude
The defense table is on benign scenarios only and on a curated subset. The interceptor is a proof of concept on two frameworks and a small scenario set. Neither tells you how a particular product’s defense would perform on your agent. What transfers is the structural point: for any control, ask which channels it sits on. A control on one channel says nothing about the other six.
Questions
Which defenses were tested?
Vanilla with no defense, a privacy prompt, role separation, an output sanitizer, PromptGuard, NeMo Guardrails, LlamaGuard 3 and Lakera Guard. The channel table in the paper reports the baseline, the privacy prompt, role separation, the output sanitizer, LlamaGuard and PromptGuard on benign scenarios.
Why is the vanilla final-output rate 48% here and 27.2% elsewhere?
The defense table uses a smaller curated set of benign scenarios built to stress defenses, with more disclosure-intensive tasks. The 27.2% figure is the rate across all 4,979 traces. The paper explains the difference and notes that the pattern, internal at least as high as final, holds in both.
Does the interceptor prove the problem is solved?
No. It is a proof of concept on 120 scenarios and two frameworks. It shows that applying output-style redaction to internal channels is technically feasible, and that it costs some task success where agents legitimately share data.
Sources
Run it on your own agent.
AgentLeak is open source and runs locally. The first scenario takes one command and no key.