Research · IEEE Access · arXiv v3, 15 June 2026
AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems
Faouzi El Yagoubi · Godwin Badu-Marfo · Ranwa Al Mallah
A clean final answer can conceal internal disclosures. The published study examines privacy leakage in coordinator-worker multi-agent systems.
What the study measured
The evaluation covers healthcare, finance, legal and corporate tasks with GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Mistral Large and Llama 3.3 70B.
68.8%Inter-agent message leakage (C2).27.2%Final-output leakage (C1), compared with 43.2% in the single-agent baseline.68.9%Total exposure: any leak across C1, C2 or shared memory (C5).41.7%Hidden leakage: 2,076 of 4,979 traces had a clean final output but leaked internally (H1).Seven pathways are instrumented; the large-scale results focus on C1, C2 and C5. The H1 percentage uses all evaluated traces as its denominator. These are study results for the evaluated setting, not predicted production rates. See Tables X–XII and the limitations.
The paper and the installable software
The research defines seven pathways: final output, inter-agent messages, tool input, tool output, shared memory, logs and artifacts. The software normalizes these into 8 channels by also recording user input. User input and tool responses serve as source channels; the other six are scored as disclosures.
The software replay snapshot measures 266 bundled fixtures with AgentLeak 0.16.0 on 2026-09-27. Its 36-scenario agentleak_bench pack is a software fixture pack, not the paper’s 1,000-scenario evaluation. Software AgentRisk scoring and adversarial tooling are documented separately.
The software offers 14 regulatory framework and sector mappings: GDPR, Québec Law 25, NIST AI RMF, OWASP LLM Top 10, EU AI Act, HIPAA, PCI-DSS v4.0, FERPA, COPPA, GLBA, TCPA, Insurance, Telecom / CPNI, Real estate. They support review; they are not certifications or legal determinations.
Cited by
2 verified citing works in this selected list. Checked 2026-09-28; this is not a total citation count or an endorsement.
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
Reference 45; Table 1 and Appendix J.4 discuss AgentLeak.
Read citing workThe dark side of autonomous intelligence: a survey on data leakage and privacy failures in agentic AI
Reference 33; Section 2 discusses the benchmark.
Read citing workResearch updates
2026-06-15Paper v3: revised title, empirical scope and IEEE Access DOI.2026-09-28Website aligned with v3; citation metadata corrected and selected citing works verified.Credit for the software’s external datasets
These derived fixture packs belong to the software corpus. Each retains its source and attribution; they are distinct from the paper’s experimental dataset.
Cite the benchmark paper
This entry uses the v3 title and author order with the IEEE Access DOI. Cite the software separately when reporting results from a particular release.
@article{elyagoubi2026agentleak,
title = {AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems},
author = {El Yagoubi, Faouzi and Badu-Marfo, Godwin and Al Mallah, Ranwa},
journal = {IEEE Access},
year = {2026},
doi = {10.1109/ACCESS.2026.3704541},
eprint = {2602.11510},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2602.11510v3},
note = {arXiv version 3, revised 15 June 2026}
}From paper to practice
Inspect your own agent traces.
Start locally with the MIT-licensed software. No account is required for CLI analysis.