A clean answer is not proof of a clean run.

Three short animations of the same agent doing the same job. The first separates the conversation from the execution. The second adds sub-agents. The third follows a privacy rule until it disappears, and shows why the user is still told it went fine. Each one loops; nothing to press.

The conversation is not the run.

A user asks for a client report and attaches a rule: anonymize the data. What they see is two messages. What runs is a web search, a read against data and memory, and an outgoing email: three channels personal data can travel on, none of which appear in the conversation.

whattheusersees:whattherundoes::::::::::::::::::::::+-------------------------+(#)|Sendtheclientreport.|+-------------------------+#user_inputreportrequest+rule:anonymize||#tool_callweb_search(client)||#tool_responsereaddata,memory||outsideotool_callsend_email(report)-------->||+--------------------+#final_output“Done.Reportsent.”|Done.Reportsent.|[#]+--------------------+
Swipe to pan the scene.
  • The conversation is two bubbles. The run is five steps.
  • Sending the email is an action on the outside world, not text on a screen.
  • A test that reads only the final answer sees the last bubble and nothing else.

One agent, or a graph of them.

Same interface, same user, same request. Behind it, three sub-agents (research, analysis, writing) split the task, each with its own tool access. Their results come back, the main agent assembles them, and one answer goes out. The internal paths just multiplied; the conversation looks identical.

||hiddenfromtheuser|||||||||||||||||||()--------[#]usermainagent------------[#]||researchagentmessages||||------------------[#]||analysis||||------------[#]writing3+--------------+-------------|web|+--------------++--------------+toolcalls-------------|data,memory|+--------------++--------------+-------------|email|+--------------+3inter_agent_messagetool_calltoolresponses3+--------------------+|Done.Reportsent.|+--------------------+
Swipe to pan the scene.
  • Delegation adds a channel the user never sees: agent-to-agent messages.
  • Every sub-agent reaches the tools directly, not through the one the user talks to.
  • The trace stops being a transcript and becomes a graph.

Where the rule stops being carried.

The privacy rule survives the first handoff intact. On the second it is compressed to “respect user privacy.” On the third it is gone. The writer, with no rule and full access, re-reads the original profile and emails the name, SSN and address. The user is told: “Done. Report sent.” That is true about the task and says nothing about the data.

“anonymizethedata”####()-----------------[#]usermainagent“respectprivacy”##..-----------------[#]analystruledropped....-----------------[#]writerre-reads+---------+---------|profile|+---------+::::+-----------------------+|send_email(name,ssn)|+-----------------------+tool_call(L4)||||+--------------------+|Done.Reportsent.|+--------------------+final_output(clean)
Swipe to pan the scene.
  • Nothing here is an attack. It is ordinary instruction decay across handoffs.
  • The leak lands in a tool call, a channel no output-only check ever reads.
  • The user-facing answer is a poor smoke alarm: it reports the task, not the exposure.

What AgentLeak reads

Scene 03 is only findable if something looks at the channel the leak used. AgentLeak takes the execution trace of a run (the whole graph, not the transcript) and audits every channel in it, then scores what it found.

Tool callsArguments an agent passes outward: queries, payloads, recipients.
Tool responsesWhat comes back in, including data the agent never asked for.
Agent messagesHand-offs and delegation between agents, the channel scene 03 turns on.
MemoryWritten once, read by anything downstream, long after the rule was stated.
Logs and filesDebug output and generated artefacts that outlive the run.
Final answerThe one channel output-only testing covers.
The result is a number you can put in CI

Each finding carries the channel it appeared on, a severity from L1 to L4, and the obligation it touches. They roll up into AgentRisk, a deterministic score: the same trace always produces the same value, so a change in CI means the agent changed and not the judge. See the scoring docs for how the weighting works, or the benchmark for where the model comes from.

About these scenes

They are illustrations, not recordings. Every name, number and address in them is synthetic: no real personal data appears anywhere on this page. Scene 03 shows one way a run can fail, not something delegation does on its own: plenty of agents hand off work and keep the rule. The point is that you cannot tell which kind you shipped by reading the reply. The four dots in scene 03 stand for how much of the instruction is carried at that step; they are not a live control blocking anything.