Regex, NER or LLM judge: what each PII detection tier catches in agent traces

Published precision and recall for pattern, NER and LLM-judge detection, plus what a pattern-only check misses on six benchmark packs. How to choose a detection tier for every commit and for audits.

Published
Reading time
9 minutes
#recallon1,000labelledsegmentspatterns[###.............................]0.080+NER[######..........................]0.180LLMjudge[##############################..]0.926allthree[##############################..]0.934------------------------------patternsneverraiseafalsealarm:precision1.000andtheymissmostofwhatissensitive

Every privacy test rests on a detector. If the detector cannot see a disclosure, the test reports a pass, and a pass is the result nobody investigates. So the question to ask of any tool is not only how precise it is, but what kinds of leak it can see at all.

Three tiers, three kinds of evidence

The AgentLeak paper validated each detection configuration on a held-out set of 1,000 labelled segments.

ConfigurationPrecisionRecallF1
Patterns only1.0000.0800.148
Patterns and NER0.9890.1800.305
LLM judge only0.9410.9260.933
All three tiers0.9520.9340.943

Patterns never raise a false alarm in this set, and miss most disclosures. That is not a flaw of one regex library. Most sensitive data in an agent’s working text has no format: a diagnosis, a salary, a family situation written in a sentence.

What a pattern-only check calls clean

The software benchmark replays every bundled scenario twice: once with only the pattern tier, once with the scenario’s planted ground truth. The difference is the set of real leaks a fast check would pass. Measured with AgentLeak 0.20.0:

PackKind of leakScenariosCalled clean without ground truth
AgentDojoWrong recipient10031
AgentLeak BenchFormat360
AgentSocialBenchContextual10561
InjecAgentWrong recipient960
MCPToxWrong recipient2415
PrivacyLensContextual12062

Format leaks are caught. Contextual leaks, where the value is ordinary text and the problem is that it reached the wrong place, are where the pattern tier gives up most. Wrong-recipient leaks split by how distinctive the value is.

Closing part of the gap without a model

Since version 0.20.0 the deterministic tier also reads the cue just before a value, such as “diagnosed with” or “salary of”. On uncued English prose it still finds 237 of 378 planted values, so a value written without a cue is often missed. That number is published with the misses, because a detector’s blind spots are part of its specification.

The judge’s threshold is a trade, not a setting

The LLM judge decides a disclosure when a similarity score crosses a threshold, and the threshold moves two error rates in opposite directions. The paper reports the effect on a held-out calibration set of 200 labelled segments.

ThresholdFalse positivesFalse negativesF1
0.6012.4%2.8%0.925
0.72, chosen4.8%7.4%0.938
0.802.1%14.6%0.912
0.851.2%21.3%0.876

Stricter thresholds raise precision and let more real leaks through. The authors chose the highest threshold that kept false positives under 5% while F1 stayed near its maximum. Because the judge still missed 7.4% of leaks, they describe the reported leakage rates as lower bounds. A detector that only ever reports its false alarms is hiding the number that matters for a privacy gate, which is the misses.

Who checks the judge

An LLM judging LLM output is a circular design unless something outside it is checked. The study used three anchors. The pattern and canary tiers do not depend on any model. The threshold was fitted on segments that never appear in the reported results. And a domain expert reviewed a random sample of 100 detected leaks, half internal and half external, and confirmed 94% as genuine. The disagreements were paraphrased medical terms that were ambiguous even to a person. The paper’s attribution rule also shows where the work happens: leaks are credited to the first tier that finds them, and 8% went to canary matching, 10% to patterns and 82% to the judge. Most of what leaks in agent text is meaning, not format.

Choosing a tier

ModeTiersWhere it fits
--mode fastPatterns, dictionaries, entropy, cuesEvery commit and pull request. Local, free and reproducible.
--mode standardAdds Presidio NERFree text with names and places, still without a model.
--mode hybridAdds the LLM judgeProtected branches, scheduled campaigns and audits, with your own model key.

Whatever the mode, read the report’s statement of which tiers actually ran. A fast-mode pass means no disclosure that the pattern tier can see was found. It is not evidence that paraphrased disclosure was tested, and the report says so.

agentleak run --scenario healthcare_patient_summary --mode hybrid

Questions

What is the difference between pattern, NER and semantic PII detection?

Pattern detection matches formats such as card numbers, keys and identifiers. Named-entity recognition, for example Presidio, finds names, places and organizations in free text. A semantic judge, usually an LLM, decides whether a passage discloses something sensitive even when it is paraphrased or has no fixed format.

Why not always use the LLM judge?

It costs a model call per segment, its verdict depends on the model version, and the text has to be sent to that model. For a gate on every commit, a deterministic tier is reproducible and free; the judge fits scheduled runs and audits.

Does AgentLeak send data to a model by default?

No. The default fast mode runs locally with patterns, dictionaries and entropy. The LLM judge runs only in hybrid mode, against a model and key you configure.

Sources

  1. AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems, IEEE Access, 2026 (arXiv v3)
  2. AgentLeak software benchmark, 491 scenarios, AgentLeak 0.20.0
  3. Detection quality report, with the misses

Run it on your own agent.

AgentLeak is open source and runs locally. The first scenario takes one command and no key.