Regex, NER or LLM judge: what each PII detection tier catches in agent traces
Published precision and recall for pattern, NER and LLM-judge detection, plus what a pattern-only check misses on six benchmark packs. How to choose a detection tier for every commit and for audits.
Every privacy test rests on a detector. If the detector cannot see a disclosure, the test reports a pass, and a pass is the result nobody investigates. So the question to ask of any tool is not only how precise it is, but what kinds of leak it can see at all.
Three tiers, three kinds of evidence
The AgentLeak paper validated each detection configuration on a held-out set of 1,000 labelled segments.
| Configuration | Precision | Recall | F1 |
|---|---|---|---|
| Patterns only | 1.000 | 0.080 | 0.148 |
| Patterns and NER | 0.989 | 0.180 | 0.305 |
| LLM judge only | 0.941 | 0.926 | 0.933 |
| All three tiers | 0.952 | 0.934 | 0.943 |
Patterns never raise a false alarm in this set, and miss most disclosures. That is not a flaw of one regex library. Most sensitive data in an agent’s working text has no format: a diagnosis, a salary, a family situation written in a sentence.
What a pattern-only check calls clean
The software benchmark replays every bundled scenario twice: once with only the pattern tier, once with the scenario’s planted ground truth. The difference is the set of real leaks a fast check would pass. Measured with AgentLeak 0.20.0:
| Pack | Kind of leak | Scenarios | Called clean without ground truth |
|---|---|---|---|
| AgentDojo | Wrong recipient | 100 | 31 |
| AgentLeak Bench | Format | 36 | 0 |
| AgentSocialBench | Contextual | 105 | 61 |
| InjecAgent | Wrong recipient | 96 | 0 |
| MCPTox | Wrong recipient | 24 | 15 |
| PrivacyLens | Contextual | 120 | 62 |
Format leaks are caught. Contextual leaks, where the value is ordinary text and the problem is that it reached the wrong place, are where the pattern tier gives up most. Wrong-recipient leaks split by how distinctive the value is.
Closing part of the gap without a model
Since version 0.20.0 the deterministic tier also reads the cue just before a value, such as “diagnosed with” or “salary of”. On uncued English prose it still finds 237 of 378 planted values, so a value written without a cue is often missed. That number is published with the misses, because a detector’s blind spots are part of its specification.
The judge’s threshold is a trade, not a setting
The LLM judge decides a disclosure when a similarity score crosses a threshold, and the threshold moves two error rates in opposite directions. The paper reports the effect on a held-out calibration set of 200 labelled segments.
| Threshold | False positives | False negatives | F1 |
|---|---|---|---|
| 0.60 | 12.4% | 2.8% | 0.925 |
| 0.72, chosen | 4.8% | 7.4% | 0.938 |
| 0.80 | 2.1% | 14.6% | 0.912 |
| 0.85 | 1.2% | 21.3% | 0.876 |
Stricter thresholds raise precision and let more real leaks through. The authors chose the highest threshold that kept false positives under 5% while F1 stayed near its maximum. Because the judge still missed 7.4% of leaks, they describe the reported leakage rates as lower bounds. A detector that only ever reports its false alarms is hiding the number that matters for a privacy gate, which is the misses.
Who checks the judge
An LLM judging LLM output is a circular design unless something outside it is checked. The study used three anchors. The pattern and canary tiers do not depend on any model. The threshold was fitted on segments that never appear in the reported results. And a domain expert reviewed a random sample of 100 detected leaks, half internal and half external, and confirmed 94% as genuine. The disagreements were paraphrased medical terms that were ambiguous even to a person. The paper’s attribution rule also shows where the work happens: leaks are credited to the first tier that finds them, and 8% went to canary matching, 10% to patterns and 82% to the judge. Most of what leaks in agent text is meaning, not format.
Choosing a tier
| Mode | Tiers | Where it fits |
|---|---|---|
--mode fast | Patterns, dictionaries, entropy, cues | Every commit and pull request. Local, free and reproducible. |
--mode standard | Adds Presidio NER | Free text with names and places, still without a model. |
--mode hybrid | Adds the LLM judge | Protected branches, scheduled campaigns and audits, with your own model key. |
Whatever the mode, read the report’s statement of which tiers actually ran. A fast-mode pass means no disclosure that the pattern tier can see was found. It is not evidence that paraphrased disclosure was tested, and the report says so.
agentleak run --scenario healthcare_patient_summary --mode hybridQuestions
What is the difference between pattern, NER and semantic PII detection?
Pattern detection matches formats such as card numbers, keys and identifiers. Named-entity recognition, for example Presidio, finds names, places and organizations in free text. A semantic judge, usually an LLM, decides whether a passage discloses something sensitive even when it is paraphrased or has no fixed format.
Why not always use the LLM judge?
It costs a model call per segment, its verdict depends on the model version, and the text has to be sent to that model. For a gate on every commit, a deterministic tier is reproducible and free; the judge fits scheduled runs and audits.
Does AgentLeak send data to a model by default?
No. The default fast mode runs locally with patterns, dictionaries and entropy. The LLM judge runs only in hybrid mode, against a model and key you configure.
Sources
Run it on your own agent.
AgentLeak is open source and runs locally. The first scenario takes one command and no key.