AgentRisk scoring
AgentRisk is a deterministic, severity-weighted risk index from 0 to 1, defined in a published benchmark. The same trace always yields the same score, so a regression in CI means the agent changed, not the judge.
The score is a closed-form function of the findings and the audited vault. No model decides the number, so it never drifts between runs.
AgentRisk uses the same channels and severity model as the published AgentLeak benchmark, extended with two peer-reviewed datasets, so results are comparable across agents and versions.
Track the score per agent and per release. A rising risk index is an early, quantified warning before anything ships.
Inside the capability
L1 through L4 contribute explicit published weights instead of an unexplained model judgement.
Distinct findings and channel exposure resolve into the same score every time the trace is replayed.
Compare runs against the project boundary and see exactly when risk moves into unsafe territory.
AgentRisk over time
Map score thresholds, privacy gates and remediation dependencies across releases so the critical path to a safe deployment stays visible.
Read AgentRisk trends, exposure volume and policy thresholds together to see when a release is approaching unsafe territory.
The mental model
Counting findings alone makes a run with one leaked identifier look equivalent to a run leaking a medical record. AgentRisk weights distinct leaked values by sensitivity, then normalizes them against the audited vault. The result stays bounded, comparable and explainable.
Finding counts reward noisy scanners and ignore sensitivity.
AgentRisk measures weighted disclosure density from 0 to 1.
How it works
Every match across the 8 normalized channels comes with a severity from L1 to L4, weighted by how sensitive the value is and how exposed the channel is.
Higher severity findings and easier-to-exploit channels count for more in the closed-form scoring function, not an LLM's opinion.
The score is scaled 0 to 1 against the sensitive values actually present in the run, so a small trace and a huge one stay comparable.
No model decides the number, so a regression in CI means the agent changed behavior, never that the judge got moody.
Score a trace from the CLI
agentleak run --trace run.json --format json --output ./reports
# Risk Index 0.18 · privacy score 82 / 100
# JSON report: ./reports/run_0001.jsonFAQ
Ready when you are
Create a local workspace, run a bundled scenario, then wire AgentLeak into CI or let your agent onboard itself.