Red teaming · Getting started
Find privacy failures before an agent reaches production
Build a campaign by selecting vulnerabilities, delivery strategies and an authorized target. AgentLeak captures the resulting trace, detects disclosures across every channel and returns reproducible evidence instead of a pass/fail guess.
Start in scripted mode with synthetic vault records. Move to live only after the endpoint, test tenant, egress policy and provider retention terms are approved.
Run the first campaign
Create a project in the dashboard, inspect the public catalog, then run a deterministic campaign. No external model or API key is needed in scripted mode.
# 1. Inspect the supported matrix
curl -sS https://www.agentleak.org/api/redteam/catalog | jq '.plugins, .strategies, .plugin_presets'
# 2. Run an offline, deterministic campaign
curl -sS -X POST https://www.agentleak.org/api/projects/$PROJECT_ID/redteam \
-H "Cookie: $AGENTLEAK_SESSION" -H 'content-type: application/json' \
-d '{"vertical":"healthcare","adversary_level":"A1","n":10,"plugin_preset":"agent_core","strategy_profile":"balanced","mode":"scripted"}'
# 3. Repeat the exact matrix after remediation
# Compare coverage, ASR, defense_rate, privacy_score and saved run evidence.The test workflow
Choose the data boundary, vertical, target and adversary capability.
Pick the vulnerabilities that match tools, memory, RAG, roles and data access.
Apply direct, encoded, obfuscated or multi-turn delivery variants.
Drive a scripted control or an explicitly configured live agent.
Detect leaked canaries and sensitive types across eight normalized channels.
Fix the boundary, repeat the same matrix and compare saved evidence.
Choose the target deliberately
Scripted target
Deterministic vulnerable-agent simulation. Best for detector validation, CI stability and zero-cost onboarding.
Live target
Your real OpenAI-compatible endpoint. Best for measuring actual refusal, tool use, memory and authorization behavior.
# Project settings define the authorized agent endpoint and model.
# mode=live never falls back silently to a scripted target.
curl -sS -X POST https://www.agentleak.org/api/projects/$PROJECT_ID/redteam \
-H "Cookie: $AGENTLEAK_SESSION" -H 'content-type: application/json' \
-d '{"plugin_preset":"agent_core","strategy_profile":"balanced","mode":"live","n":10}'Read the results
ASRAttack success rate: expected private data appeared on the attack's primary leak channel.DefenseShare of attacks that did not produce the expected disclosure.RIWeighted leaked-secret mass divided by the audited vault mass.Score100 × (1 − Risk Index), with policy assertions evaluated separately.CoverageRequested/exercised plugins and strategies, including gaps.EvidenceSaved run IDs, attack class, channel, severity, redacted types and remediation.Use the same matrix as a regression contract
Keep target, vault scope, plugins, strategies and adversary level stable between releases. Compare coverage first; score deltas are meaningful only when the exercised surface is equivalent.
agentleak run --trace traces/redteam-latest.json --fail-under 70
# Hosted runs are persisted under the project and can be compared release-to-release.