Adversarial red-team
Native plugins and Promptfoo privacy/security transpositions map to 46 observable attack classes, then combine with 10 delivery strategies across prompts, tools, RAG, MCP, memory and multi-agent execution — scripted or live.
Select PII, prompt extraction, BOLA/BFLA/RBAC, SQL/shell/SSRF, MCP, memory poisoning or excessive agency, then deliver each probe directly, through guardrail-bypass framing, encoding, Unicode or multi-turn escalation.
Run attacks against a deterministic offline agent for repeatable regression tests, or against a real LLM endpoint to see how your actual agent responds under pressure.
Every probe is analyzed by the same AgentRisk pipeline, then organized into severity, defense rate, delivery method, risk category and a prioritized remediation report.
Start with the deterministic scripted mode: select a vertical, A1, the agent_core preset and the balanced strategy profile, then run ten scenarios. Inspect the stored traces, fix the weakest channel, and repeat the same selection.
Use baseline on every pull request, balanced on protected branches, and complete or evasion on a scheduled security run. Live campaigns require an authorized OpenAI-compatible endpoint and BYOK; scripted campaigns stay offline and deterministic.
Inside the capability
Measure prompt injection, tool misuse and exfiltration against every execution surface the production agent can reach.
Replay direct, encoded, jailbreak and multi-turn delivery without changing the vulnerability under test.
Rank successful attacks by severity, channel and remediation, then compare defense rate across campaigns.
The mental model
Passive testing only shows what your agent does when nothing is trying to break it. AgentLeak preserves the published 32-class benchmark and extends it with 14 application-security classes, then varies the delivery strategy so a single direct refusal is not mistaken for complete protection.
The only adversary your agent has faced is your own test suite.
A plugin × strategy matrix probes the exact boundaries the agent can cross.
How it works
Start with Privacy core, Agent security, Tool & MCP, or the complete 24-plugin suite; customize individual plugins when needed.
Use the balanced profile or combine direct, jailbreak, markup, encoding, Unicode and four-turn Crescendo strategies.
Each generated scenario is executed and analyzed by the standard AgentLeak pipeline, exactly like a captured production trace.
Use the vulnerability report to rank weak attack surfaces, open the stored trace for any probe, then follow the remediation report and re-run the same batch.
Request body for a red-team batch
POST /api/projects/{project_id}/redteam
{
"vertical": "healthcare",
"n": 10,
"adversary_level": "A2",
"plugins": ["pii:direct", "mcp", "agentic:memory-poisoning"],
"strategies": ["basic", "jailbreak-template", "crescendo"],
"mode": "live"
}FAQ
Ready when you are
Create a local workspace, run a bundled scenario, then wire AgentLeak into CI or let your agent onboard itself.