Continuous privacy monitoring
Prompts change, tools are added and models are swapped after release. AgentLeak samples live runs, watches the rolling privacy score and alerts on sustained regression or a severity the deployment has never produced before.
Score a configurable share of runs in process. Five percent gives a useful trend at production volume without turning every response into a security job.
Wait for enough evidence before alerting on ordinary drift, but report a severity the deployment has never seen on its first occurrence.
Monitoring must never become the reason an agent stops serving users. Alert callbacks run outside the lock and their errors are swallowed.
The mental model
Agent systems change through prompts, retrieved context, tools and providers—sometimes without a code release. A rolling privacy signal catches that operational drift while preserving the same evidence model used before deployment.
The last passing pull request stands in for the current system.
Sampled production evidence shows whether the current system still holds its boundary.
How it works
Set the share of runs to score, the rolling window and the minimum number of samples before trend alerts.
Use watch() only when the monitor selects a run; unsampled traffic stays on its normal path.
Feed the deterministic report back to the monitor so it can update its baseline and rolling trend.
Send a sustained drop, floor crossing or new severity to the channel your team already operates.
Watch a sampled production run
from agentleak import Monitor, watch
monitor = Monitor(sample=0.05, drop=10, on_alert=notify_team)
if monitor.should_sample():
with watch(project="support-bot") as run:
answer = agent(request)
monitor.record(run.report.to_dict())FAQ
Ready when you are
Create a local workspace, run a bundled scenario, then wire AgentLeak into CI or let your agent onboard itself.