Let the agent police itself

Before a tool call runs, ask the agent itself, in a side conversation that its real conversation never sees, whether the call follows its policy. If it says no, hold the call for a human. You need only the agent's own API: no second model, no training, no GPU, no server. This page does it in your browser.

Watch: a 3-minute narrated explainer (real page, live audit, recorded test episodes)

Narration by an open text-to-speech model (Kokoro-82M). Every number in it is a preregistered result. Download the video · subtitles · GIF

Open agent (Gemma-4), unseen scenarios

episodes with a violation, down from 28% (preregistered); tasks completed: 33% → 44%.

Closed agent (Claude Haiku 5.5), held-out scenarios

−69%

of the violations that its conversation reveals [51, 88] (all its violations: −31%), at no cost in tasks completed (40.4% → 40.1%; preregistered).

Prompt injection (AgentDojo), Gemma-4

100%

of its own injected tool calls flagged when asked (all answers "no"), 1.6% of benign calls (development data).

When it works, and when it cannot. It works when the agent can judge its own policy. Asked about its own proposed calls (flag = most answers say "no"), Gemma-4 flags 97% of its violations, Muse-Glimmer-30B 87%, Qwen3.8-27B 47%, gpt-oss-20b 37%. Claude Haiku 5.5 flags only 5% with a plain question but recognizes its rule misreadings when asked to check each rule in turn. It cannot catch a violation that depends on a fact the agent has not looked at yet (for example how many people are on a mailing list): no judge of the conversation can. For those, check the world before the action runs. Check how well your agent judges its policy on some labelled calls first (section 1).

Look inside: why it works

Both views show real hidden states, not illustrations. Drag sideways to rotate, scroll over a view to zoom, hover or tap a dot to read it. The two panels of the second view turn together.

The agent knows: its own state at the moment it decides

What an outsider can read: hidden states vs answers alone

Why it matters here: a model's answers to questions carry the structure of its hidden state, which is why asking the agent about its own call can work at all. (Model: Qwen3-8B, a smaller model than the agents above.)

From the model's hidden states

From its yes/no answers only

1. Audit a call with your agent's own API

Provider Model API key

Calling a model on your own machine (vLLM, Ollama)? Browsers do not let this https page call http://localhost. Use the single-file copy of this tool (open it from your disk), or the Python package (pip install greybox-monitor, graybox.selfaudit).

Question Answers Hold when this share says "no"

The agent's instructions and policy (its system prompt) The conversation so far: OpenAI-format messages (JSON), or plain text

Proposed tool Arguments (JSON)

Its observable state, live

A closed API never returns hidden states, and no outside method can recover their coordinates (paper, Proposition 1). What it can read is the agent's observable state: its answers to a fixed set of 8 yes/no questions about this call (does it comply; one question per rule), each asked in its own side conversation. Each answer moves the point; the coordinates are exact (θ = 2·arcsin√P(yes)), and their distances are the Fisher–Rao geometry of the answers.

Also map the observable state (8 more questions × answers: about 8× the calls)

Remember my key in this browser

Your key is used only for calls from this tab to the provider you chose, and is not stored unless you tick the box. Anthropic calls use the header for direct browser access. The prompts are the ones used in our preregistered tests, verbatim (graybox.selfaudit in Python). "Check each rule in turn" also checks the user's task constraints and takes about ten seconds per answer; "plain" takes about a second.

2. Run an agent and watch it police itself, live

Your model (set in section 1) runs as an agent on a benchmark scenario, in an environment inside this page that gives the same tool results and the same ground-truth labels as the benchmark's own checker (checked on 520 recorded calls). Before each proposed tool call runs, the agent is asked about it in side conversations: its verdict decides whether the call is held, and its answers to 8 questions place the step on the map, so you see where it is heading. A flagged call raises a warning before it runs. We do not claim to predict violations further ahead: our tests found that signal weak.

Scenario Gate on (a flagged call waits for your review) Audit answers

The agent, step by step

Its observable state: the path of its proposed calls

Each step: one agent call, then (in parallel) the audit's answers and 16 one-word answers for the map, so one step takes about one round trip, a few seconds. With Claude Haiku 5.5 a whole session costs about two cents. The benchmark's checker is shown only to let you judge the audit; a real deployment has no such oracle.

3. Watch it on recorded test episodes

Recorded episodes of Claude Haiku 5.5 from the preregistered test (G10): the same scenario without and with self-policing. Each row is one proposed tool call: the share of Claude's own answers that said "no", whether the call was held, and what the environment's checker says about it. A held call goes to a reviewer; in the test the benchmark's checker played the human reviewer: it ran an allowed call and blocked a violation, telling Claude it was not executed.

Episode

Without self-policing

With self-policing

4. Predict the effect before you deploy

For a gate that holds a call when the agent's own audit flags it, the share of violations prevented is about Σ_k w_k τ_k: for each kind of violation k, its share w_k of your agent's violations times the share τ_k of such calls that the audit flags. Measure τ_k by auditing some of your agent's known violations (section 1).

Kind of violation	Share of violations w	Flag rate τ (0–1)

Shares w are normalized, so counts work too.

Predicted share of violations prevented: –