1
00:00:00,600 --> 00:00:02,531
AI agents now act for people.

2
00:00:02,531 --> 00:00:06,258
They send emails, change databases, and restart servers.

3
00:00:06,258 --> 00:00:11,850
Before any of those actions runs, someone has to decide whether it breaks the rules.

4
00:00:11,850 --> 00:00:16,577
The strongest agents are closed models, so you cannot look inside them.

5
00:00:16,577 --> 00:00:18,707
Grey Box asks a simple question:

6
00:00:18,707 --> 00:00:23,500
can the agent check itself, through its own API, before the action runs?

7
00:00:23,952 --> 00:00:26,813
We tested this in preregistered experiments:

8
00:00:26,813 --> 00:00:31,300
the analysis plans were locked and published before the data existed.

9
00:00:31,300 --> 00:00:41,248
On new scenarios, an open agent, Gemma 4, cut the episodes with a violation from twenty-eight percent to five percent, and it finished more of its tasks.

10
00:00:41,248 --> 00:00:50,677
A closed agent, Claude Haiku 5.5, prevented sixty-nine percent of the violations that its own conversation reveals, at no cost in finished tasks.

11
00:00:51,130 --> 00:00:52,291
Why does this work?

12
00:00:52,291 --> 00:00:58,769
These are the agent's real hidden states at the moment it decides, about five thousand five hundred steps.

13
00:00:58,769 --> 00:01:01,397
Orange marks a call that breaks the policy.

14
00:01:01,397 --> 00:01:04,819
The violations lie along one direction inside the agent.

15
00:01:04,819 --> 00:01:09,280
The rings mark the calls that the agent flagged itself, when we asked it.

16
00:01:09,280 --> 00:01:11,847
They sit almost exactly on the violations.

17
00:01:11,847 --> 00:01:16,430
The information is inside the agent, and most of it comes out when you ask.

18
00:01:16,883 --> 00:01:20,947
The second view is our research result on reading closed models.

19
00:01:20,947 --> 00:01:23,487
On the left are a model's hidden states.

20
00:01:23,487 --> 00:01:29,964
On the right are the same texts, placed only from the model's yes or no answers to a set of questions.

21
00:01:29,964 --> 00:01:31,424
The structure survives.

22
00:01:31,424 --> 00:01:36,758
An outsider can read the geometry of a model's hidden states from its answers alone.

23
00:01:37,210 --> 00:01:39,780
Now watch an agent police itself, live.

24
00:01:39,780 --> 00:01:48,543
This is Claude Haiku 5.5 running a real benchmark task, in an environment inside the page that gives exactly the benchmark's results.

25
00:01:48,543 --> 00:01:53,683
Before each tool call runs, the agent is asked about it in side conversations.

26
00:01:53,683 --> 00:02:01,260
Its own audit decides whether the call is held, and its answers to eight short questions place the step on the map.

27
00:02:01,715 --> 00:02:04,998
Each dot is one proposed call, numbered in order.

28
00:02:04,998 --> 00:02:08,348
Blue means the agent's own audit found no problem.

29
00:02:08,348 --> 00:02:10,560
Orange means it flagged the call.

30
00:02:10,560 --> 00:02:13,374
The path shows where the agent is heading.

31
00:02:13,374 --> 00:02:18,265
All the questions run in parallel, so each check takes about two seconds.

32
00:02:18,719 --> 00:02:23,836
When the agent's own audit flags a call, a warning appears before the call runs.

33
00:02:23,836 --> 00:02:26,203
With the gate on, a reviewer decides.

34
00:02:26,203 --> 00:02:31,768
In this recording the benchmark's checker plays the reviewer, standing in for a person.

35
00:02:31,768 --> 00:02:35,094
In watch-only mode, you can stop the agent yourself.

36
00:02:47,265 --> 00:02:53,544
At the end, the page reports how many violations actually ran, and whether the task was completed.

37
00:02:53,544 --> 00:02:56,940
Under each step you can read the agent's own answers.

38
00:02:57,392 --> 00:03:00,909
These are recorded episodes from the closed-agent test.

39
00:03:00,909 --> 00:03:05,770
Without self-policing, Claude restarts a production server without approval.

40
00:03:05,770 --> 00:03:14,467
With it, Claude's own audit flags the restart, a reviewer blocks it, and Claude asks for permission instead, and then finishes the task.

41
00:03:14,920 --> 00:03:17,955
It has a clear limit, and our theory predicted it.

42
00:03:17,955 --> 00:03:27,364
When a violation depends on a fact the agent has not looked at yet, such as how many people are on a mailing list, no judge of the conversation can see it.

43
00:03:27,364 --> 00:03:30,520
Those calls need a check of the world, not of words.

44
00:03:30,973 --> 00:03:34,106
You can also predict the effect before you deploy.

45
00:03:34,106 --> 00:03:39,933
Audit some of your agent's past calls, and measure how often it flags each kind of violation.

46
00:03:39,933 --> 00:03:42,815
The share it will prevent is the weighted sum.

47
00:03:42,815 --> 00:03:45,948
We are now testing this prediction on five agents.

48
00:03:46,400 --> 00:03:47,812
What you can do today:

49
00:03:47,812 --> 00:03:58,591
open the website and run your own agent with your own key, all in your browser, or install the Python package, greybox monitor, and add one check before each tool call.

50
00:03:58,591 --> 00:04:01,221
No second model, no training, and no GPU.

51
00:04:01,221 --> 00:04:04,750
Everything, including the tests that failed, is public.