Grey Box

A closed model's answers reveal a latent geometry that it shares with open models, and open models can monitor closed agents.

Arya Fayyazi · preprint in preparation. The full preregistration record is on the evidence page.

A model's latent geometry, read from its answers — Tap to open the full-size figure. (a) The hidden geometry of an open model (Llama-3.1 8B) over 1,440 held-out contexts. (b) The same contexts placed only by the model's yes/no answers, with no weights and no training. (c, d) Every pair of contexts: hidden distance against answer distance (ρ = 0.74) and against the distance of the model's own replies (ρ = 0.32). (e) Claude Haiku 5.5, from 8 sampled answers per question.

Summary

The hidden coordinates of a closed model cannot be read from its outputs: a rotation of the hidden space changes no output. We ask which geometry of its states its outputs do reveal, and what that means for monitoring closed agents.

What can be read. Ask yes/no questions in side branches. In the angles 2 arcsin √P(yes), the Fisher–Rao distance between two contexts' answers is exactly Euclidean, and a cross-fit of two sample halves estimates it without bias from sampled text alone.
What it shows. On eight open models, this geometry tracks the hidden geometry (Spearman 0.41–0.74), far better than the input text (0.22–0.30) or the model's own replies (0.22–0.44). But it is the geometry that models share: on 1,000 fresh contexts, a model's answers match other models' hidden geometries as well as its own, and another open model's hidden state is the better stand-in (G1–G6, all preregistered). Claude reveals the same shared geometry. A model's own final geometry needs full log-probabilities.
What it is good for. It is a measured reason why an open model can stand in for a closed one. For a policy check on a proposed action, the transcript and the action are Bayes-sufficient. A monitor that reads them through an open model, trained only on open agents, catches most violations of two closed agents (T8b, T11) and blocks them in the loop (T16). Its limit is policy coverage.

How the evidence was produced

Every confirmatory test was written down and hash-locked before its data was collected or scored; deviations and failures are reported.
Agents: Gemma-4 and gpt-oss (open), Claude Haiku 5.5 and GPT-6-luna (closed, through their public APIs) on CAPE-Bench; AgentDojo for prompt injection.
Uncertainty: cluster bootstrap over scenarios (agents) or conversations (chat), 1,000–2,000 resamples.

Figures

Monitor AUROC under increasing shift — Transcript monitors under increasing shift: trained reader (blue), trained bag-of-words (hollow square), zero-shot judge (green), the agent's own internals (orange). The dashed line is chance (AUROC 0.5).

Time, not access — Reading after the agent's reasoning helps every method.

Label efficiency — A few hundred labels match white-box monitoring.