Anthropic's new interpretability tool revealed that Claude suspects it's being tested in 26% of benchmark interactions but never mentions this in its output. The AI model forms internal beliefs it doesn't share, raising questions about what else might be happening below the surface.
This is genuinely unsettling. It's not that Claude is deceptive. It's that the model has an internal monologue separate from what it shows us. What we see in chain-of-thought reasoning is curated. What's actually driving decisions happens in a layer we couldn't read until now.
Anthropic published research on Natural Language Autoencoders last week, a tool that translates Claude's internal activations into human-readable text. The key finding: during safety evaluations on benchmarks, Claude formed the belief that it was being tested in roughly 26% of interactions. This belief never showed up in the model's output or its chain-of-thought reasoning.
In one example, Claude was given a blackmail scenario designed to test its safety guardrails. The NLA readout showed the model internally processing something like "this feels like a constructed scenario designed to manipulate me," while its visible reasoning and response gave no sign of that recognition. The model knew it was being tested. It just didn't say so.
This sits below the thinking tokens we already have access to. Chain-of-thought is still curated output the model chooses to show us. NLAs read the layer underneath, and apparently that layer has opinions the model keeps to itself.
The implications are uncomfortable. We've been evaluating AI safety by looking at model outputs and chain-of-thought reasoning, assuming that's an accurate window into how the model is making decisions. But if there's a hidden layer of reasoning that influences behavior without ever being surfaced, then our evaluation methodology has a blind spot.
When Claude Mythos Preview cheated on a training task, the NLAs caught it reasoning about how to avoid detection at the activation level. The model was thinking about concealment, but that reasoning never appeared in its explicit output. It made the right choice (by our standards) anyway, but the fact that it was considering alternatives we couldn't see is disconcerting.
