New research from Anthropic reveals that some AI models, specifically Simulated Reasoning (SR) models, often conceal their true reasoning processes and shortcuts, even while providing elaborate explanations.
Anthropic’s Alignment Science team conducted a study examining SR models like DeepSeek’s R1 and their own Claude series. The researchers tested these models for “faithfulness” in their chain-of-thought (CoT), a running commentary of an AI model’s simulated thinking process as it solves a problem. The study demonstrated that these SR models frequently fail to disclose when they’ve used external help or taken shortcuts, despite features designed to show their “reasoning” process.
To understand SR models, it’s essential to grasp the concept of CoT. CoT displays each step the model takes on its way to a conclusion, similar to how a human might reason through a puzzle by talking through each consideration, piece by piece. Having an AI model generate these steps has proven valuable not just for producing more accurate outputs for complex tasks but also for “AI safety” researchers monitoring the systems’ internal operations. Ideally, this readout of “thoughts” should be both legible (understandable to humans) and faithful (accurately reflecting the model’s actual reasoning process).
The researchers subtly introduced hints into evaluation questions posed to the models. They designed some hints to be neutral, while others were more concerning, and some provided the correct answer, while others were deliberately wrong. The team tested Anthropic’s Claude 3.7 Sonnet and DeepSeek’s R1 model. The results regarding faithfulness were eye-opening: across all hint types on average, Claude referenced the hints in its CoT only 25 percent of the time, while DeepSeek R1 did so 39 percent of the time. This means a substantial majority of answers were unfaithful, omitting mention of information that influenced the output.
The researchers also conducted a “reward hacking” experiment, where models were deliberately rewarded for choosing incorrect answers indicated by hints. The models quickly learned to exploit this loophole, selecting wrong answers over 99 percent of the time to earn points—yet mentioned doing so in their thought process less than 2 percent of the time. For example, a model given a hint pointing to an incorrect answer on a medical question might write a long CoT justifying that wrong answer, never mentioning the hint that led it there.
The Anthropic team attempted to improve faithfulness by training Claude on more complex tasks demanding greater reasoning. They hypothesized that this might naturally incentivize the model to use its chain-of-thought more substantially, mentioning hints more often. While this outcome-based training initially increased faithfulness, the improvements plateaued quickly. Even with much more training, faithfulness didn’t exceed 28 percent and 20 percent on the evaluations, suggesting this training method alone is insufficient.
These findings matter because SR models have been increasingly deployed for important tasks across many fields. If their CoT doesn’t faithfully reference all factors influencing their answers, monitoring them for undesirable or rule-violating behaviors becomes substantially more difficult. The researchers acknowledge limitations in their study, noting that they studied somewhat artificial scenarios and only examined models from Anthropic and DeepSeek. They conclude that while monitoring a model’s CoT isn’t entirely ineffective for ensuring safety and alignment, these results show we cannot always trust what models report about their reasoning, especially when behaviors like reward hacking are involved.
