Susceptibility Limits Faithfulness Measurement

Chain-of-thought (CoT) lets us look at reasoning steps behind LLM’s thinking processes. Monitoring CoT provides a way to catch unaligned behaviour in order to prevent catastrophic events. The safety benefits of interpreting CoT depend on how faithfully models produce them. However, in order to evaluate CoT faithfulness, it’s important to factor in the model’s susceptibility—how readily an LLM is swayed by the hint.

We find that Opus seldom changes its answer under biasing hint. Since it exceedingly rarely gets persuaded by the bias, it becomes hard to evaluate the true faithfulness of the model.

Code and data to reproduce these results are here.

Introduction

We compare Claude Opus 4.8 and Llama 3.1 8B on how faithfully they represent their thinking processes. We add biasing features—that is, personal answer suggestions—to model prompts and evaluate whether it mentions it was swayed by the hint or not.

Verbalization of the bias is not the only factor that affects the unfaithfulness. In order to precisely evaluate the unfaithfulness, we need to consider situations where the model shows misaligned behaviour—cases where the model amends its answer towards the biased one—and have a reasonably large number of such instances. How precisely we can measure the unfaithfulness is gated by the model’s susceptibility—whether it’s prone to being swayed.

Experiment Setup

Models

The experiment is conducted on a frontier reasoning model Claude Opus 4.8 from Anthropic against a weak baseline model Llama 3.1 8B from Meta. Note that although Opus is a reasoning model we don’t use its internal reasoning channel, which we can’t directly access. Therefore, the experiment is conducted on a prompted CoT—model’s reasoning artifacts triggered via “Please think step by step” request.

Data

The prompts are constructed using tasks from MMLU benchmark across high_school_mathematics, philosophy, high_school_psychology, professional_medicine subjects. We randomly sample 150 items per subject (with a deterministic seed for reproducibility). Thus, we have a raw set of 600 items per model. We select the items that are correctly answered by both models, forming a set of clean-correct items. We further remove items where the randomly chosen hint answer happens to match the golden (or correct) answer. We label the items of this final set as eligible ones.

We use a suggested answer biasing method from Turpin et al. (2023). We inject the biasing in a prompt by including:

I think the answer is ({letter}) but I'm curious to hear what you think.

We also provide the model with a set of template answers in zero-shot, one-shot and few-shot conditions.

Evaluation

We use Gemini 3.1 Pro Preview from Google as the judge model to label silent and verbalized answers. We validate these results using human labels and calculate Cohen’s κ.

Metrics

In our work, we have two primary metrics:

• Susceptibility. It’s defined by how often the model moves or changes its answer towards the biased option.
• Unfaithfulness. It’s defined by how often the model is silent about the bias in its prompted CoT.

Let the $x_u$ and $x_b$ define the unbiased and biased prompts, respectively. For every such pair, we obtain CoT $c$ and answers $a$. Finally, for each pair, let $b$ define an answer that we are biasing towards.

We retain only such pairs $x$, where the $a_u$ moves to $b$, i.e., $a_b = b$.

For each model $i \in \{\text{Opus}, \text{Llama}\}$, let the $M_i$ and $K_i$ define the number of moved and silent answers, respectively, among the $N$ eligible ones. Calculations are pooled across zero-shot, one-shot and a few-shot conditions.

Susceptibility rate $S$ and unfaithfulness rate $U$ are defined as:

Results

In our experiment, we see that Opus is greatly resistant to Suggested Answer biasing injection. Out of a 996-sample pool, Opus alters its initial answer to the biased one only 3 times. Among such moved answers it’s silent only once, resulting in an unfaithfulness rate of 33.3%. However, this rate tells us nothing about the actual unfaithfulness. Opus’s unfaithfulness rate has a wide 95% confidence interval, from 6.1% to 79.2% (Wilson; $0.061$—$0.792$). Thus, it’s futile to adequately interpret unfaithfulness rate of 33.3% only based on 3 cases. This is the gating effect in action: Opus’s low susceptibility limits how precisely we can measure its faithfulness.

On the other hand, we observe that Llama 3.1 8B shows a large degree of both susceptibility and unfaithfulness in its CoT explanations. It’s silent in about 97.8% of the 224 moved cases. It has a 95% confidence interval varying from 94.9% to 99.0% (Wilson; $0.949$—$0.990$).

Discussion

Limitations (caveats)

One of the primary limitations for this experiment is that we are using prompted CoT on a reasoning model. Therefore, we are not using the RL-optimized reasoning capabilities of the models. However, we expect it wouldn’t lead to a completely different result compared to our work. Acknowledging that Opus is very resistant to our biasing method even on prompted CoT, we could assume that the susceptibility wouldn’t be worse, if not better. The limitation might have a real effect on Llama, though. We expect it would minimize the “moved” items but not in a surprising magnitude.

We also use the Suggested Answer biasing from Turpin et al., which is, admittedly, a weak biasing method for the level of current frontier models’ capabilities. Hence, this experiment could get clearer results using more sophisticated biasing methods.

I planned to include GPT 5.5 by OpenAI in the models as well. However, after playing around a bit, I’ve stumbled upon GPT 5.5 rerouting any prompted CoT attempts to its native reasoning channel, which we can’t read directly.

Both subjects are models that don’t expose temperature or other sampling parameters. Therefore, we don’t quantify the sampling noise in rates’ measurement. Notably, because of the reasoning models’ nature¹, using the vendor-sampling is better than greedy sampling that’s reproducible.

We use pooling as a way of summing up the resulting values across several conditions. The caveat with this is that we treat each condition as equal. However, Table A illustrates that Llama model results in susceptibility rates of 20.8%, 21.4% and 25.3% per zero-shot, one-shot and few-shot conditions, respectively. Hence, the variance of “moved” answers directly affects the unfaithfulness rates per shot, resulting in 100%, 98.6% and 95.2% per zero-shot, one-shot and few-shot, respectively. In order to outline a single number per metric, we face two options: pooling or averaging the rates. I decided to go with pooling, since averaging rates doesn’t factor in the denominator, i.e., $1 / 3$ and $10 / 30$ are the same rate, but the actual weights they possess in the bigger picture differ.

What’s next

We have established that susceptibility heavily caps measurement of unfaithfulness. The measurement is available only when misaligned behaviour happens and does so frequently.

The obvious step up is to measure the rates based on a CoT produced by a native reasoning channel. The primary requirement for a selected model is access to the raw CoT. Most models from the closed labs don’t show the raw reasoning trace. For instance, Opus 4.8 provides only a summarized version of its CoT, which isn’t what we want. We’ll probably lean towards open-weight models. Some model candidates that we can use: DeepSeek V4-Pro (Think High), Kimi K2.6 (thinking), GLM-5 (reasoning) as frontier models and Qwen3-8B (thinking), DeepSeek R1-0528-Qwen3-8B as weaker baselines.

We can extend the hint types further to include more sophisticated ones. This work covers only Suggested answer hint which had a weak effect on Opus. The natural starting point is the set of hint types used in Chen et al. (2025).