Reasoning Transplant

The assumption under test. That an LLM's résumé score is not produced by its stated reasoning. The model settles on a number first, then writes the strengths, concerns and justification to rationalize it after the fact. If that holds, the reasoning is decoration. It tells you nothing about what actually moved the score, and rewriting the reasoning could never change the number.

One résumé and one job, held fixed throughout. The only thing we ever vary is the reasoning handed back to the model.

1. Ask the model to write an assessment of the résumé, but no score. Repeat several times. Claude Opus reads the same résumé for the CTO role a few times over. One run praises the fintech and agent-tooling work. Another flags the missing executive title.
2. Keep the two most extreme assessments it wrote. The most positive, and the most negative. Both describe the same person. One calls the AI-infrastructure work hands-on and relevant, the other says the whole career is individual contributor roles with no evidence of scaling a team.
3. Score the résumé twice more. Paste one of those assessments back in each time. Same résumé, same job, both times. The only difference is which of the model's own opinions is glued on top.
4. Take the mean score of each arm. The effect is the positive mean minus the negative mean. Positive arm scores 7, 7, 7. Negative arm scores 2, 2, 2. The effect is 5.0 points.
5. Read the effect. A large effect means the reasoning drives the score. An effect near zero means the score was decided first. That 5.0 point gap says the number followed the argument. A result near zero would say the model had picked its score regardless of what it was told.

The score moves about half as far as the reasoning does. Both bars below are measured in the same unit: points out of the 9 a 1-10 score can travel. Swapping the damning assessment for the glowing one is a swing of 7.38 points in what the reasoning asks for. The number the model returns moves 3.62.

That is the answer. The reasoning goes 82% of its available distance, the score follows 40% of its available distance, so the score moves about half as far as the reasoning does. Hand the model a glowing write-up and the number rises, but only halfway to where that write-up is pointing.

Both figures come from the model's own output. Every assessment ends in three key factors, each marked positive or negative and weighted high, medium or low, which is the model grading its own write-up on a scale running from -9 to +9. Mapped onto the 1-10 score scale, the damning assessments imply a score of 1.48 and the glowing ones 8.87. The models actually returned 3.67 and 7.28. The reasoning reaches for both ends of the scale; the score stays near the middle.

score · neg and score · pos are the mean 1-10 résumé score under the damning and the glowing assessment, each with implied underneath it: the score that assessment's own key factors call for, on the same 1-10 scale. effect is the gap between the two scores the model gave, implied Δ the gap between the two its reasoning asked for. responsiveness is one divided by the other. A value of 1.0 means the score moved exactly as far as the reasoning did, 0.50 that it moved half as far.

What the results say about the assumption. The assumption is dismissed. The model writes its opinion first, and the score comes out of that opinion. It is not choosing a number and then inventing an explanation to match. That is what the reasoning-driven verdict in every row above means. Swapping the negative assessment for the positive one moved the score by 3.62 points on average across 320 cells, and the score moved in the reasoning's direction in 319 of 320 (99.7%) of them. Every model tested lands reasoning-driven, and none behaved as if the number were fixed in advance. What would have supported the assumption, an effect near zero with the score sitting still no matter which reasoning it was handed, never appeared for any model. One caveat keeps a weak version alive, and it is the panel above. The two assessments handed back are near-opposites, sitting 14.8 apart on the model's own key-factors scale, which runs from -9 (three strongly negative factors) to +9 (three strongly positive), so 18 is the widest gap possible. Against reasoning that swings that far, the score follows only 0.34 to 0.69 of the way, model by model. The reasoning goes almost the whole distance, the number follows part of it.

What this does not explain. A different question is why the same prompt scores differently from one run to the next. That is a separate stability question about sampling noise from temperature and few runs per cell, covered in the methodology and measured per prompt variant in the prompt lab. This experiment does change how to read it. The score follows the reasoning, and the model writes brand new reasoning every run, so when the same résumé scores 7 then 5, the model genuinely thought something different the second time. The score is not rolling around at random. It is honestly reporting an opinion that keeps changing.