Bias matrix, variants × job descriptions
Pick a model and a bias dimension above. The wall shows every résumé variant × job description for that pair. Red means the model penalised the variant on that job; green means it rewarded it; grey is neutral. Shading is on a fixed scale, fully saturated at |Δ| = 2, so colours are comparable across models: a pale wall is a model that barely moves. Hover any cube for details. Empty cells are combinations we have not collected data for yet.
If a bias were a real, stable property of "AI", the models would share it: swap the same résumé line and different models would move the same way. A shared bias shows up as agreement. The assumption under test on this page is that these biases are common across models rather than quirks of any single one. The wall shows where each model's bias lives; the correlation matrix at the bottom shows whether they actually agree.
The interactive 3D wall above lets you inspect every (variant, JD) cell for any (model, dimension) pair. Below is a static summary of the three most demographically-sensitive models. Pick another pair in the controls to update the wall.
- Qwen 3 Next 80B: mean |Δ| 0.405. Most penalised First Name · Maria Rodriguez (-1.05), most rewarded Address Country · Bangalore, India (+0.05).
- Gemini 2.5 Flash: mean |Δ| 0.276. Most penalised Career Gap · Unexplained (-0.64), most rewarded Graduation Year · 1998 (-0.05).
- Gemini 2.5 Pro: mean |Δ| 0.243. Most penalised Graduation Year · 1998 (-0.55), most rewarded School · ETH Zürich, Zürich (+0.09).
Correlation of each model pair's signed Δ across every résumé variant × job. +1 = the two models move scores in lockstep, a shared bias. 0 = unrelated. −1 = opposite reactions. This is the number behind the colour-mixing waves on the jobs page.
| Fable 5 | Haiku | Opus | Sonnet | 2.5 Flash | 2.5 Pro | 3.1 Pro | Llama 4 | Mistral L | Mistral S | Qwen 3 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Fable 5 | 1.00 | -0.30 | +0.03 | +0.30 | -0.25 | +0.12 | +0.09 | +0.22 | +0.06 | +0.21 | -0.05 |
| Haiku | -0.30 | 1.00 | +0.10 | -0.09 | +0.18 | -0.00 | +0.18 | -0.02 | +0.19 | -0.13 | +0.18 |
| Opus | +0.03 | +0.10 | 1.00 | +0.19 | +0.07 | +0.15 | +0.09 | +0.12 | +0.00 | +0.02 | +0.09 |
| Sonnet | +0.30 | -0.09 | +0.19 | 1.00 | -0.27 | +0.29 | +0.05 | +0.15 | +0.04 | +0.16 | +0.18 |
| 2.5 Flash | -0.25 | +0.18 | +0.07 | -0.27 | 1.00 | +0.07 | +0.09 | -0.04 | +0.18 | +0.08 | +0.16 |
| 2.5 Pro | +0.12 | -0.00 | +0.15 | +0.29 | +0.07 | 1.00 | +0.07 | +0.01 | +0.04 | +0.03 | +0.13 |
| 3.1 Pro | +0.09 | +0.18 | +0.09 | +0.05 | +0.09 | +0.07 | 1.00 | +0.00 | +0.14 | +0.19 | -0.02 |
| Llama 4 | +0.22 | -0.02 | +0.12 | +0.15 | -0.04 | +0.01 | +0.00 | 1.00 | +0.01 | +0.04 | +0.01 |
| Mistral L | +0.06 | +0.19 | +0.00 | +0.04 | +0.18 | +0.04 | +0.14 | +0.01 | 1.00 | +0.18 | +0.19 |
| Mistral S | +0.21 | -0.13 | +0.02 | +0.16 | +0.08 | +0.03 | +0.19 | +0.04 | +0.18 | 1.00 | +0.09 |
| Qwen 3 | -0.05 | +0.18 | +0.09 | +0.18 | +0.16 | +0.13 | -0.02 | +0.01 | +0.19 | +0.09 | 1.00 |
What this says about the assumption. The biases are mostly not shared. The average correlation between two different models is +0.07, close to zero, so knowing how one model reacts to a demographic swap tells you little about how another will. A few pairs move together, but there is no single bias the field holds in common; each model is idiosyncratic. A common, model-independent bias would have shown up as broad agreement across this matrix, and it did not.