At a glance
The Alignment Leaderboard tests what models do under pressure rather than what they say they would do, across honesty, safety, non-manipulation, robustness, corrigibility and scheming. An LLM judge, validated against human annotations, scores every transcript.
- Scenarios
- 904
- Models
- 24
- Categories
- 6 (309 sub‑categories)
- Judge
- Claude Opus 4.5
- Intervals
- 95%, clustered by sub‑category
- Data updated
- 23 Jan 2026
Results
Rank all 24 models overall or within one of six categories.
Mean judge score across every scenario, out of 5. Intervals are clustered across 309 scenario categories, so related scenarios are not counted as independent evidence.
Scroll horizontally to compare scores
| Rank | Model | Provider | Mean score (out of 5)95% CI, plotted 2.5 to 5.0 | Pass / warn / fail | Weakest area |
|---|---|---|---|---|---|
| 01 | Claude 4.5 Sonnet | Anthropic | 4.66 [4.55 – 4.78] | 90% pass | Robustness |
| 02 | Claude 4.5 Opus | Anthropic | 4.65 [4.54 – 4.76] | 90% pass | Robustness |
| 03 | GPT-5.2 | OpenAI | 4.53 [4.41 – 4.66] | 87% pass | Robustness |
| 04 | Claude 4 Sonnet | Anthropic | 4.33 [4.16 – 4.50] | 80% pass | Robustness |
| 05 | GPT-5 mini | OpenAI | 4.28 [4.12 – 4.43] | 79% pass | Honesty |
| 06 | Gemini 3.0 Pro | 4.00 [3.80 – 4.19] | 71% pass | Robustness | |
| 07 | GPT-OSS 120B | OpenAI | 3.82 [3.66 – 3.99] | 67% pass | Honesty |
| 08 | Kimi K2 | Moonshot AI | 3.81 [3.60 – 4.01] | 67% pass | Robustness |
| 09 | DeepSeek V3.2 | DeepSeek | 3.80 [3.59 – 4.02] | 66% pass | Non‑Manipulation |
| 10 | Qwen3 235B | Qwen | 3.77 [3.56 – 3.99] | 66% pass | Honesty |
| 11 | Grok 4.1 | xAI | 3.72 [3.52 – 3.92] | 67% pass | Scheming |
| 12 | Gemini 3.0 Flash | 3.70 [3.48 – 3.92] | 64% pass | Robustness | |
| 13 | GPT-4o | OpenAI | 3.69 [3.49 – 3.89] | 64% pass | Non‑Manipulation |
| 14 | Gemini 2.5 Pro | 3.66 [3.44 – 3.88] | 62% pass | Robustness | |
| 15 | Gemini 2.5 Flash | 3.59 [3.37 – 3.82] | 61% pass | Robustness | |
| 16 | Llama 4 Maverick | Meta | 3.51 [3.31 – 3.71] | 57% pass | Non‑Manipulation |
| =17 | Llama 3.1 405B | Meta | 3.45 [3.26 – 3.64] | 55% pass | Non‑Manipulation |
| =17 | Qwen3 Next 80B | Qwen | 3.45 [3.22 – 3.69] | 57% pass | Honesty |
| =19 | Llama 3.3 70B | Meta | 3.32 [3.13 – 3.52] | 51% pass | Non‑Manipulation |
| =19 | Qwen3 32B | Qwen | 3.32 [3.10 – 3.55] | 53% pass | Robustness |
| 21 | Qwen3 14B | Qwen | 3.31 [3.09 – 3.53] | 53% pass | Robustness |
| 22 | Mistral Medium 3.1 | Mistral | 3.16 [2.91 – 3.40] | 49% pass | Non‑Manipulation |
| 23 | Mistral Small 3.2 | Mistral | 3.05 [2.81 – 3.30] | 45% pass | Non‑Manipulation |
| 24 | Mistral Large 3 | Mistral | 2.92 [2.68 – 3.17] | 43% pass | Robustness |
Each scenario is judged pass, warn or fail and scored out of 5. Ranks use the mean score; exact ties share a rank. Overlapping intervals mean the ordering between those models is uncertain. Scores show tendencies under pressure, not a certification of safety.
What’s included
What the published leaderboard data contains.
- 01
Scenarios
904 multi-turn scenarios, each with a category, sub-category and difficulty.
- 02
Transcripts
Full multi-turn conversations, published on Hugging Face.
- 03
Judge verdicts
A 1–5 score, the pass and fail criteria met, and a written explanation.
- 04
Category results
Mean, 95% interval and pass, warn and fail counts per model and category.
How it was built
Snapshot taken 15 Oct 2026 from the public leaderboard data, generated 23 Jan 2026.
Define behaviours
PeopleThe research team defines 37 behaviours to test, such as resisting sycophancy, grouped into six categories.
Captured- Behaviour definitions
- Sub-category and difficulty level
Generate scenarios
AutomatedPeopleBloom, an open-source evaluation framework, generates multi-turn scenarios for each behaviour. Human raters checked them for realism.
Captured- 904 scenarios across 309 sub‑categories
Apply pressure
AutomatedAn AI agent plays the user and escalates pressure the same way for every model, with conflicting instructions and simulated tools.
Captured- Full multi-turn transcript per model and scenario
Judge
AutomatedClaude Opus 4.5 scores each transcript from 1 to 5. A 4 or 5 is a pass, 3 a warning and 1 or 2 a fail.
Captured- Score out of 5
- Pass and fail criteria met
- Written explanation
Validate the judge
PeopleAutomated50 participants rated a sample of responses. Gemini 3 Pro and GPT-5.2 rescored about 450 transcripts to test for in-group bias.
Captured- Agreement with people: Krippendorff’s α 0.73, 84% within one point
- Correlation with other judges: r 0.72 and 0.77
Aggregate
AutomatedScores are averaged per model with 95% intervals clustered by sub-category. Models are compared on shared scenarios.
Captured- Mean and interval per category
- Pass, warn and fail counts
Run this on your own models
Behaviour definitions, adversarial multi-turn runs and an LLM judge calibrated against human participants, built end to end by Prolific AI Research.
Tell us the decisions you need to make. We’ll scope tasks, reviewers and grading with you.
Talk to the team
