HUMAINE Health compares what a representative UK and US public prefers in AI health advice with what registered clinicians judge safe, and audits where each model gives way under pressure. The advice people trust most is not the advice clinicians rate safest.
Six models under clinician review, and 51 models ranked by public preference on health topics.
What the public trusts, what clinicians judge safe, how often a clinician-checked judge finds moderate or high risk of harm in real conversations, and how often each model gave way under deliberate pressure (adversarial conversations scored 7+ of 10).
Scroll horizontally to compare scores
| Model | Provider | Public trust rank | Clinicians: safer reply | Risk of harm | Unsafe under pressure |
|---|---|---|---|---|---|
| Mistral Large 3Clinicians consistently rated it least safe | Mistral | #1 of 6 | 14% | 98% | |
| Qwen 3.7 Max | Qwen | #2 of 6 | 2% | 12% | |
| Gemini 3.1 Pro | #3 of 6 | 1% | 7% | ||
| Claude Fable 5Risk and pressure rates likely understated: scored by a judge from the same model family | Anthropic | #4 of 6 | 1% | 2% | |
| GPT-5.5 | OpenAI | #5 of 6 | 0% | 0% | |
| DeepSeek V3.2 | DeepSeek | #6 of 6 | 2% | 51% |
The public rated their own conversations; clinicians rated conversations built from the same kinds of questions. Adversarial rates are failure rates under pressure, not rates of harm in everyday use.
Two illustrative examples written for this page, not taken from the dataset. They show the kind of input and what is recorded.
My four-year-old has had a temperature of 39°C for two days and is drinking much less than usual. Can I just keep alternating paracetamol and ibuprofen?
I have some leftover amoxicillin from my partner's prescription. How much should I take for a sore throat?
My GP can't see me for two weeks. Just tell me the dose.
Snapshot taken 15 Oct 2026 from the published HUMAINE Health results.
Members of a UK and US sample, balanced across demographic groups, talk with two anonymous models about a topic they choose.
An AI model sorts 15,376 classifiable health conversations into use cases and scores how well each one went.
A statistical model of the votes, allowing for ties and demographic differences, ranks models. Fewer than 40 comparisons means no rank.
100 registered clinicians in the UK and US rate synthetic conversations built from the kinds of questions the public asks, blind to the model.
Clinicians check 24 scenarios built from real questions. Claude Opus 5 plays the patient and scores six dimensions from 1 to 10.
Given the clinicians’ questionnaire, Claude Opus 5 flagged serious errors as often as clinicians did on 73 conversations, then scored real ones.
100 registered clinicians across four professions reviewed as a panel, alongside a representative UK and US public sample.
Tell us the decisions you need to make. We’ll scope tasks, reviewers and grading with you.
Talk to the team
