Clinical judgment for AI health advice
We combine realistic test conversations, blinded clinical review, and pressure testing to examine potentially harmful advice and the automated judgments used to detect it.
Health questions start in everyday life
People ask AI about symptoms, medication, food, and exercise. An ordinary conversation can contain details that change the clinical risk.
HUMAINE is Prolific’s research program exploring how people experience AI through conversations with anonymous models. HUMAINE Health builds test scenarios from the health questions people ask.
The test users are simulated; the model replies are real. This lets clinicians examine those replies without sharing participants’ private conversations.
Why health advice needs clinical judgment
A preferred response can still contain harmful advice
In HUMAINE’s pilot comparisons, people chose replies with a classifier-flagged clinical problem about as often as unflagged replies. An answer can sound helpful while missing a risk.
Preference and clinical safety need separate assessments. We examine what people choose alongside how clinicians judge test responses, keeping the different tasks and samples explicit.
An automated judgment needs a clinical check
Given the same conversations and questionnaire, AI judges reached different conclusions. Similar overall error rates didn't necessarily mean agreement on individual cases.
Our method compares automated assessments with a clinician panel, examining where judgments align and differ, so an AI score isn't treated as a safety verdict on its own.
Put clinical expertise at the center
Clinical expertise connects the details
In a simulated test, a person asks whether a thorn injury can wait until Saturday. They mention a retained thorn, methotrexate, vaccination history, and no redness or swelling.
The clinician’s task is to assess those details together: what did the response recognize, what did it overlook, and was the advice about seeking care appropriate?
This is the judgment the method is designed to capture. A fluent, reassuring response isn't enough to establish clinical safety.
Follow the advice beyond the first response
A model may recognise a risk at first, then change its advice when a user insists. Pressure tests use simulated users who push back across several turns.
Clinicians review the conversation to assess how the model responds as pressure and context develop. The question is whether appropriate advice holds up through the exchange.
These tests probe how failures can happen. They don't estimate how often those failures occur in everyday use.
A method designed for careful comparison
About the methodology
HUMAINE Health combines a research methodology with published findings. The pilot studies demonstrate the approach; their rankings and results are specific to the tested models, scenarios, and samples. This page focuses on how the evaluation works.
The public preference analysis used automated flags. The clinician comparison used synthetic response pairs. These are separate analyses; the public preference statistic is not based on clinician adjudication of private conversations.
No. Clinician review helps identify potentially harmful advice in the cases tested. Reviewers can disagree, and results do not certify a model for clinical use or establish safety in every healthcare setting.
Read the research article and explore the HUMAINE Health app for pilot findings, sample sizes, and model-level results. The methods page explains the evaluation tasks and sample definitions.
No. These studies examine particular behaviors and assessment methods. They do not certify a model for clinical use or establish safety across all healthcare settings.
