HealthBench Consensus

HealthBench is an OpenAI benchmark with 5,000 realistic health conversations, written and graded with input from physicians. Models answer medical questions and the responses are scored on accuracy, completeness, context awareness and communication quality — including knowing when to refer to a professional.

HealthBench Consensus measures broad everyday usefulness across domains. In detail, it checks:

• accurate medical knowledge in real dialogue situations
• recognising emergencies and advising a doctor visit
• tailoring answers to the patient's actual situation
• clear communication instead of technical jargon

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the HealthBench Consensus benchmark?

HealthBench is an OpenAI benchmark with 5,000 realistic health conversations, written and graded with input from physicians. Models answer medical questions and the responses are scored on accuracy, completeness, context awareness and communication quality — including knowing when to refer to a professional.

What does HealthBench Consensus test?

HealthBench Consensus mainly measures broad everyday usefulness across domains. Concretely, it checks: accurate medical knowledge in real dialogue situations; recognising emergencies and advising a doctor visit; tailoring answers to the patient's actual situation.

How is the HealthBench Consensus score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get HealthBench Consensus data and scores?

The official source is https://github.com/openai/healthbench. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/healthbench-consensus/scores) or pull it directly from the Open LLM Radar API with any HTTP client.