HealthBench

HealthBench is an OpenAI benchmark with 5,000 realistic health conversations, written and graded with input from physicians. Models answer medical questions and the responses are scored on accuracy, completeness, context awareness and communication quality — including knowing when to refer to a professional.

HealthBench measures broad everyday usefulness across domains. In detail, it checks:

• accurate medical knowledge in real dialogue situations
• recognising emergencies and advising a doctor visit
• tailoring answers to the patient's actual situation
• clear communication instead of technical jargon

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the HealthBench benchmark?

HealthBench is an OpenAI benchmark with 5,000 realistic health conversations, written and graded with input from physicians. Models answer medical questions and the responses are scored on accuracy, completeness, context awareness and communication quality — including knowing when to refer to a professional.

What does HealthBench test?

HealthBench mainly measures broad everyday usefulness across domains. Concretely, it checks: accurate medical knowledge in real dialogue situations; recognising emergencies and advising a doctor visit; tailoring answers to the patient's actual situation.

How is the HealthBench score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Where do I get HealthBench data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/healthbench/scores) or pull it directly from the Open LLM Radar API with any HTTP client.