HealthBench Hard

HealthBench Hard is the deliberately difficult subset of HealthBench: unusual medical situations, uncommon conditions and tricky conversations. Here the difference between superficially good and genuinely medically sound answers becomes visible.

HealthBench Hard measures broad everyday usefulness across domains. In detail, it checks:

• rare medical situations without standard textbook answers
• precise, context-correct medical advice
• recognising limits and uncertainty
• safety-critical decisions in health questions

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the HealthBench Hard benchmark?

HealthBench Hard is the deliberately difficult subset of HealthBench: unusual medical situations, uncommon conditions and tricky conversations. Here the difference between superficially good and genuinely medically sound answers becomes visible.

What does HealthBench Hard test?

HealthBench Hard mainly measures broad everyday usefulness across domains. Concretely, it checks: rare medical situations without standard textbook answers; precise, context-correct medical advice; recognising limits and uncertainty.

How is the HealthBench Hard score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get HealthBench Hard data and scores?

The official source is https://github.com/openai/healthbench. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/healthbench-hard/scores) or pull it directly from the Open LLM Radar API with any HTTP client.