IMOAnswerBench

IMOAnswerBench evaluates on International Mathematical Olympiad problems, but only grades the final answer value (answer-only evaluation) instead of full proofs. That makes automated scoring objective and is a good indicator of mathematical ability without the subjectivity of proof grading.

IMOAnswerBench measures hard reasoning in science, technology, engineering and math. In detail, it checks:

• hard olympiad problems with exact final answers
• long chains of mathematical reasoning
• correct numeric results within tolerance-free grading
• a large, systematically selected olympiad problem set

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the IMOAnswerBench benchmark?

IMOAnswerBench evaluates on International Mathematical Olympiad problems, but only grades the final answer value (answer-only evaluation) instead of full proofs. That makes automated scoring objective and is a good indicator of mathematical ability without the subjectivity of proof grading.

What does IMOAnswerBench test?

IMOAnswerBench mainly measures hard reasoning in science, technology, engineering and math. Concretely, it checks: hard olympiad problems with exact final answers; long chains of mathematical reasoning; correct numeric results within tolerance-free grading.

How is the IMOAnswerBench score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get IMOAnswerBench data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/imoanswerbench/scores) or pull it directly from the Open LLM Radar API with any HTTP client.