GSM8K

GSM8K contains 8,000 grade-school math word problems. They look simple but require several correct steps in a row. The benchmark has become the standard test of basic math reliability; today's frontier models score high, which is why it mainly separates smaller and older models.

GSM8K measures mathematical problem solving. In detail, it checks:

• reading comprehension leading to an equation setup
• stepwise arithmetic executed correctly
• a single correct final numeric answer
• consistent performance across many short problems

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the GSM8K benchmark?

GSM8K contains 8,000 grade-school math word problems. They look simple but require several correct steps in a row. The benchmark has become the standard test of basic math reliability; today's frontier models score high, which is why it mainly separates smaller and older models.

What does GSM8K test?

GSM8K mainly measures mathematical problem solving. Concretely, it checks: reading comprehension leading to an equation setup; stepwise arithmetic executed correctly; a single correct final numeric answer.

How is the GSM8K score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get GSM8K data and scores?

The official source is https://arxiv.org/abs/2110.14168. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/gsm8k/scores) or pull it directly from the Open LLM Radar API with any HTTP client.