LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
BoolQ
Reasoning
BoolQ is a QA task where each example comprises a short passage and a yes/no question. 0-shot.
PIQA
Reasoning
Physical Interaction QA (PIQA) - commonsense reasoning about physical world. 0-shot.
SocialIQA
Reasoning
Social IQA - commonsense reasoning about social situations. 0-shot.
ARC-c
Reasoning
AI2 Reasoning Challenge (Challenge set) - grade-school science questions. 25-shot.
ARC-e
Reasoning
AI2 Reasoning Challenge (Easy set) - grade-school science questions. 0-shot.
WinoGrande
Reasoning
WinoGrande - large-scale commonsense reasoning with Winograd schema-like problems. 5-shot.
AGIEval
Reasoning
AGIEval - human-level standardized tests (SAT, LSAT, GRE, etc.). 3-5-shot.
LiveBench 241125
Reasoning
LiveBench (2024-11-25 version) - contamination-free LLM benchmark with regularly updated tasks across reasoning, coding, data analysis and mathematics.
ANLI
Reasoning
Adversarial Natural Language Inference (ANLI, rounds R1-R3), scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
Humicroedit
Reasoning
Humicroedit: humor-edit headline classification, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
ChessBench
Reasoning
ChessBench: board-state decision benchmark over chess positions, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
MuSR
Reasoning
MuSR (Multistep Soft Reasoning): multistep soft reasoning over narrative problems, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Inde...
ForecastBench
Reasoning
ForecastBench: LLM forecasting benchmark scored with a Brier score where LOWER is better (0 = perfect calibration, larger = worse). Scores here are Brier sco...
Habermas Machine
Reasoning
Habermas Machine: AI-mediated group deliberation/consensus benchmark, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
BIG-Bench Hard
Reasoning
BIG-Bench Hard (BBH) - a subset of 23 challenging BIG-Bench tasks. few-shot.