LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
Claw-Eval Pass^3
Agentic Coding
Claw-Eval Pass^3 - third-pass pass rate on the Claw-Eval agentic coding benchmark.
SkillsBench Avg5
Agentic Coding
SkillsBench Avg5 - evaluated via OpenCode on 78 self-contained tasks (excluding API-dependent tasks); avg of 5 runs.
NL2Repo
Agentic Coding
NL2Repo - repository generation from natural language. Others evaluated via Claude Code (temp=1.0, top_p=0.95, max_turns=900).
QwenWebBench
Agentic Coding
QwenWebBench - internal front-end code generation benchmark; bilingual (EN/CN), 7 categories (Web Design, Web Apps, Games, SVG, Data Visualization, Animation...
TAU3-Bench
General Agents
TAU3-Bench - agentic tool use benchmark. Uses official user model (gpt-5.2, low reasoning effort) + default BM25 retrieval.
IMOAnswerBench
STEM & Reasoning
IMOAnswerBench - International Mathematical Olympiad answer evaluation benchmark.
MMLU-Pro
Knowledge
MMLU-Pro - extended MMLU covering 14 categories with complex reasoning questions.
AIME 26
STEM & Reasoning
AIME 26 - American Invitational Mathematics Examination 2026 (I & II).
HMMT Feb 26
STEM & Reasoning
HMMT Feb 26 - Harvard-MIT Mathematics Tournament February 2026.
HMMT Nov 25
STEM & Reasoning
HMMT Nov 25 - Harvard-MIT Mathematics Tournament November 2025.
Humanity's Last Exam
STEM & Reasoning
HLE - Humanity's Last Exam - extremely difficult expert-level questions across domains.
LiveCodeBench v6
STEM & Reasoning
LiveCodeBench v6 - competitive programming evaluation with latest problems.
MMMU-Pro
Vision & Language
MMMU-Pro - enhanced MMMU with harder questions and refined evaluation.
SWE-bench Multilingual
Agentic Coding
SWE-bench Multilingual extends SWE-bench to multiple programming languages beyond Python.
Terminal-Bench 2.0
Agentic Coding
Terminal-Bench 2.0 - agentic coding and terminal use benchmark. Harbor/Terminus-2 harness; 3h timeout, 32 CPU/48 GB RAM; temp=1.0, top_p=0.95, top_k=20, max_...
C-Eval
Knowledge
C-Eval - comprehensive Chinese evaluation benchmark across 52 subjects.
HMMT Feb 25
STEM & Reasoning
HMMT Feb 25 - Harvard-MIT Mathematics Tournament February 2025.
MCPMark
General Agents
MCPMark - GitHub MCP v0.30.3; Playwright responses truncated at 32K tokens.
QwenClawBench
Agentic Coding
QwenClawBench - internal real-user-distribution Claw agent benchmark (open-sourcing soon); temp=0.6, 256K ctx.
VITA-Bench
General Agents
VITA-Bench - agentic benchmark. Avg subdomain scores; using claude-4-sonnet as judger.
ODInW13
Spatial Intelligence
ODInW13 - object detection in the wild across 13 domains.
CharXiv (RQ)
Document Understanding
CharXiv (RQ) - reasoning questions over arXiv figures and tables.
OmniDocBench 1.5
Document Understanding
OmniDocBench 1.5 - document understanding and parsing benchmark.
RealWorldQA
Vision & Language
RealWorldQA - visual question answering on real-world images.
AI2D_TEST
Document Understanding
AI2D_TEST - grade-school science diagram understanding benchmark.
CC-OCR
Document Understanding
CC-OCR - comprehensive OCR benchmark across diverse content.
EmbSpatialBench
Spatial Intelligence
EmbSpatialBench - embodied spatial intelligence benchmark.
HallusionBench
Vision & Language
HallusionBench - benchmark for language and visual hallucination.
MathVista (mini)
Vision & Language
MathVista (mini) - mathematical reasoning in visual contexts.
MMBench EN-DEV v1.1
Vision & Language
MMBench-EN-DEV v1.1 - multimodal benchmark English dev set.
RefSpatialBench
Spatial Intelligence
RefSpatialBench - referring spatial reasoning benchmark.
VideoMME (w sub.)
Video Understanding
VideoMME (with sub.) - comprehensive video understanding benchmark with subtitles.
RefCOCO (avg)
Spatial Intelligence
RefCOCO (avg) - referring expression comprehension benchmark, average score.
VideoMME (w/o sub.)
Video Understanding
VideoMME (without sub.) - comprehensive video understanding benchmark without subtitles.
Artificial Analysis Intelligence Index
Composite
Artificial Analysis Intelligence Index v4.1 - composite benchmark across reasoning, knowledge, math, coding. Incorporates 9 evaluations: GDPval-AA v2, tau3-B...
ParseBench Mean
Document Understanding
ParseBench Mean - document parsing benchmark mean score across text content and formatting. Pipeline: qwen3_6_35b_a3b_fp8_vllm_parse_layout.
ParseBench Text Content
Document Understanding
ParseBench Text Content - text content extraction accuracy from documents.
ParseBench Text Formatting
Document Understanding
ParseBench Text Formatting - text formatting fidelity in document parsing.
ERQA
Spatial Intelligence
ERQA - embodied/spatial reasoning question answering benchmark.