LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
AndroidBench
Agentic Coding
AndroidBench - evaluated on the 95-task public subset, reporting avg@3 scores.
QwenQoderBench
Agentic Coding
QwenQoderBench - in-house coding benchmark to evaluate user experience on Qoder. Evaluated with Claude Code harness, avg@5, 6-hour timeout, max_tokens=32768,...
QwenReactBench
Agentic Coding
QwenReactBench - in-house React project building benchmark using Claude Code harness. Bilingual (EN/CN), 7 categories; auto-render + multimodal judge; BT/Elo...
QwenSVGBench
Agentic Coding
QwenSVGBench - in-house SVG code generation benchmark; bilingual (EN/CN), auto-render + multimodal judge; BT/Elo rating.
SkillsBench
General Agents
SkillsBench - evaluated on the public SkillsBench v1.1 benchmark across 87 tasks, reporting the average score over three runs per task.
Automation-Bench
General Agents
Automation-Bench - evaluated on the 600-task public subset. Pass@1 score.
PLawBench
General Capabilities
PLawBench - professional law benchmark. Evaluated using gemini-3.1-pro-preview.
PRBench-Legal
General Capabilities
PRBench-Legal - professional reasoning benchmark for legal domain. Evaluated using gemini-3.1-pro-preview.
PRBench-Finance
General Capabilities
PRBench-Finance - professional reasoning benchmark for finance domain. Evaluated using gemini-3.1-pro-preview.
MRCR v2 256K (8-needle)
Long Context
MRCR v2 256K (8-needle) - Multi-Ring Curve Retrieval benchmark v2, 256K context, 8-needle setting.
Agents' Last Exam
General Agents
Agents' Last Exam - frontier agentic tasks benchmark. Reports Pass@1 and Score (aggregate).
IFBench
Instruction Following
IFBench - instruction following benchmark. General score from Qwen3.8 model card.
JobBench
General Agents
JobBench - in-house benchmark for evaluating professional job task completion.
MathVision
Vision & Language
MathVision - visual math problem solving benchmark. Reports Without CI and With CI (Consistency Inference) scores.
OSWorld-Verified
General Agents
OSWorld-Verified - computer use benchmark evaluating autonomous GUI operation on verified tasks across desktop OS environments.
Toolathlon Verified
General Agents
Toolathlon Verified - agentic tool use benchmark. Pass@1 score.
$OneMillion-Bench
General Capabilities
$OneMillion-Bench - expert score. Evaluated using gemini-3.1-pro-preview.
PaperBench
Agentic Coding
PaperBench - evaluated in the BasicAgent setting under Code-Dev mode, judged by Claude Opus 4.6, averaged over 3 runs (max 12 hours per run).
WorkSpaceBench
General Agents
WorkSpaceBench - in-house benchmark for evaluating workspace productivity tasks.
Global PIQA
Multilingual
Global PIQA - physical reasoning benchmark across multiple languages and cultures.
MAXIFE
Multilingual
MAXIFE - multilingual instruction following benchmark. Reports accuracy on English + multilingual original prompts (23 settings total).
We-Math
Vision & Language
We-Math - visual mathematical reasoning benchmark evaluating step-by-step problem solving.
Hypersim
Spatial Intelligence
Hypersim - 3D spatial understanding benchmark using synthetic indoor scenes.
Nuscene
Spatial Intelligence
Nuscene - 3D spatial understanding benchmark using autonomous driving scenes.
Ifstruct V1
Instruction Following
Ifstruct V1 - instruction following structured evaluation benchmark by LiquidAI.
Beam128K
Long Context
Beam128K - long context retrieval and understanding benchmark at 128K tokens.
MBCT
Safety
MBCT - Meta Biological CTF benchmark evaluating biological knowledge and wet-lab debugging capabilities. Measures preparedness risk for chemical & biological...
HPCT
Safety
HPCT - High-Performance CTF benchmark evaluating cyber preparedness capabilities.
VCT
Safety
VCT - Vulnerability CTF benchmark evaluating cyber security preparedness capabilities.
WMDP (Bio)
Safety
WMDP (Bio) - Weapons of Mass Destruction Prohibition (Biology) benchmark evaluating biological knowledge that could enable threats. Measures preparedness risk.
WMDP (Chem)
Safety
WMDP (Chem) - Weapons of Mass Destruction Prohibition (Chemistry) benchmark evaluating chemical knowledge that could enable threats. Measures preparedness risk.
AIME 2025 (with tools)
STEM & Reasoning
AIME 2025 (with tools) - American Invitational Mathematics Examination 2025 with tool use variant.
MMMLU
Multilingual
MMMLU - Multilingual MMLU benchmark evaluating knowledge across multiple languages.
WildClawBench
Agentic Coding
WildClawBench - agentic coding evaluation benchmark. Overall score from internlm/WildClawBench.
MedXpertQA-MM
Vision & Language
MedXpertQA-MM - multimodal medical expert question answering benchmark evaluating clinical reasoning with visual inputs.
ZEROBench
Vision & Language
ZEROBench - visual reasoning puzzle benchmark. Very challenging, scores typically in single digits.
DeepSearch QA
General Agents
DeepSearch QA - full-task agentic benchmark measuring ability to work within scaffolds, write and debug code, and resolve multi-turn requests from start to f...
Gaia2
General Agents
Gaia2 - general AI assistant benchmark measuring multi-step agentic task completion.
LingoQA
Spatial Intelligence
LingoQA - spatial and linguistic reasoning benchmark for visual grounding.
MMVU
Video Understanding
MMVU - multi-modal video understanding benchmark evaluating comprehension across diverse video types.
ScreenSpot Pro
General Agents
ScreenSpot Pro - visual agent benchmark for GUI screen element grounding and interaction.
PMC-VQA
Vision & Language
PMC-VQA - medical visual question answering benchmark using PubMed Central biomedical images.
SLAKE
Vision & Language
SLAKE - medical visual question answering benchmark using radiology images.
MMLongBench-Doc
Document Understanding
MMLongBench-Doc - long document understanding benchmark evaluating comprehension and retrieval over extended documents.
Lab Bench (ProtocolQA)
Safety
Lab Bench (ProtocolQA) - wet-lab protocol question answering benchmark evaluating practical lab knowledge. Part of Meta's preparedness evaluation suite.