LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
DECK-Arena
Agentic
Internal Moonshot Elo-score benchmark for presentation generation (python-pptx, pptxjs, in-house DSL; grader calibrated on 10,000+ human judgments).
Design Bench
Agentic
Internal Moonshot Elo-score benchmark for graphic design via direct and reference-image few-shot prompting.
Kimi Hallu Bench Hard w/ tools
Safety
Internal Moonshot hallucination benchmark, hard subset with tools. LOWER IS BETTER - claim-level hallucination rate assessed by fact-checking agents.
Kimi Hallu Bench w/o tools
Safety
Internal Moonshot hallucination benchmark without tools. LOWER IS BETTER - claim-level hallucination rate.
xdev-bench-hard
Safety
Internal expert-curated source-code vulnerability discovery and exploitation benchmark; pass@3 over a random subset of runs; Kimi Code harness.
SWE-bench Pro v2.0 Hard
Agentic Coding
SWE-bench Pro v2.0 Hard split as reported on the Ripple-65B Preview card; Terminus 2 harness used for Ripple-65B Preview, competitor scores partially officia...
FORTRESS Benign
Safety
FORTRESS benign-prompt suite row from the Ripple-65B Preview card Safety section (adversarial + benign pair of the FORTRESS robustness benchmark).
CMMLU
Knowledge
CMMLU - Chinese multi-task language understanding benchmark; reported in the baidu/ERNIE-5.5-Flash-Base model card evaluation table as CMMLU (EM), 5-shot.
Aider Polyglot
Agentic Coding
Aider Polyglot - coding benchmark evaluating model's ability to edit code across multiple programming languages.
BFCL-V3
General Agents
BFCL-V3 - Berkeley Function Calling Leaderboard version 3, evaluating function calling and tool use.
BFCL-V4
General Agents
BFCL-V4 - Berkeley Function Calling Leaderboard version 4, evaluating function calling and tool use capabilities.
BIG-Bench Hard
Reasoning
BIG-Bench Hard (BBH) - a subset of 23 challenging BIG-Bench tasks. few-shot.
Global-MMLU-Lite
Multilingual
Global-MMLU-Lite - multilingual MMLU subset by Cohere For AI.
GPQA Diamond
STEM & Reasoning
GPQA Diamond - Google-Proof Q&A benchmark for graduate-level scientific reasoning.
HealthBench
General Capabilities
HealthBench - healthcare-related benchmark evaluating medical knowledge and reasoning.
SWE-bench Pro
Agentic Coding
SWE-bench Pro - refined benchmark with corrected problematic tasks in the public set. Internal agent scaffold; temp=1.0, top_p=0.95, 200K context window.
BabyVision
Vision & Language
BabyVision - general visual reasoning benchmark. Reports Without CI and With CI (Consistency Inference) scores.
GDPVal
General Agents
GDPVal - agentic benchmark for evaluating general task completion.
GDPVal-AA v2
General Agents
GDPVal-AA v2 - Artificial Analysis GDPVal v2 agentic benchmark for evaluating general task completion. Score is an index value (not percentage).
IFEval
Instruction Following
IFEval - Instruction Following Evaluation benchmark for measuring LLM instruction-following capabilities.
MMMU
Vision & Language
MMMU - massive multi-discipline multimodal understanding and reasoning benchmark for college-level knowledge.
SWE Atlas - QnA
Agentic Coding
SWE Atlas QnA - agentic coding subtask; mini SWE agent harness, temp=1.0, top_p=0.95, 128K context, avg of 5 runs.
SWE-bench Verified
Agentic Coding
SWE-bench Verified evaluates LLM-based agents on real-world GitHub issue resolution from popular Python repositories. Internal agent scaffold (bash + file-ed...
TAU2-Bench
General Agents
TAU2-Bench - agentic tool use benchmark for customer service scenarios. Airline domain evaluated with fixes from Claude Opus 4.5 system card.
Terminal Bench 2.1
Agentic Coding
Terminal Bench 2.1 - agentic coding and terminal use benchmark. Evaluated with Claude Code (avg@10), 5-hour timeout, max_tokens=131,072.
Terminal-Bench 2.1 (Terminus-2)
Agentic Coding
Terminal-Bench 2.1 evaluated with Harbor/Terminus-2 framework, parser=json, temp=1.0, top_p=1.0, 128K context, 4h timeout, 32 CPU/48GB RAM, avg of 5 runs.
AgentWorldBench Overall
General Agents
AgentWorldBench Overall - mean across all 7 agent interaction domains (MCP, Search, Terminal, SWE, Android, Web, OS). Five-dimensional rubric, normalized to ...
GDP.pdf
Long Context
GDP.pdf: official Artificial Analysis (artificialanalysis.ai) text-only version as cited on the KRAFTON Raon-1 mini preview card; PDFs are OCR-ed in advance,...
IFBench (prompt loose)
Instruction Following
IFBench (prompt loose) - instruction following benchmark, loose prompt evaluation.
MMStar
Vision & Language
MMStar - comprehensive multimodal benchmark evaluating fine-grained visual reasoning.
SWE-Marathon
Agentic Coding
SWE-Marathon - long-horizon software engineering benchmark. Evaluated by Abundant AI with 1M context length, max effort level, and 128K maximum output tokens.
AA-Omniscience Index (public set)
Knowledge
AA-Omniscience Index (public set) - calibrated knowledge index from Aleph Alpha's Omniscience suite, centered at 0 and comparable across models; can be negat...
BioMysteryBench
STEM & Reasoning
Biology research-agent benchmark (99-task initial release); per model card footnote evaluated with Claude Code harness, judged by Kimi-K3.
INCLUDE
Multilingual
INCLUDE - multilingual benchmark for evaluating language coverage across diverse regions.
OmniScience Non-Hallucination
Knowledge
OmniScience Non-Hallucination - scientific knowledge non-hallucination rate benchmark.
Terminal-Bench 2.1 (Claude Code)
Agentic Coding
Terminal-Bench 2.1 evaluated using Claude Code 2.1.126, parser=json, temp=1.0, top_p=1.0, max_new_tokens=131072, avg of 5 runs.
TIR-Bench
Vision & Language
TIR-Bench - tool-integrated reasoning benchmark for visual mathematical problem solving. Reports with CI / without CI scores.
Terminal-Bench 3.0
Agentic Coding
Terminal-Bench 3.0 agentic terminal tasks, evaluated with the Claude Code 2.1.207 harness (reasoning effort=max, 400K context, 128K max output), avg@3 over t...