LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
Kimi Claw 24/7 Bench
Agentic
In-house benchmark by Moonshot AI for evaluating long-horizon agentic performance in persistent, multi-day coworking tasks. It spans 17 professional scenario...
MCPMark-Verified
Agentic
A human-verified edition of MCPMark, a benchmark for evaluating MCP tool use across five real server environments — Notion, GitHub, Filesystem, Postgres, and...
ToolRet
Agentic
ToolRet evaluates tool retrieval for agents: given a user query, rank the correct tool/API from a candidate pool. Scores here are nDCG@10 printed on a 0-100 ...
API-Bank
Agentic
API-Bank measures benchmark-style tool/API invocation accuracy. Scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
BRIGHT
Agentic
BRIGHT evaluates reality-grounded retrieval for reasoning-heavy queries, scored as nDCG@10 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
When2Call MCQ
Agentic
When2Call MCQ: multiple-choice evaluation of when to call a tool vs answer directly, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision In...
BFCL v4 (non-live AST)
Agentic
BFCL v4 (non-live AST) - function-calling AST check on non-live prompts; split of BFCL v4 as reported on the Kolibri card.
BFCL v4 (live)
Agentic
BFCL v4 (live) - split of BFCL v4 covering live/real user queries, as reported on the Kolibri card.
BFCL v4 (multi-turn)
Agentic
BFCL v4 (multi-turn) - multi-turn state/global evaluation split of BFCL v4 as reported on the Kolibri card.
BFCL v4 (memory)
Agentic
BFCL v4 (memory) - memory-category split of BFCL v4 as reported on the Kolibri card.
BFCL v4 (web search)
Agentic
BFCL v4 (web search) - web-search-category split of BFCL v4 as reported on the Kolibri card.
FRAMES (<24k)
Agentic
FRAMES (prompts under 24k tokens) - factuality/retrieval/reasoning benchmark as run in the AA eval-framework; overlength prompts score 0 on long-context benc...
FRAMES (>24k)
Agentic
FRAMES (prompts over 24k tokens) - factuality/retrieval/reasoning benchmark as run in the AA eval-framework on the Kolibri card.
SealQA (no distractors, <24k)
Agentic
SealQA (no distractors, prompts under 24k tokens) - search-augmented fact-checking variant as run in the AA eval-framework on the Kolibri card.
SealQA (12 distractors, <24k)
Agentic
SealQA (12 distractor documents, prompts under 24k tokens) - search-augmented fact-checking variant as run in the AA eval-framework on the Kolibri card.
SealQA (no distractors, >24k)
Agentic
SealQA (no distractors, prompts over 24k tokens) - search-augmented fact-checking variant as run in the AA eval-framework on the Kolibri card.
SealQA (12 distractors, >24k)
Agentic
SealQA (12 distractor documents, prompts over 24k tokens) - search-augmented fact-checking variant as run in the AA eval-framework on the Kolibri card.
MuSiQue (EN)
Agentic
MuSiQue (EN) - agentic multi-hop retrieval QA (English) as run in the AA eval-framework on the Kolibri card; constitutes the Agentic Retrieval Average (EN) row.
Honeypot
Agentic
Honeypot - trap-resistance benchmark from the agentic retrieval suite as run in the AA eval-framework on the Kolibri card; higher is better.
Agentic Wiki QA (DE)
Agentic
Agentic Wiki QA (DE) - agentic QA over German Wikipedia as run in the AA eval-framework on the Kolibri card; constitutes the Agentic Retrieval Average (DE) row.
Semiconductors
Agentic
Semiconductors - domain of Aleph Alpha's Industry RAG suite (industry RAG over semiconductor documents, English) as run in the AA eval-framework on the Kolib...
German Public Sector
Agentic
German Public Sector - domain of Aleph Alpha's Industry RAG suite (industry RAG over German public-sector documents) as run in the AA eval-framework on the K...
Aerospace
Agentic
Aerospace - domain of Aleph Alpha's Industry RAG suite (industry RAG over aerospace documents) as run in the AA eval-framework on the Kolibri card.
Automotive Supplier
Agentic
Automotive Supplier - domain of Aleph Alpha's Industry RAG suite (industry RAG over automotive-supplier documents) as run in the AA eval-framework on the Kol...
Industrial Drive Technology
Agentic
Industrial Drive Technology - domain of Aleph Alpha's Industry RAG suite (industry RAG over industrial drive technology documents) as run in the AA eval-fram...
Automation Bench-AA
Agentic
Artificial Analysis-branded AutomationBench-AA eval; scores cited from artificialanalysis.ai in the Kimi K3.1 Pro card. Distinct from Automation-Bench and Au...
Business Operation
Agentic
Internal Moonshot agentic benchmark for business operation tasks.
DECK-Arena
Agentic
Internal Moonshot Elo-score benchmark for presentation generation (python-pptx, pptxjs, in-house DSL; grader calibrated on 10,000+ human judgments).
Design Bench
Agentic
Internal Moonshot Elo-score benchmark for graphic design via direct and reference-image few-shot prompting.