LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
TAU3-Bench
General Agents
TAU3-Bench - agentic tool use benchmark. Uses official user model (gpt-5.2, low reasoning effort) + default BM25 retrieval.
MCPMark
General Agents
MCPMark - GitHub MCP v0.30.3; Playwright responses truncated at 32K tokens.
VITA-Bench
General Agents
VITA-Bench - agentic benchmark. Avg subdomain scores; using claude-4-sonnet as judger.
AgentWorldBench Android
General Agents
AgentWorldBench Android domain - open-ended rubric evaluation for language world models simulating Android device environments. Normalized to 0-100.
AgentWorldBench MCP
General Agents
AgentWorldBench MCP domain - open-ended rubric evaluation (5 dimensions: Format, Factuality, Consistency, Realism, Quality) for language world models simulat...
AgentWorldBench OS
General Agents
AgentWorldBench OS domain - open-ended rubric evaluation for language world models simulating operating-system environments. Normalized to 0-100.
AgentWorldBench Search
General Agents
AgentWorldBench Search domain - open-ended rubric evaluation for language world models simulating search environments. Normalized to 0-100.
AgentWorldBench SWE
General Agents
AgentWorldBench SWE domain - open-ended rubric evaluation for language world models simulating software engineering environments. Normalized to 0-100.
AgentWorldBench Terminal
General Agents
AgentWorldBench Terminal domain - open-ended rubric evaluation for language world models simulating terminal environments. Normalized to 0-100.
AgentWorldBench Web
General Agents
AgentWorldBench Web domain - open-ended rubric evaluation for language world models simulating web browser environments. Normalized to 0-100.
AndroidWorld
General Agents
AndroidWorld - visual agent benchmark for Android device task completion.
TauBench V3 Airline
General Agents
TauBench V3 - Airline domain. Agentic tool use benchmark for airline customer service.
TauBench V3 Retail
General Agents
TauBench V3 - Retail domain. Agentic tool use benchmark for retail customer service.
TauBench V3 Telecom
General Agents
TauBench V3 - Telecom domain. Agentic tool use benchmark for telecom customer service.
TauBench V3 Banking
General Agents
TauBench V3 - Banking domain. Agentic tool use benchmark for banking customer service.
TauBench V3 Average
General Agents
TauBench V3 - Average across all domains (Airline, Retail, Telecom, Banking).
CoWorkBench
General Agents
CoWorkBench - in-house cowork benchmark for evaluating long-horizon tasks across computer science, finance, law, medical, and other productivity domains.
Vals.ai Financial Agent 1.1 (without web search)
General Agents
Vals.ai Financial Agent 1.1 - without web search mode. Financial agentic benchmark.
Vals.ai Financial Agent 1.1 (with web search)
General Agents
Vals.ai Financial Agent 1.1 - with web search mode. Financial agentic benchmark.
BrowseComp
General Agents
BrowseComp - web browsing and comprehension agentic benchmark.
WebArena-Verified
General Agents
WebArena-Verified - browser use benchmark evaluating autonomous web navigation and task completion. Scores computed with official WebArena-Verified grader un...
RecreationBench
General Agents
RecreationBench - in-house long-horizon application-recreation benchmark evaluating hybrid-agent capabilities across five platforms: desktop (Ubuntu, macOS, ...
SkillsBench
General Agents
SkillsBench - evaluated on the public SkillsBench v1.1 benchmark across 87 tasks, reporting the average score over three runs per task.
Automation-Bench
General Agents
Automation-Bench - evaluated on the 600-task public subset. Pass@1 score.
Agents' Last Exam
General Agents
Agents' Last Exam - frontier agentic tasks benchmark. Reports Pass@1 and Score (aggregate).
JobBench
General Agents
JobBench - in-house benchmark for evaluating professional job task completion.
OSWorld-Verified
General Agents
OSWorld-Verified - computer use benchmark evaluating autonomous GUI operation on verified tasks across desktop OS environments.
Toolathlon Verified
General Agents
Toolathlon Verified - agentic tool use benchmark. Pass@1 score.
WorkSpaceBench
General Agents
WorkSpaceBench - in-house benchmark for evaluating workspace productivity tasks.
DeepSearch QA
General Agents
DeepSearch QA - full-task agentic benchmark measuring ability to work within scaffolds, write and debug code, and resolve multi-turn requests from start to f...
Gaia2
General Agents
Gaia2 - general AI assistant benchmark measuring multi-step agentic task completion.
ScreenSpot Pro
General Agents
ScreenSpot Pro - visual agent benchmark for GUI screen element grounding and interaction.
BrowseComp-zh
General Agents
BrowseComp-zh - Chinese web browsing and comprehension agentic benchmark.
Cybergym
General Agents
Cybergym - agentic cybersecurity benchmark evaluating autonomous hacking and security task capabilities.
BrowseComp Agent Swarm
General Agents
BrowseComp evaluated with agent swarm mode - multiple coordinated sub-agents performing web browsing tasks.
Tau-Bench Airline
General Agents
τ-Bench Airline - function calling ability benchmark in an airline domain. Measures tool use with developer-defined function schemas.
Tau-Bench Retail
General Agents
τ-Bench Retail - function calling ability benchmark in a retail domain. Measures tool use with developer-defined function schemas.
OSWorld 2.0 (Partial)
General Agents
OSWorld 2.0 - Partial score. The partial score aggregates the partial rewards obtained across all tasks.