LLM Benchmarks — Scores, Methodology & Live Rankings
What each benchmark measures, how models score, and the current rankings
Hy-FinAgentBench (Internal)
Finance
Tencent internal (Hy team) financial-agent benchmark, reported in the Hy4 preview model card appendix.
Hy-FinmodelBench v2 (Internal)
Finance
Tencent internal (Hy team) financial-modelling benchmark, reported in the Hy4 preview model card appendix.
OSWorld 2.0 (Binary)
General Agents
OSWorld 2.0 - Binary score. The binary score is the percentage of tasks that receive the full task reward.
SUPERChem
STEM & Reasoning
Chemistry reasoning benchmark reported in the Tencent Hy4 preview model card appendix.
ArXivMath
STEM & Reasoning
Math reasoning benchmark over arXiv-style problems, reported in the Tencent Hy4 preview model card appendix.
BrokenArXiv
STEM & Reasoning
Math reasoning benchmark reported in the Tencent Hy4 preview model card appendix.
Workspace Bench
Gaokao 2026
LCB-Pro 25Q2 (Easy)
STEM & Reasoning
LiveCodeBench Pro 25Q2 (Easy) - competitive programming evaluation, easy difficulty subset of LCB-Pro 2025 Q2.
LCB-Pro 25Q2 (Medium)
STEM & Reasoning
LiveCodeBench Pro 25Q2 (Medium) - competitive programming evaluation, medium difficulty subset of LCB-Pro 2025 Q2.
Terminal-Bench 4.0
Agentic Coding
Terminal-Bench 4.0 agentic terminal tasks. DeepSeek-V4.1-Flash evaluated with the Minimal mode of DeepSeek Harness and a 1M-token context window; Pass@1.
SEC-Bench Pro
General Agents
Security exploitation agent benchmark; DeepSeek-V4.1-Flash evaluated with the Claude Code harness per the official setup; Pass@1.
LiveBench 241125
Reasoning
LiveBench (2024-11-25 version) - contamination-free LLM benchmark with regularly updated tasks across reasoning, coding, data analysis and mathematics.
CFEval
Agentic Coding
CFEval - Qwen's CodeForces-style competitive programming evaluation; score is an Elo-like rating (higher is better).
Arena-Hard v2
Instruction Following
Arena-Hard v2 - instruction-following chat benchmark; win rate evaluated by GPT-4.1 for reproducibility.
WritingBench
Instruction Following
WritingBench - benchmark evaluating fine-grained writing capabilities with verifiable constraints.
TauBench V2 Retail
General Agents
tau2-bench - Retail domain. Agentic tool use benchmark for retail customer service scenarios.
TauBench V2 Airline
General Agents
tau2-bench - Airline domain. Agentic tool use benchmark for airline customer service scenarios.
TauBench V2 Telecom
General Agents
tau2-bench - Telecom domain. Agentic tool use benchmark for telecom customer service scenarios.
Creative Writing v3
Instruction Following
Creative Writing v3 - Qwen's creative writing quality evaluation benchmark.
DeepresearchBII
General Agents
DeepresearchBII - deep-research agentic benchmark used in Xing4.0-29B-A4B and competitor model cards. Evaluation setup as printed: OpenCode harness with an E...
Multi-IF
Instruction Following
Multi-IF - multi-turn instruction following benchmark evaluating complex multi-constraint instruction adherence.
LongBenchPro
Long Context
LongBenchPro - long context understanding benchmark with professional-level tasks.
MATH-500
Math
MATH-500 - 500 competition-level mathematics problems, pass@1 without tools.
NoLiMa
Long Context
NoLiMa - long context benchmark evaluating retrieval and reasoning without explicit long-context matching.
MathArena Apex 2025
STEM & Reasoning
MathArena Apex 2025 - hardest math contest problems of the 2025 MathArena set.
HorizonMath (pass@4)
STEM & Reasoning
HorizonMath - long-horizon math reasoning, pass@4 protocol, reported in the Tencent Hy4 preview model card appendix.
ToolRet
Agentic
ToolRet evaluates tool retrieval for agents: given a user query, rank the correct tool/API from a candidate pool. Scores here are nDCG@10 printed on a 0-100 ...
API-Bank
Agentic
API-Bank measures benchmark-style tool/API invocation accuracy. Scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
BANKING77
Finance
BANKING77 intent classification over the Banking77 banking-query dataset, scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 s...
CLINC150+OOS
Classification
CLINC150 with out-of-scope (OOS) intent classification, scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
RouterBench
General Agents
RouterBench measures LLM-router quality (selected route quality when deciding between models). Scored on a 0-100 scale in Cloudflare's internal Decision Inde...
Home appliance simulator
General Agents
Home appliance simulator: control of a simulated home appliance environment, scored as case exact accuracy on a 0-100 scale in Cloudflare's internal Decision...
SGD/SGD-X
Instruction Following
Schema-Guided Dialogue (SGD) evaluated with SGD-X robustness prompts, scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
ContractNLI
Document Understanding
ContractNLI evaluates natural-language inference over non-disclosure agreements, scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index ...
ANLI
Reasoning
Adversarial Natural Language Inference (ANLI, rounds R1-R3), scored as macro-F1 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
BPoMP
Multilingual
BPoMP (Best Prompt of Multiple Problems): multi-prompt prompt-selection eval, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2...
Humicroedit
Reasoning
Humicroedit: humor-edit headline classification, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
POP909-CL
General Capabilities
POP909-CL: classification variant over the POP909 piano-performance dataset, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2....
cfcolor
Vision & Language
cfcolor: Cloudflare-internal ("cf") color/asset recognition eval, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
ChessBench
Reasoning
ChessBench: board-state decision benchmark over chess positions, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
MuSR
Reasoning
MuSR (Multistep Soft Reasoning): multistep soft reasoning over narrative problems, scored as accuracy on a 0-100 scale in Cloudflare's internal Decision Inde...
SATA-Bench
General Agents
SATA-Bench: structured-output attribute/typed answer evaluation (case exact accuracy), scored on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 ...
BRIGHT
Agentic
BRIGHT evaluates reality-grounded retrieval for reasoning-heavy queries, scored as nDCG@10 on a 0-100 scale in Cloudflare's internal Decision Index 0.2.1 suite.
Amazon ESCI
Finance
Amazon ESCI: e-commerce search relevance (Exact/ Substitute/ Complement/ Irrelevant) classification over the Amazon ESCI shopping queries dataset, scored as ...