OpenAI o3 vs Kimi K3.1 Pro
New ComparisonScores are fetched live from the current database on every visit — nothing is cached.
Benchmark Scores Comparison
OpenAI o3
Kimi K3.1 Pro
OpenAI o3
Score: 15.0
Kimi K3.1 Pro
2.8T params
Coverage matrix
2 models × 47 benchmarks — hover a benchmark name for its category
Shading = score relative to the best score among the selected models:
Low
Mid
High
Missing (—)
The totals row shows how many of the 47 benchmarks each model has scores for.
| Benchmark | Kimi K3.1 Pro Moonshot AI | OpenAI o3 OpenAI |
|---|---|---|
| PerceptionBench | 68.3 | — |
| GPQA Diamond | 94.2 | — |
| Design Bench | 957 | — |
| BioMysteryBench | 80.6 | — |
| BenchCAD w/o tools | 72.8 | — |
| BabyVision w/ python | 95.4 | — |
| Agents' Last Exam | 43.2 | — |
| Kimi Hallu Bench w/o tools | 5.4 | — |
| Kimi Hallu Bench Hard w/ tools | 8.3 | — |
| ZeroBench_main w/ python (pass@5) | 63 | — |
| OfficeQA Pro | 67.4 | — |
| GDP.pdf w/ python | 85.7 | — |
| BenchCAD w/ tools | 92.7 | — |
| ChartoGraphy w/ python | 90.2 | — |
| MMMU-Pro | 84.1 | — |
| Program Bench | 31.5 | — |
| PostTrainBench | 36.5 | — |
| MMMLU | — | 88.8 |
| OSWorld V2.1 | 79.6 | — |
| Apex-Agents | 69.4 | — |
| Terminal-Bench Science 0.1 | 38.1 | — |
| ExploitGym (6h) | 20.7 | — |
| Frontier Finance | 58.6 | — |
| xdev-bench-hard | 81 | — |
| HLE (with tools) | 70 | — |
| CharXiv (RQ) | 89 | — |
| Automation Bench-AA | 73.3 | — |
| GDP.pdf | 34.6 | — |
| BrowseComp | 92.1 | — |
| Harvey Legal Agent | 23.7 | — |
| OfficeQA Pro v2 | 64.4 | — |
| CharXiv (RQ) w/ python | 93.4 | — |
| Agents' Last Exam (CLI) | 44.2 | — |
| Humanity's Last Exam | 51.1 | — |
| HANDBOOK.md | 44.6 | — |
| HLE-Diamond (no tools) | 39.4 | — |
| MMMU-Pro w/ python | 90 | — |
| ExploitBench | 74.7 | — |
| Kimi Code Bench 3.0 Long-horizon | 61.2 | — |
| Kimi Code Bench V2 | 77.1 | — |
| DECK-Arena | 1456 | — |
| SWE-Marathon | 51.7 | — |
| Terminal-Bench 4.0 | 65.7 | — |
| DeepSWE 1.1 | 75.5 | — |
| Business Operation | 37.8 | — |
| AA-Briefcase v1.1 | 1678 | — |
| GDPval-AA v2.1 | 1846 | — |
| Covered | 46/47 | 1/47 |
Scorecard
Overall 0–100 score — category strengths (50%), shared-benchmark wins (30%), coverage (20%)
| Model | Overall score | Shared benchmark wins | Coverage | Strongest at | Weakest at |
|---|---|---|---|---|---|
| Kimi K3.1 Pro | 97.9 | — | 46 of 47 (98.0%) | — | — |
| OpenAI o3 | 2.1 | — | 1 of 47 (2.0%) | — | — |
Partial coverage benchmarks
Scored by at least one but not all selected models (47) — missing scores show as —
| Benchmark | Category | OpenAI o3 | Kimi K3.1 Pro |
|---|---|---|---|
| PerceptionBench 1/2 models | vision_language | — | 68.3 |
| GPQA Diamond 1/2 models | stem_reasoning | — | 94.2 |
| Design Bench 1/2 models | agentic | — | 957 |
| BioMysteryBench 1/2 models | stem_reasoning | — | 80.6 |
| BenchCAD w/o tools 1/2 models | coding_agent | — | 72.8 |
| BabyVision w/ python 1/2 models | vision_language | — | 95.4 |
| Agents' Last Exam 1/2 models | general_agent | — | 43.2 |
| Kimi Hallu Bench w/o tools 1/2 models | safety | — | 5.4 |
| Kimi Hallu Bench Hard w/ tools 1/2 models | safety | — | 8.3 |
| ZeroBench_main w/ python (pass@5) 1/2 models | vision_language | — | 63 |
| OfficeQA Pro 1/2 models | general_agent | — | 67.4 |
| GDP.pdf w/ python 1/2 models | document_understanding | — | 85.7 |
| BenchCAD w/ tools 1/2 models | coding_agent | — | 92.7 |
| ChartoGraphy w/ python 1/2 models | vision_language | — | 90.2 |
| MMMU-Pro 1/2 models | vision_language | — | 84.1 |
| Program Bench 1/2 models | coding_agent | — | 31.5 |
| PostTrainBench 1/2 models | coding_agent | — | 36.5 |
| MMMLU 1/2 models | multilingual | 88.8 | — |
| OSWorld V2.1 1/2 models | general_agent | — | 79.6 |
| Apex-Agents 1/2 models | general_agent | — | 69.4 |
| Terminal-Bench Science 0.1 1/2 models | coding_agent | — | 38.1 |
| ExploitGym (6h) 1/2 models | cybersecurity | — | 20.7 |
| Frontier Finance 1/2 models | domain_finance | — | 58.6 |
| xdev-bench-hard 1/2 models | safety | — | 81 |
| HLE (with tools) 1/2 models | stem_reasoning | — | 70 |
| CharXiv (RQ) 1/2 models | document_understanding | — | 89 |
| Automation Bench-AA 1/2 models | agentic | — | 73.3 |
| GDP.pdf 1/2 models | long_context | — | 34.6 |
| BrowseComp 1/2 models | general_agent | — | 92.1 |
| Harvey Legal Agent 1/2 models | general_agent | — | 23.7 |
| OfficeQA Pro v2 1/2 models | document_understanding | — | 64.4 |
| CharXiv (RQ) w/ python 1/2 models | vision_language | — | 93.4 |
| Agents' Last Exam (CLI) 1/2 models | general_agent | — | 44.2 |
| Humanity's Last Exam 1/2 models | stem_reasoning | — | 51.1 |
| HANDBOOK.md 1/2 models | general_agent | — | 44.6 |
| HLE-Diamond (no tools) 1/2 models | stem_reasoning | — | 39.4 |
| MMMU-Pro w/ python 1/2 models | vision_language | — | 90 |
| ExploitBench 1/2 models | cybersecurity | — | 74.7 |
| Kimi Code Bench 3.0 Long-horizon 1/2 models | coding_agent | — | 61.2 |
| Kimi Code Bench V2 1/2 models | coding_agent | — | 77.1 |
| DECK-Arena 1/2 models | agentic | — | 1456 |
| SWE-Marathon 1/2 models | coding_agent | — | 51.7 |
| Terminal-Bench 4.0 1/2 models | coding_agent | — | 65.7 |
| DeepSWE 1.1 1/2 models | coding_agent | — | 75.5 |
| Business Operation 1/2 models | agentic | — | 37.8 |
| AA-Briefcase v1.1 1/2 models | general_agent | — | 1678 |
| GDPval-AA v2.1 1/2 models | general_agent | — | 1846 |