PaperBench
PaperBench - evaluated in the BasicAgent setting under Code-Dev mode, judged by Claude Opus 4.6, averaged over 3 runs (max 12 hours per run).
Category
Agentic Coding
Max Score
—
Score Type
percent
Active
Yes
Model Scores
Scores for PaperBench
| Model | Score | Date | Verified |
|---|---|---|---|
| GPT 5.5 |
43.11%
|
11.06.2026 | Unverified |
| MiniMax-M2.7 |
30.60
|
11.06.2026 | Unverified |
| Qwen3.8-2.4T-A95B |
100.00%
|
18.08.2026 | Verified |
| Fable-5 |
93.27%
|
18.08.2026 | Unverified |
| GPT-5.6 Sol |
95.99%
|
18.08.2026 | Unverified |
| Claude Opus 4.8 |
79.65%
|
18.08.2026 | Unverified |
| Qwen3.7-Max |
54.81%
|
18.08.2026 | Unverified |
| Claude Opus 4.7 |
44.71%
|
11.06.2026 | Unverified |
| MiniMax-M3 |
35.26%
|
11.06.2026 | Unverified |
| Gemini 3.1 Pro |
25.80%
|
11.06.2026 | Unverified |