Automation-Bench
Automation-Bench - evaluated on the 600-task public subset. Pass@1 score.
Category
General Agents
Max Score
—
Score Type
percent
Active
Yes
Model Scores
Scores for Automation-Bench
| Model | Score | Date | Verified |
|---|---|---|---|
| Fable-5 |
41.59%
|
18.08.2026 | Unverified |
| GPT-5.6 Sol |
42.95%
|
18.08.2026 | Unverified |
| Claude Opus 4.8 |
37.27%
|
18.08.2026 | Unverified |
| Qwen3.7-Max |
7.73%
|
18.08.2026 | Unverified |
| Qwen3.8-2.4T-A95B |
37.50%
|
18.08.2026 | Verified |
| Kimi K3 |
45.45%
|
18.08.2026 | Verified |
| DeepSeek-V4-Flash-0731 |
32.50%
|
31.07.2026 | Verified |
| GLM-5.2 |
4.77%
|
31.07.2026 | Verified |
| DeepSeek-V4-Flash |
10.80
|
13.08.2026 | Verified |
| DeepSeek-V4-Pro |
4.55%
|
13.08.2026 | Verified |
| Opus-4.8 |
37.27%
|
31.07.2026 | Verified |
| DeepSeek-V4-Pro-0813 |
47.73%
|
13.08.2026 | Verified |
| GLM-5.3-Flash |
86.36%
|
27.08.2026 | Verified |
| DeepSeek-V4-Vision-Exp |
63.64%
|
27.08.2026 | Verified |
| GPT-5.3 Terra |
60.00%
|
27.08.2026 | Verified |
| Gemini 3.7 Flash |
94.32%
|
27.08.2026 | Verified |
| GLM-5.3 |
85.00%
|
28.08.2026 | Verified |
| Hy3 |
12.05%
|
28.08.2026 | Verified |
| Hy4 preview |
48.41%
|
28.08.2026 | Verified |
| DeepSeek V4 Pro |
44.55%
|
28.08.2026 | Verified |
| Claude Opus 5 |
86.14%
|
28.08.2026 | Verified |
| DeepSeek-V4-Flash-Vision-Exp |
33.86%
|
08.09.2026 | Verified |
| DeepSeek-V4.1-Flash |
100.00%
|
10.09.2026 | Verified |