DeepSWE 1.1
DeepSWE 1.1 - agentic coding benchmark evaluated with Claude Code harness at temp=1.0, top_p=0.95, 256K context window.
Category
Agentic Coding
Max Score
100.0
Score Type
percent
Active
Yes
Model Scores
Scores for DeepSWE 1.1
| Model | Score | Date | Verified |
|---|---|---|---|
| Qwen3.8-27B |
51.66%
|
18.08.2026 | Verified |
| Qwen3.6-27B |
8.01%
|
18.08.2026 | Unverified |
| Qwen3.7-Plus |
9.37%
|
18.08.2026 | Unverified |
| Qwen3.8-2.4T-A95B |
73.41%
|
18.08.2026 | Verified |
| Fable-5 |
93.66%
|
18.08.2026 | Unverified |
| GPT-5.6 Sol |
98.19%
|
18.08.2026 | Unverified |
| Claude Opus 4.8 |
77.04%
|
18.08.2026 | Unverified |
| Qwen3.7-Max |
15.11%
|
17.06.2026 | Unverified |
| Kimi K3 |
89.88%
|
18.08.2026 | Verified |
| GLM-5.2 |
57.70%
|
17.06.2026 | Verified |
| DeepSeek-V4-Pro |
8.00
|
17.06.2026 | Unverified |
| Gemini 3.1 Pro |
3.02%
|
17.06.2026 | Unverified |
| GLM-5.1 |
15.11%
|
17.06.2026 | Unverified |
| GPT-5.5 |
93.66%
|
17.06.2026 | Unverified |
| MiniMax-M3 |
18.13%
|
17.06.2026 | Unverified |
| Qwen3.8-Flash-Next |
76.59%
|
26.08.2026 | Verified |
| DeepSeek-V4-Flash-0731 |
70.09%
|
26.08.2026 | Unverified |
| GLM-5.3-Flash |
83.69%
|
27.08.2026 | Verified |
| DeepSeek-V4-Vision-Exp |
77.49%
|
27.08.2026 | Verified |
| GPT-5.3 Terra |
93.05%
|
27.08.2026 | Verified |
| Gemini 3.7 Flash |
86.56%
|
27.08.2026 | Verified |
| GLM-5.3 |
88.97%
|
28.08.2026 | Verified |
| DeepSeek-V4-Pro-0813 |
82.63%
|
28.08.2026 | Verified |
| Opus-4.8 |
75.53%
|
28.08.2026 | Verified |
| DeepSeek-V4.1-Flash |
100.00%
|
10.09.2026 | Verified |