SWE-bench Verified
SWE-bench Verified evaluates LLM-based agents on real-world GitHub issue resolution from popular Python repositories. Internal agent scaffold (bash + file-edit tools); temp=1.0, top_p=0.95, 200K context window.
Category
Agentic Coding
Max Score
100.0
Score Type
percent
Active
Yes
Model Scores
Scores for SWE-bench Verified
| Model | Score | Date | Verified |
|---|---|---|---|
| Qwen3.6-35B-A3B |
83.72%
|
16.04.2026 | Verified |
| GPT 5.5 |
94.61%
|
11.06.2026 | Unverified |
| Gemma4-31B-it |
59.17%
|
16.04.2026 | Unverified |
| Qwen3.5-35B-A3B |
79.82%
|
16.04.2026 | Unverified |
| Gemma4-26B-A4B |
19.50%
|
16.04.2026 | Unverified |
| Qwen3.6-27B |
88.07%
|
22.04.2026 | Verified |
| Qwen3.5-397B-A17B |
86.93%
|
22.04.2026 | Unverified |
| Claude 4.5 Opus |
92.32%
|
22.04.2026 | Unverified |
| Qwen3.5-27B |
85.55%
|
16.04.2026 | Unverified |
| Ornith-1.0-9B |
79.13%
|
25.06.2026 | Verified |
| Qwen3.5-9B |
60.44%
|
25.06.2026 | Verified |
| Gemma4-12B |
50.23%
|
25.06.2026 | Unverified |
| Ornith-1.0-35B |
86.24%
|
25.06.2026 | Verified |
| NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 |
80.62%
|
04.06.2026 | Verified |
| MiniMax-M2.7 |
85.89%
|
04.06.2026 | Unverified |
| GLM-5.1 |
86.93%
|
04.06.2026 | Unverified |
| Kimi K2.6 |
86.35%
|
04.06.2026 | Unverified |
| DeepSeek-V4-Pro |
84.98%
|
04.06.2026 | Unverified |
| Muse-Glimmer-30B |
86.70%
|
01.08.2026 | Verified |
| MiniMax-M3 |
91.86%
|
11.06.2026 | Verified |
| DeepSeek V4 Pro |
91.97%
|
11.06.2026 | Unverified |
| Claude Sonnet 4.6 |
90.83%
|
11.06.2026 | Unverified |
| Claude Opus 4.7 |
100.00%
|
11.06.2026 | Unverified |
| Gemini 3.1 Pro |
91.97%
|
11.06.2026 | Unverified |
| Qwen3.5-122B-A10B |
82.11%
|
01.02.2026 | Verified |
| GPT-5-mini 2025-08-07 |
82.11%
|
01.02.2026 | Unverified |
| Qwen3-Coder-Next |
80.50%
|
03.02.2026 | Verified |
| DeepSeek-V3.2 |
80.05%
|
03.02.2026 | Verified |
| GLM-4.7 |
84.63%
|
03.02.2026 | Verified |
| MiniMax M2.1 |
85.32%
|
03.02.2026 | Verified |
| Ornith-1.5-35B-A3B |
90.14%
|
19.08.2026 | Verified |
| Ornith-1.0-35B-A3B |
86.24%
|
19.08.2026 | Unverified |
| Gemma-4-31B |
59.17%
|
19.08.2026 | Unverified |
| Claude Opus 4.6 |
92.20%
|
23.08.2026 | Verified |
| Kimi K2.5 |
87.61%
|
23.08.2026 | Verified |
| GPT-OSS-120B |
71.10%
|
08.08.2025 | Verified |
| GPT-OSS-20B |
69.15%
|
05.08.2025 | Verified |
| DeepSeek-V4-Flash |
90.14%
|
26.06.2026 | Verified |
| Opus-4.6 |
92.20%
|
26.06.2026 | Verified |
| DeepSeek-V4-Pro |
91.97%
|
26.06.2026 | Verified |
| Spark-X2.5-4B |
47.25%
|
— | Verified |
| Spark-X2.5-1.7B |
32.00%
|
— | Verified |
| Qwen3.5-4B |
44.04%
|
— | Verified |
| Qwen3.5-2B |
7.34%
|
— | Verified |
| Gemma4-E4B |
15.60%
|
— | Verified |
| MiniCPM5-2B |
52.75%
|
— | Verified |
| LFM2.5-2.6B |
6.42%
|
— | Verified |
| Gemma4-E2B |
1.83%
|
— | Verified |
| granite-4.2-3B |
41.74%
|
— | Verified |
| Nemotron-3-Nano-4B |
2.98%
|
— | Verified |
| LFM2.5-8B-A1B |
0.40
|
— | Verified |