SWE Atlas - QnA
SWE Atlas QnA - agentic coding subtask; mini SWE agent harness, temp=1.0, top_p=0.95, 128K context, avg of 5 runs.
Category
Agentic Coding
Max Score
100.0
Score Type
percent
Active
Yes
Model Scores
Scores for SWE Atlas - QnA
| Model | Score | Date | Verified |
|---|---|---|---|
| GPT 5.5 |
66.06%
|
11.06.2026 | Unverified |
| MiniMax-M2.7 |
3.83%
|
11.06.2026 | Unverified |
| Ornith-1.0-9B |
15.88%
|
25.06.2026 | Verified |
| Qwen3.5-9B |
9.20
|
25.06.2026 | Unverified |
| Qwen3.5-35B-A3B |
7.30%
|
25.06.2026 | Unverified |
| Ornith-1.0-35B |
50.91%
|
25.06.2026 | Verified |
| Qwen3.6-35B-A3B |
11.50%
|
25.06.2026 | Unverified |
| Qwen3.5-397B-A17B |
20.44%
|
25.06.2026 | Unverified |
| Claude Sonnet 4.6 |
40.15%
|
11.06.2026 | Unverified |
| Claude Opus 4.7 |
65.69%
|
11.06.2026 | Unverified |
| MiniMax-M3 |
52.37%
|
11.06.2026 | Unverified |
| Gemini 3.1 Pro |
7.85%
|
11.06.2026 | Unverified |
| Ornith-1.5-35B-A3B |
55.84%
|
19.08.2026 | Verified |
| Ornith-1.0-35B-A3B |
50.91%
|
19.08.2026 | Unverified |
| Hy3 |
39.42%
|
28.08.2026 | Verified |
| Hy4 preview |
100.00%
|
28.08.2026 | Verified |
| DeepSeek V4 Pro |
80.66%
|
28.08.2026 | Verified |
| Qwen3.8-2.4T-A95B |
84.31%
|
28.08.2026 | Verified |
| GLM-5.3 |
85.04%
|
28.08.2026 | Verified |
| Kimi K3 |
47.45%
|
28.08.2026 | Verified |
| GPT-5.6 Sol |
89.23%
|
28.08.2026 | Verified |
| Claude Opus 5 |
88.14%
|
28.08.2026 | Verified |