Qwen3.5-397B-A17B
Qwen Team (Alibaba Cloud)
Parameters
—
Architecture
—
Released
—
License
—
About
No detailed description available for this model.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
SWE-bench Verified
coding_agent
|
86.93%
|
22.04.2026 |
#19/60
better than 68%
|
|
SWE-bench Multilingual
coding_agent
|
76.09%
|
22.04.2026 |
#26/48
better than 40%
|
|
Terminal-Bench 2.0
coding_agent
|
56.28%
|
22.04.2026 |
#11/23
better than 55%
|
|
SkillsBench Avg5
coding_agent
|
41.09%
|
22.04.2026 |
#11/16
better than 33%
|
|
QwenWebBench
coding_agent
|
37.28%
|
22.04.2026 |
#5/8
better than 43%
|
|
NL2Repo
coding_agent
|
33.69%
|
22.04.2026 |
#32/41
better than 22%
|
|
Claw-Eval Avg
coding_agent
|
85.32%
|
22.04.2026 |
#9/40
better than 79%
|
|
QwenClawBench
coding_agent
|
96.73%
|
22.04.2026 |
#5/17
better than 75%
|
|
MMLU-Redux
knowledge
|
97.06%
|
22.04.2026 |
#2/30
better than 97%
|
|
SuperGPQA
knowledge
|
99.55%
|
22.04.2026 |
#2/28
better than 96%
|
|
GPQA Diamond
stem_reasoning
|
88.84%
|
22.04.2026 |
#34/80
better than 58%
|
|
Humanity's Last Exam
stem_reasoning
|
50.38%
|
22.04.2026 |
#35/71
better than 51%
|
|
LiveCodeBench v6
stem_reasoning
|
86.49%
|
22.04.2026 |
#21/49
better than 58%
|
|
HMMT Feb 25
stem_reasoning
|
100.00%
|
22.04.2026 |
#1/17
better than 100%
|
|
HMMT Nov 25
stem_reasoning
|
80.71%
|
22.04.2026 |
#9/20
better than 58%
|
|
HMMT Feb 26
stem_reasoning
|
87.31%
|
22.04.2026 |
#16/44
better than 65%
|
|
IMOAnswerBench
stem_reasoning
|
82.81%
|
22.04.2026 |
#14/30
better than 55%
|
|
AIME 26
stem_reasoning
|
92.47%
|
22.04.2026 |
#24/45
better than 45%
|
|
MMMU
vision_language
|
100.00%
|
22.04.2026 |
#1/19
better than 100%
|
|
DynaMath
vision_language
|
84.27%
|
22.04.2026 |
#2/14
better than 92%
|
|
RealWorldQA
vision_language
|
89.30%
|
22.04.2026 |
#8/23
better than 68%
|
|
MMStar
vision_language
|
100.00%
|
22.04.2026 |
#1/16
better than 100%
|
|
CC-OCR
document_understanding
|
100.00%
|
22.04.2026 |
#1/14
better than 100%
|
|
CountBench
spatial_intelligence
|
92.31%
|
22.04.2026 |
#4/13
better than 75%
|
|
MVBench
video_understanding
|
100.00%
|
22.04.2026 |
#1/12
better than 100%
|
|
V-Star
vision_language
|
93.47%
|
22.04.2026 |
#4/20
better than 84%
|
|
AgentWorldBench MCP
general_agent
|
88.20%
|
24.06.2026 |
#4/14
better than 77%
|
|
AgentWorldBench Search
general_agent
|
55.86%
|
24.06.2026 |
#5/14
better than 69%
|
|
AgentWorldBench Terminal
general_agent
|
77.90%
|
24.06.2026 |
#5/14
better than 69%
|
|
AgentWorldBench SWE
general_agent
|
86.96%
|
24.06.2026 |
#6/14
better than 62%
|
|
AgentWorldBench Web
general_agent
|
19.18%
|
24.06.2026 |
#13/14
better than 8%
|
|
AgentWorldBench OS
general_agent
|
32.88%
|
24.06.2026 |
#9/14
better than 38%
|
|
AgentWorldBench Overall
general_agent
|
68.47%
|
24.06.2026 |
#7/14
better than 54%
|
|
Terminal-Bench 2.1 (Terminus-2)
coding_agent
|
47.94%
|
25.06.2026 |
#34/44
better than 23%
|
|
Terminal-Bench 2.1 (Claude Code)
coding_agent
|
59.88%
|
25.06.2026 |
#5/8
better than 43%
|
|
SWE Atlas - QnA
coding_agent
|
20.44%
|
25.06.2026 |
#18/24
better than 26%
|
|
SWE Atlas - RF
coding_agent
|
25.31%
|
25.06.2026 |
#10/14
better than 31%
|
|
SWE Atlas - TW
coding_agent
|
21.93%
|
25.06.2026 |
#16/20
better than 21%
|
|
Vals.ai Financial Agent 1.1 (without web search)
general_agent
|
100.00%
|
04.06.2026 |
#1/7
better than 100%
|
|
Vals.ai Financial Agent 1.1 (with web search)
general_agent
|
72.03%
|
04.06.2026 |
#4/7
better than 50%
|
|
CritPt (no tools)
stem_reasoning
|
5.68%
|
04.06.2026 |
#20/21
better than 5%
|
|
GDPVal
general_agent
|
34.60
|
04.06.2026 |
#20/20
better than 0%
|
|
IOI 2025
stem_reasoning
|
441.30
|
04.06.2026 |
#5/5
better than 0%
|
|
ProfBench (Search)
general_agent
|
50.36%
|
04.06.2026 |
#5/7
better than 33%
|
|
IMOAnswerBench (with tools)
stem_reasoning
|
50.56%
|
04.06.2026 |
#6/7
better than 17%
|
|
Apex-Shortlist (no tools)
stem_reasoning
|
53.02%
|
04.06.2026 |
#10/11
better than 10%
|
|
IFBench (prompt loose)
instruction_following
|
80.51%
|
04.06.2026 |
#6/14
better than 62%
|
|
PinchBench
general_agent
|
65.69%
|
04.06.2026 |
#6/9
better than 38%
|
|
TauBench V3 Airline
general_agent
|
11.43%
|
04.06.2026 |
#6/7
better than 17%
|
|
Apex-Shortlist (with tools)
stem_reasoning
|
24.57%
|
04.06.2026 |
#6/7
better than 17%
|
|
Multi-Challenge
instruction_following
|
99.07%
|
04.06.2026 |
#2/18
better than 94%
|
|
TauBench V3 Retail
general_agent
|
90.32%
|
04.06.2026 |
#3/7
better than 67%
|
|
AA-LCR
long_context
|
81.31%
|
04.06.2026 |
#13/42
better than 66%
|
|
RULER (1M)
long_context
|
34.29%
|
04.06.2026 |
#3/4
better than 33%
|
|
TauBench V3 Telecom
general_agent
|
96.55%
|
04.06.2026 |
#2/7
better than 83%
|
|
SciCode (subtask)
stem_reasoning
|
47.09%
|
04.06.2026 |
#9/15
better than 43%
|
|
Longbench v2 (≤ 1M)
long_context
|
100.00%
|
04.06.2026 |
#1/4
better than 100%
|
|
MMLU-ProX
multilingual
|
100.00%
|
04.06.2026 |
#1/19
better than 100%
|
|
TauBench V3 Average
general_agent
|
64.47%
|
04.06.2026 |
#4/7
better than 50%
|
|
WMT24++ (en→xx)
multilingual
|
100.00%
|
04.06.2026 |
#1/17
better than 100%
|
|
MMMU-Pro
vision_language
|
91.16%
|
22.04.2026 |
#8/35
better than 79%
|
|
HLE (with tools)
stem_reasoning
|
61.32%
|
04.06.2026 |
#31/49
better than 38%
|
|
DeepSWE
coding_agent
|
1.38%
|
19.08.2026 |
#19/21
better than 10%
|
|
Frontier-Bench v0.1
coding_agent
|
1.40
|
19.08.2026 |
#2/4
better than 0%
|
|
Toolathlon Verified
general_agent
|
20.96%
|
19.08.2026 |
#34/35
better than 3%
|
|
WideSearch
general_agent
|
75.93%
|
19.08.2026 |
#12/24
better than 52%
|
|
WildClawBench
coding_agent
|
97.35%
|
19.08.2026 |
#2/20
better than 95%
|
|
BrowseComp
general_agent
|
42.02%
|
04.06.2026 |
#32/45
better than 30%
|
|
C-Eval
knowledge
|
99.08%
|
22.04.2026 |
#2/15
better than 93%
|
|
CharXiv (RQ)
document_understanding
|
58.50%
|
22.04.2026 |
#10/34
better than 73%
|
|
AgentWorldBench Android
general_agent
|
26.77%
|
24.06.2026 |
#12/14
better than 15%
|
|
MMLU-Pro
knowledge
|
89.68%
|
22.04.2026 |
#6/48
better than 89%
|
|
TauBench V3 Banking
general_agent
|
42.78%
|
04.06.2026 |
#11/17
better than 38%
|
|
MCP-Atlas
general_agent
|
77.94%
|
19.08.2026 |
#24/54
better than 57%
|
|
OmniScience Accuracy
knowledge
|
72.56%
|
04.06.2026 |
#3/10
better than 78%
|
|
RefCOCO (avg)
spatial_intelligence
|
98.94%
|
22.04.2026 |
#2/11
better than 90%
|
|
VideoMME (w sub.)
video_understanding
|
88.10%
|
22.04.2026 |
#5/17
better than 75%
|
|
SimpleVQA
vision_language
|
82.57%
|
22.04.2026 |
#2/16
better than 93%
|
|
OmniScience Non-Hallucination
knowledge
|
5.10%
|
04.06.2026 |
#8/10
better than 22%
|
|
MLVU
video_understanding
|
96.25%
|
22.04.2026 |
#2/13
better than 92%
|
|
ERQA
spatial_intelligence
|
85.80%
|
22.04.2026 |
#3/21
better than 90%
|
|
SWE-bench Pro
coding_agent
|
56.74%
|
22.04.2026 |
#47/77
better than 39%
|
|
Claw-Eval Pass^3
coding_agent
|
43.34%
|
22.04.2026 |
#18/21
better than 15%
|
|
VideoMMMU
video_understanding
|
91.64%
|
22.04.2026 |
#3/19
better than 89%
|
What is a good SWE-bench Verified score? What is a good SWE-bench Multilingual score? What is a good Terminal-Bench 2.0 score?
No comments yet
Be the first to share your thoughts!