ProgramBench
ProgramBench - agentic coding benchmark evaluated with Claude-Code 2.1.156, 200 instances, temp=1.0, top_p=1.0, max_tokens=64000, max_turns=2000, sample_timeout=6h, reasoning_effort=max, 400K context window. Each instance runs in a (4 CPUs, 8 GB RAM) sandbox with internet access disabled.
Category
Agentic Coding
Max Score
100.0
Score Type
percent
Active
Yes
Model Scores
Scores for ProgramBench
| Model | Score | Date | Verified |
|---|---|---|---|
| GLM-5.2 |
88.10%
|
17.06.2026 | Verified |
| DeepSeek-V4-Pro |
65.02%
|
17.06.2026 | Unverified |
| Gemini 3.1 Pro |
52.98%
|
17.06.2026 | Unverified |
| GLM-5.1 |
69.52%
|
17.06.2026 | Unverified |
| GPT-5.5 |
98.40%
|
17.06.2026 | Unverified |
| Claude Opus 4.8 |
100.00%
|
17.06.2026 | Unverified |
| GLM-5.3 |
23.22%
|
28.08.2026 | Verified |
| Kimi K3 |
21.04%
|
28.08.2026 | Verified |
| Qwen3.8-2.4T-A95B |
10.89%
|
28.08.2026 | Verified |
| Opus-4.8 |
18.14%
|
28.08.2026 | Verified |
| Fable-5 |
43.54%
|
28.08.2026 | Verified |
| Hy3 |
3.00
|
28.08.2026 | Verified |
| Hy4 preview |
21.04%
|
28.08.2026 | Verified |
| DeepSeek V4 Pro |
18.14%
|
28.08.2026 | Verified |
| Claude Opus 5 |
52.98%
|
28.08.2026 | Verified |
| DeepSeek-V4.1-Flash |
25.11%
|
10.09.2026 | Verified |
| GPT-5.6 Sol |
29.03%
|
28.08.2026 | Verified |