NL2Repo
NL2Repo - repository generation from natural language. Others evaluated via Claude Code (temp=1.0, top_p=0.95, max_turns=900).
Category
Agentic Coding
Max Score
100.0
Score Type
percent
Active
Yes
Model Scores
Scores for NL2Repo
| Model | Score | Date | Verified |
|---|---|---|---|
| Qwen3.6-35B-A3B |
29.38%
|
16.04.2026 | Verified |
| Gemma4-31B-it |
8.00%
|
16.04.2026 | Unverified |
| Qwen3.5-35B-A3B |
15.69%
|
16.04.2026 | Unverified |
| Gemma4-26B-A4B |
2.00%
|
16.04.2026 | Unverified |
| Qwen3.6-27B |
39.85%
|
22.04.2026 | Verified |
| Qwen3.5-397B-A17B |
33.69%
|
22.04.2026 | Unverified |
| Claude 4.5 Opus |
50.62%
|
22.04.2026 | Unverified |
| Qwen3.5-27B |
26.15%
|
16.04.2026 | Unverified |
| Ornith-1.0-9B |
26.00%
|
25.06.2026 | Verified |
| Qwen3.5-9B |
9.08%
|
25.06.2026 | Unverified |
| Gemma4-12B |
10.30
|
25.06.2026 | Unverified |
| Ornith-1.0-35B |
37.38%
|
25.06.2026 | Verified |
| Qwen3.8-27B |
49.23%
|
18.08.2026 | Verified |
| Qwen3.7-Plus |
47.38%
|
18.08.2026 | Unverified |
| Opus4.6 Max |
57.38%
|
18.08.2026 | Unverified |
| GLM-5.2 |
59.38%
|
17.06.2026 | Verified |
| DeepSeek-V4-Pro |
38.77%
|
17.06.2026 | Unverified |
| Gemini 3.1 Pro |
35.54%
|
17.06.2026 | Unverified |
| GLM-5.1 |
49.85%
|
17.06.2026 | Unverified |
| GPT-5.5 |
62.15%
|
17.06.2026 | Unverified |
| Claude Opus 4.8 |
91.38%
|
17.06.2026 | Unverified |
| Qwen3.7-Max |
56.77%
|
17.06.2026 | Unverified |
| MiniMax-M3 |
48.92%
|
17.06.2026 | Unverified |
| DeepSeek-V4-Flash |
44.77%
|
13.08.2026 | Verified |
| Opus-4.8 |
91.38%
|
31.07.2026 | Verified |
| DeepSeek-V4-Pro |
43.38%
|
13.08.2026 | Verified |
| Ornith-1.5-35B-A3B |
55.23%
|
19.08.2026 | Verified |
| Ornith-1.0-35B-A3B |
37.38%
|
19.08.2026 | Unverified |
| Gemma-4-31B |
8.00%
|
19.08.2026 | Unverified |
| DeepSeek-V4-Pro-0813 |
78.77%
|
13.08.2026 | Verified |
| GLM-5.3 |
73.38%
|
28.08.2026 | Verified |
| Kimi K3 |
73.38%
|
28.08.2026 | Verified |
| Qwen3.8-2.4T-A95B |
70.15%
|
28.08.2026 | Verified |
| Hy3 |
54.31%
|
28.08.2026 | Verified |
| Hy4 preview |
74.77%
|
28.08.2026 | Verified |
| DeepSeek V4 Pro |
78.77%
|
28.08.2026 | Verified |
| GPT-5.6 Sol |
71.54%
|
28.08.2026 | Verified |
| Claude Opus 5 |
100.00%
|
28.08.2026 | Verified |
| DeepSeek-V4-Flash-Vision-Exp |
72.92%
|
08.09.2026 | Verified |
| DeepSeek-V4-Flash-0731 |
67.54%
|
31.07.2026 | Verified |