Ornith-1.0-35B-A3B

Ornith AI

Parameters

35.0B total / 3.0B active

MoE: total / active

Architecture

Mixture-of-Experts (qwen3_5_moe)

Released

01.07.2026

License

MIT License

Open Weights Commercial Use Multimodal BF16 Ornith-1.0 en multi

Input Modalities

text image

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

262,144 tokens

Openness Index Score 100.0/100

About

Ornith-1.0-35B (ornith-ai/Ornith-1.0-35B) is the lightweight 35B MoE member of Ornith 1.0, deepreinforce-ai's self-improving family of open-source models for agentic coding, designed for efficient single-GPU deployment. Ornith 1.0 ships in 9B-Dense, 31B-Dense, 35B-MoE and 397B-MoE variants post-trained on top of Gemma 4 and Qwen 3.5, achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.

Its signature is the self-improving training framework: reinforcement learning teaches the model to generate not only solution rollouts but also the scaffolds that drive those rollouts - jointly optimizing scaffold-rollout co-optimization so the model discovers better search trajectories and produces higher-quality solutions. The architecture is a Qwen3.5-style sparse Mixture-of-Experts with 35B total and 3B active parameters: 40 layers mixing 30 linear attention layers (Gated DeltaNet-style: conv kernel 4, 16 key heads / 32 value heads of dim 128) with 10 full-attention layers (interval 4; 16 Q / 2 KV heads, head dim 256, partial RoPE 0.25, theta 10M, interleaved MRoPE [11,11,10]), 256 routed experts with 8 active per token plus 1 shared expert (expert FFN 512), one Multi-Token Prediction (MTP) layer, 248K vocabulary, a 262,144-token context, and a Qwen3.5 ViT (depth 27, patch 16, spatial merge 2) for image input.

Developed on top of Qwen3.5 and Gemma 4 with additional continued pretraining, mid-training and post-training. Released July 1, 2026 under the MIT License.

Training Data Developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training. Uses scaffold and rollout optimization.

Benchmark Scores

Benchmark Score Date Standing
Terminal-Bench 2.1 (Terminus-2)
coding_agent
63.72%
19.08.2026
#23/44
better than 47%
Terminal-Bench 2.1 (Claude Code)
coding_agent
88.51%
19.08.2026
#2/8
better than 71%
SWE-bench Verified
coding_agent
86.24%
19.08.2026
#23/60
better than 61%
SWE-bench Multilingual
coding_agent
76.09%
19.08.2026
#26/48
better than 40%
DeepSWE
coding_agent
0.00
19.08.2026
#20/21
better than 0%
Frontier-Bench v0.1
coding_agent
1.40
19.08.2026
#2/4
better than 0%
NL2Repo
coding_agent
37.38%
19.08.2026
#29/41
better than 28%
SWE Atlas - QnA
coding_agent
50.91%
19.08.2026
#13/24
better than 43%
Humanity's Last Exam
stem_reasoning
35.42%
19.08.2026
#49/71
better than 31%
HLE (with tools)
stem_reasoning
28.88%
19.08.2026
#44/49
better than 10%

Showing 10 of 17 results

What is a good Terminal-Bench 2.1 (Terminus-2) score? What is a good Terminal-Bench 2.1 (Claude Code) score? What is a good SWE-bench Verified score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 70.0 GB 0.5 GB 74.0 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 35.0 GB 0.2 GB 37.0 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 17.5 GB 0.1 GB 18.5 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 17.5 GB 0.1 GB 18.5 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 74.0 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB Apple M1 Ultra · 64GB

Architecture

Decoder Block ×40 input Embedding vocab 248K · d 2048 Full Attention Hybrid 16:2 · dₕ 256 ×10 Linear / Recurrent Hybrid 16:2 · dₕ 256 ×30 MoE FFN 256 experts · top-8 · +1 shared · dᴻ 512 MTP Head ×1 speculative layer Final RMSNorm LM Head vocab 248K output
Attention
Hybrid Attention (16:2)
MoE
256 experts · top-8 per token
Layers
40
Hidden size
2048
Context
262K tokens
RoPE θ
10M
Parameters
35000M
Active params
3000M

Source: extracted model record · model repo

Type: Sparse MoE hybrid linear-attention decoder (Ornith-1.0-35B, Qwen3.5 base)
Attention: Hybrid: 30 linear-attention layers (Gated DeltaNet style, conv kernel 4, key heads 16x128, value heads 32x128) + 10 full-attention layers (interval 4; 16 Q / 2 KV heads, head dim 256, partial RoPE 0.25, theta 10M, interleaved MRoPE [11,11,10])
Decoder: Sparse MoE decoder-only with linear-attention/full hybrid (Qwen3_5Moe)
MoE: yes (256 experts)
Routing: Top-8 of 256 routed experts + 1 shared expert (shared expert FFN 512)

Training Pipeline

  1. 1
    cpt

    Continued pretraining on top of Qwen3.5 and Gemma 4

    Additional continued pretraining, mid-training and post-training on top of Gemma 4 and Qwen 3.5 bases (DB training_data_info).

  2. 2
    other

    Mid-training

    Mid-training stage between CPT and post-training (DB training_data_info).

  3. 3
    rl

    Self-improving scaffold-rollout RL (post-training)

    RL jointly optimizing scaffolds and solution rollouts; discovers better search trajectories, higher-quality solutions (card Self-Improving Training Framework).

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Agentic coding RL rollouts (solution + scaffold) rl — —

Linked Resources

github

deepreinforce-ai (GitHub)

https://github.com/deepreinforce-ai

 github

GGUF builds (deepreinforce-ai/Ornith-1.0-35B, llama.cpp/Ollama serve)

https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B

 collection

Ornith (HuggingFace collection)

https://huggingface.co/ornith-ai

 demo

Hermes agent runtime (OpenAI-compatible endpoint)

https://huggingface.co/ornith-ai/Ornith-1.0-35B

 website

OpenClaw / OpenHands / opencode agent frameworks usage

https://huggingface.co/ornith-ai/Ornith-1.0-35B#agentic-usage

 citation

Citation (Ornith-1.0)

https://huggingface.co/ornith-ai/Ornith-1.0-35B#citation

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Ornith-1.0-35B-A3B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON