Ornith-1.5-35B-A3B

Ornith AI

Parameters

36.0B total / 3.0B active

MoE: total / active

Architecture

Mixture-of-Experts (qwen3_5_moe)

Released

18.08.2026

License

MIT License

Open Weights Commercial Use Multimodal BF16 Ornith-1.5 en multi

Input Modalities

text image

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

1,048,576 tokens

Openness Index Score 100.0/100

About

Ornith-1.5-35B-A3B (ornith-ai/Ornith-1.5-35B-A3B) is the mid-size mixture-of-experts member of the Ornith-1.5 family - ~36B total parameters with only ~3B activated per token (qwen3_5_moe architecture, built on Qwen3.5 and Gemma4 with continued pretraining, mid-training and post-training) - released August 18, 2026 under the MIT License with a 262,144-token native context extensible to 1,048,576.

Ornith-1.5 is a major step toward foundation models through end-to-end self-improvement: it extends Ornith-1.0's scaffold-rollout co-optimization (jointly optimizing scaffold and solution rollouts) by also optimizing task generation. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. Despite activating only ~3B parameters per token, it significantly outperforms its similar-sized peer Qwen 3.6-35B across all coding and agentic benchmarks, and beats dense models such as Gemma 4-31B and Muse Glimmer-30B by wide margins on agentic coding. GGUF builds are available for llama.cpp, Ollama, Atomic.chat, Hermes, OpenClaw and coding CLIs.

Training Data Self-improvement loop jointly optimizing task generation, scaffold construction, and solution rollouts. Built on Qwen3.5 and Gemma4 with continued pretraining, mid-training, and post-training. Continuously generates new training tasks, discovers effective strategies, and improves policy through reinforcement learning.

Benchmark Scores

Benchmark Score Date Standing
Terminal-Bench 2.1 (Terminus-2)
coding_agent
69.03%
19.08.2026
#20/44
better than 56%
Terminal-Bench 2.1 (Claude Code)
coding_agent
100.00%
19.08.2026
#1/8
better than 100%
SWE-bench Verified
coding_agent
90.14%
19.08.2026
#12/60
better than 80%
SWE-bench Multilingual
coding_agent
78.58%
19.08.2026
#24/48
better than 51%
DeepSWE
coding_agent
30.26%
19.08.2026
#16/21
better than 25%
Frontier-Bench v0.1
coding_agent
100.00%
19.08.2026
#1/4
better than 100%
NL2Repo
coding_agent
55.23%
19.08.2026
#18/41
better than 57%
SWE Atlas - QnA
coding_agent
55.84%
19.08.2026
#11/24
better than 57%
Humanity's Last Exam
stem_reasoning
44.51%
19.08.2026
#38/71
better than 47%
HLE (with tools)
stem_reasoning
34.76%
19.08.2026
#42/49
better than 15%

Showing 10 of 17 results

What is a good Terminal-Bench 2.1 (Terminus-2) score? What is a good Terminal-Bench 2.1 (Claude Code) score? What is a good SWE-bench Verified score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 72.0 GB 0.5 GB 76.1 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 36.0 GB 0.2 GB 38.0 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 18.0 GB 0.1 GB 19.0 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 18.0 GB 0.1 GB 19.0 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 76.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB Apple M1 Ultra · 64GB

Architecture

Decoder Block ×40 input Embedding vocab 248K · d 2048 Linear / Recurrent Hybrid 16:2 · dₕ 256 ×30 Full Attention Hybrid 16:2 · dₕ 256 ×10 MoE FFN 256 experts · top-8 · dᴻ 512 Final Norm LM Head vocab 248K output
Attention
Hybrid Attention (16:2)
MoE
256 experts · top-8 per token
Layers
40
Hidden size
2048
Context
262K tokens
Parameters
36000M
Active params
3000M

Source: Hugging Face config.json · Qwen3_5MoeForConditionalGeneration · exact layer pattern · model repo

Type: Mixture-of-Experts
Attention: Grouped Query Attention
Decoder: Causal Transformer
MoE: yes (128 experts)
Routing: Top-K router
Base Architecture

qwen3_5_moe

Training Pipeline

  1. 1
    cpt

    Continued Pretraining on Qwen3.5/Gemma4 base

    Ornith-1.0 was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training.

  2. 2
    sft

    Mid-training and SFT

    Mid-training phase with scaffold and rollout optimization to build foundational agentic capabilities.

  3. 3
    rl

    Self-improvement RL loop (Ornith-1.5)

    Ornith-1.5 expands the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Self-generated RL tasks (task generation + scaffold + solution rollouts) rl — —

Linked Resources

website

Ornith-1.5 Blog

https://ornith.ai/ornith_1_5.html

 website

Ornith Blog (Deep Reinforce)

https://deep-reinforce.com/ornith.html

 website

Ornith-1.5-35B-A3B HuggingFace Model Card

https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B

 website

Ornith-1.5-35B-A3B-GGUF (Quantized)

https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF

 website

GPQA Diamond Leaderboard

https://huggingface.co/datasets/Idavidrein/gpqa

 website

SWE-bench Verified Leaderboard

https://huggingface.co/datasets/SWE-bench/SWE-bench_Verified

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Ornith-1.5-35B-A3B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON