Qwen3.5-35B-A3B

Qwen Team (Alibaba Cloud)

Parameters

35.0B total / 3.0B active

MoE: total / active

Architecture

Hybrid Gated DeltaNet + Gated Attention MoE

Released

24.02.2026

License

Apache License 2.0

Open Weights Commercial Use Multimodal BF16 Qwen3.5 en zh multilingual (201 languages)

Input Modalities

text image video

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

1,010,000 tokens

Openness Index Score 100.0/100

About

Qwen3.5-35B-A3B (Qwen/Qwen3.5-35B-A3B) is the mid-size sparse Mixture-of-Experts multimodal model of Alibaba's Qwen3.5 generation - 35B total parameters with 3B activated per token, handling text, image and video input across 201 languages and dialects. Its 40-layer hybrid stack repeats 10x (3x (Gated DeltaNet -> MoE) + 1x (Gated Attention -> MoE)): 30 linear-attention layers (Gated DeltaNet, 32 V / 16 QK heads, head dim 128) and 10 full-attention layers (Gated Attention, 16 Q / 2 KV heads, head dim 256, RoPE dim 64), with 256 experts (8 routed + 1 shared per token, expert intermediate dimension 512), hidden size 2048, 248K vocabulary, Multi-Token Prediction (MTP) trained with multi-steps, and a 262,144-token native context extensible to 1,010,000.

Qwen3.5's unified vision-language foundation uses early fusion training on multimodal tokens with near-100% multimodal training efficiency versus text-only training, achieving cross-generational parity with Qwen3 and outperforming Qwen3-VL across reasoning, coding, agents and visual understanding. Post-training scales RL across million-agent environments with progressively complex task distributions for robust real-world adaptability. This model is also the base of Qwen-AgentWorld-35B-A3B (world-model post-training). Released February 24, 2026 under Apache 2.0.

Training Data Pre-training on multimodal tokens (early fusion, near-100% multimodal training efficiency vs text-only); post-training with scaled RL across million-agent environments. 201 languages and dialects.

Benchmark Scores

Benchmark Score Date Standing
SWE-bench Verified
coding_agent
79.82%
16.04.2026
#40/60
better than 34%
SWE-bench Multilingual
coding_agent
65.44%
16.04.2026
#37/48
better than 23%
SWE-bench Pro
coding_agent
49.72%
16.04.2026
#53/77
better than 32%
Terminal-Bench 2.0
coding_agent
23.50%
16.04.2026
#17/23
better than 27%
Claw-Eval Avg
coding_agent
78.67%
16.04.2026
#14/40
better than 67%
SkillsBench Avg5
coding_agent
4.40
16.04.2026
#16/16
better than 0%
QwenClawBench
coding_agent
88.34%
16.04.2026
#6/17
better than 69%
NL2Repo
coding_agent
15.69%
16.04.2026
#36/41
better than 12%
QwenWebBench
coding_agent
978.00
16.04.2026
#8/8
better than 0%
TAU3-Bench
general_agent
100.00%
16.04.2026
#1/21
better than 100%

Showing 10 of 111 results

What is a good SWE-bench Verified score? What is a good SWE-bench Multilingual score? What is a good SWE-bench Pro score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 70.0 GB 0.5 GB 74.0 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 35.0 GB 0.2 GB 37.0 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 17.5 GB 0.1 GB 18.5 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 17.5 GB 0.1 GB 18.5 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 74.0 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB Apple M1 Ultra · 64GB

Architecture

Decoder Block ×40 input Embedding vocab 248K · d 2048 Linear / Recurrent Hybrid 16:2 · dₕ 256 ×30 Full Attention Hybrid 16:2 · dₕ 256 ×10 MoE FFN 256 experts · top-8 · dᴻ 512 Final Norm LM Head vocab 248K output
Attention
Hybrid Attention (16:2)
MoE
256 experts · top-8 per token
Layers
40
Hidden size
2048
Context
262K tokens
Parameters
35000M
Active params
3000M

Source: Hugging Face config.json · Qwen3_5MoeForConditionalGeneration · exact layer pattern · model repo

Type: Hybrid (Gated DeltaNet + Gated Attention) MoE
Attention: Gated Attention (16 Q heads, 2 KV heads, head_dim=256, RoPE dim=64) + Gated DeltaNet linear attention (32 V heads, 16 QK heads, head_dim=128)
Decoder: Causal Language Model with Vision Encoder
MoE: yes (256 experts)
Routing: 8 Routed + 1 Shared expert routing
Lm Output

248K

Token Embedding

248K

Training Pipeline

  1. 1
    pretraining

    Pre-training

    Pre-training on multimodal tokens with early fusion, achieving near-100% multimodal training efficiency compared to text-only training. Coverage of 201 languages and dialects. Next-generation training infrastructure.

  2. 2
    sft

    Supervised Fine-Tuning

    Post-training supervised fine-tuning stage following pre-training. Part of the Pre-training & Post-training pipeline described in the model card.

  3. 3
    rl

    Reinforcement Learning

    Reinforcement learning scaled across million-agent environments with progressively complex task distributions for robust real-world adaptability. Asynchronous RL frameworks supporting massive-scale agent scaffolds and environment orchestration.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Multimodal pre-training corpus (early fusion, 201 languages) pretraining — —
Million-agent RL environments (progressively complex tasks) rl — —

Linked Resources

blog

Qwen3.5 Blog Post

https://qwen.ai/blog?id=qwen3.5

 website

Qwen Chat

https://chat.qwen.ai/

 api_docs

Alibaba Cloud Model Studio - Text Generation User Guide

https://www.alibabacloud.com/help/en/model-studio/text-generation

 github

SGLang

https://github.com/sgl-project/sglang

 github

vLLM

https://github.com/vllm-project/vllm

 website

vLLM Qwen3.5 Recipe

https://docs.vllm.ai/projects/recipes/en/latest/Qwen/Qwen3.5.html

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Qwen3.5-35B-A3B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON