Qwen3.5-122B-A10B

About

Qwen3.5-122B-A10B (Qwen/Qwen3.5-122B-A10B) is a flagship sparse Mixture-of-Experts model of Alibaba's Qwen3.5 generation - a unified vision-language foundation model with 122B total parameters and 10B activated per token, handling text, image and video input and generating text in 201 languages and dialects. It was trained with Early fusion (multimodal pre-training) on multimodal tokens, achieving cross-generational parity with Qwen3 and outperforming Qwen3-VL across reasoning, coding, agents and visual understanding.

Its efficient hybrid architecture combines Gated DeltaNet linear attention with sparse MoE: 48 layers in a 12x (3 Gated DeltaNet + 1 Gated Attention) layout (36 linear-attention layers with 64 V / 16 QK heads of dim 128; 12 full-attention layers with 32 Q / 2 KV heads, head dim 256, partial RoPE 0.25 theta 10M, interleaved MRoPE [11,11,10]), 256 experts with 8 routed + 1 shared activated per token (expert dim 1024), hidden size 3072, 248K vocabulary, Multi-Token Prediction (MTP) trained with multi-steps, and a 262,144-token native context extensible up to 1,010,000 tokens.

Qwen3.5 scales RL generalization via million-agent RL environments with progressively complex task distributions, supported by next-generation infrastructure: near-100% multimodal training efficiency and asynchronous RL frameworks for massive-scale agent scaffolds. Released February 24, 2026 under Apache 2.0.

Training Data Early-fusion multimodal pre-training on unified text/image tokens (201 languages); post-training RL scaled across million-agent environments (card Qwen3.5 Highlights).