Qwen3.8-Flash-Next

About

Qwen3.8-Flash-Next (Qwen/Qwen3.8-Flash-Next) is the first open-weight release of the Qwen3.8 architecture - a 125B-parameter unified vision-language model (text, image and video input) with 6B activated parameters plus 51B n-gram embedding and 4B MTP parameters, released August 26, 2026 under the Qwen Community License 1.0.

It introduces three architecture changes over Qwen3.5: Qwen Sparse Attention (QSA) replaces Gated Attention in the hybrid pairing (Gated DeltaNet + QSA) and selects 512-token micro-blocks rather than individual tokens (24 Q / 2 KV heads, head dim 256, MQA indexer with 4 query + 1 shared key head of dim 128, budget 512 blocks / 2048 tokens), cutting long-context latency for agentic workloads; Gated Residual modulates widened residual streams with an element-wise data-dependent read gate and per-branch scalar write gates (4 branches, bottleneck rank 320) for finer cross-layer expressiveness at low inference overhead; and N-gram Embedding scales parameters through 20M bigram/trigram embeddings at layer 2 - more offload-friendly and computation-cheaper than MoE. The 48-layer stack repeats 12x (3 Gated DeltaNet -> MoE + 1 QSA -> MoE) with 512 experts (10 routed + 1 shared per token, expert dim 640), hidden size 2560, 248K vocabulary, one Multi-Token Prediction (MTP) layer trained with multi-steps, and a 262,144-token native context extensible to 1,000,000.

Its training recipe applies the Muon and AdamW optimizers to distinct weight categories, and - guided by refitted scaling laws - eliminates batch-size warmup, starting directly at the target batch size to cut total optimizer steps while supporting larger learning rates.

Training Data Pre-training & post-training (card Model Overview); tailored recipe: Muon optimizer for specific weight categories + AdamW for others, refitted scaling laws, no batch-size warmup (card Highlights). No public token count.