Qwen3.8-Flash-Next

Qwen3.8-Flash-Next achieves best-in-class results across multiple benchmark categories with only 6B activated parameters:

Language Benchmarks (best results in bold):

• DeepSWE 1.1: 58.7 (vs 42.2 Qwen3.8-27B, 54.4 DeepSeek-V4-Flash)
• SWE-bench Pro: 62.5 (vs 61.7 Qwen3.8-27B)
• SWE-bench Multilingual: 81.0 (vs 77.5 Claude-Opus-4.6)
• CoWorkBench: 73.9 (vs 70.7 Qwen3.8-27B)
• JobBench: 55.7 (vs 41.3 DeepSeek-V4-Flash)
• Agents' Last Exam Score: 51.2 (vs 42.9 Qwen3.8-27B)
• Toolathlon Verified: 73.5 (vs 70.3 DeepSeek-V4-Flash)
• IFBench: 81.3 (vs 79.5 Qwen3.8-27B)
• GPQA Diamond: 91.7 (vs 91.3 Claude-Opus-4.6)
• LiveCodeBench v6: 91.9 (vs 90.6 DeepSeek-V4-Flash)

Vision Benchmarks (best results in bold):

• ClawEval-MM Pass@3: 64.4 (vs 57.4 Qwen3.8-27B)
• ClawEval-MM Average: 60.4 (vs 60.1 Qwen3.7-Plus)
• RecreationBench: 49.9 (vs 47.1 Qwen3.8-27B)
• AndroidWorld: 84.5 (vs 81.9 Qwen3.8-27B)
• OSWorld 2.0 Binary: 19.4 (tied with Qwen3.8-27B)
• OSWorld 2.0 Partial: 52.3 (vs 48.0 Qwen3.8-27B)
• Vision2Web: 64.0 (vs 62.9 Qwen3.8-27B)
• ERQA: 72.3 (vs 69.8 Qwen3.7-Plus)
• LVBench: 76.6 (vs 76.2 Qwen3.7-Plus)
• RealWorldQA: 88.5 (vs 86.9 Qwen3.7-Plus)
• MathVision: 90.6 (vs 90.3 Qwen3.7-Plus)
• CharXiv (RQ): 84.6 (vs 85.8 Qwen3.7-Plus)

Qwen3.8-Flash-Next includes a Vision Encoder and supports image, video, and text inputs.

Input Modalities: text, image, video Output Modalities: text

The model can process:

• Images via image_url content type
• Videos via video_url content type with configurable fps sampling (default fps=2)
• Long videos with recommended video_preprocessor_config longest_edge=469,762,048 (224k video tokens)

Qwen3.8-Flash-Next operates in thinking mode by default, generating thinking content signified by \u003cthink\u003e\n...\n\u003c/think\u003e before producing final responses.

Sampling Parameters:

• Thinking Mode: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
• Instruct (non-thinking) Mode: temp=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Reasoning Effort Levels: xhigh (default), medium, low.

Preserved Thinking: Retains thinking blocks from all historical messages for full context continuity, beneficial for agent scenarios. Can be disabled via preserve_thinking=False.

Qwen3.8-Flash-Next natively supports 262,144 tokens context length and is extensible up to 1,000,000 tokens using YaRN (Yet another RoPE extensioN) scaling.

YaRN configuration:

• rope_type: yarn
• rope_theta: 10,000,000
• partial_rotary_factor: 0.25
• mrope_interleaved: true
• mrope_section: [11, 11, 10]

Supported by vLLM, SGLang, and TokenSpeed inference frameworks.

Qwen3.8-Flash-Next includes a 1-layer Multi-Token Prediction (MTP) module trained with multi-steps, contributing 4B parameters to the total count.

MTP helps the model predict multiple future tokens simultaneously, improving both training efficiency and inference speed.

Qwen3.8-Flash-Next uses a tailored training recipe combining the Muon and AdamW optimizers applied to specific weight categories to maximize efficiency.

Key innovations:

• Guided by refitted scaling laws
• Eliminates traditional batch-size warmups, starting directly at the target batch size
• Substantially reduces total optimizer steps
• Safely supports larger learning rates for robust convergence

The model undergoes both pre-training and post-training stages.

The MoE component has 512 experts with 10 routed + 1 shared activated experts per token. Expert intermediate dimension is 640.

This is a key efficiency innovation: only 6B parameters are activated per token out of 125B total, making the model extremely efficient at inference while maintaining high quality.

Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate. This brings finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.

Configuration: 4 branches, bottleneck rank 320.

N-gram Embedding provides a unique axis for parameter scaling that requires less computation and is more amenable to offloading than Mixture-of-Experts (MoE). By indexing with short n-grams (bigrams/trigrams at layer 2), this approach makes parameter scaling highly efficient for memory-constrained accelerators without sacrificing quality.

Scale: 20,000,000 n-gram embeddings, contributing 51B parameters to the total 125B parameter count.

Token Embedding: 248,320 (padded).

Qwen3.8-Flash-Next introduces Hybrid Attention combining Gated DeltaNet (linear attention) and Qwen Sparse Attention (QSA). Unlike traditional sparse attention that selects individual tokens, QSA operates at the micro-block level, significantly cutting long-context latency. This is critical for agentic workloads that dominate real-world usage.

Architecture Layout: 48 layers arranged as 12 \u00d7 (3 \u00d7 (Gated DeltaNet \u2192 MoE) \u2192 1 \u00d7 (Qwen Sparse Attention \u2192 MoE)).

Gated DeltaNet: 48 V heads, 16 QK heads, head dim 128.

Qwen Sparse Attention: 24 Q heads, 2 KV heads, head dim 256, RoPE dim 64, MQA indexer with 4 query heads and 1 shared key head (indexer head dim 128), budget of 512 blocks or 2048 tokens.