Qwen3.8-2.4T-A95B

Qwen

Parameters

2.4T total / 95.0B active

MoE: total / active

Architecture

92-layer hybrid MoE: 23 x (3 x (Gated DeltaNet -> MoE) + 1 x (Gated Attention -> MoE)); 512 experts (10 routed + 1 shared, dim 2048); hidden 8192; GDN 128V/16QK dim 128; GA 64Q/4KV dim 256; MTP multi-steps; ctx 262144 -> 1010000.

Released

08.08.2026

License

Qwen3.8-Max License

Open Weights Commercial Use Multimodal BF16 Qwen3.8 en zh

Input Modalities

text

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

1,010,000 tokens

Openness Index Score 70.0/100

About

Qwen3.8-2.4T-A95B (Qwen/Qwen3.8-2.4T-A95B) is the flagship sparse Mixture-of-Experts model of Alibaba's Qwen3.8 generation - 2.4T total parameters with 95B activated per token - released August 8, 2026 under the Qwen3.8-Max License. Its 92-layer hybrid stack repeats 23x (3x (Gated DeltaNet -> MoE) + 1x (Gated Attention -> MoE)): 69 linear-attention layers (Gated DeltaNet, 128 V / 16 QK heads, head dim 128) and 23 full-attention layers (Gated Attention, 64 Q / 4 KV heads, head dim 256, RoPE dim 64), with 512 experts (10 routed + 1 shared per token, expert intermediate dim 2048), hidden size 8192, 248K vocabulary, Multi-Token Prediction (MTP) trained with multiple steps, and a 262,144-token native context extensible to 1,010,000 tokens.

Qwen3.8 delivers comprehensive improvements across coding, professional work, research and long-horizon agentic tasks; stronger autonomous planning and handling of environment feedback for reliable end-to-end agent execution; broader compatibility with popular harnesses and development tools; and flexible thinking control - reasoning depth tunable via reasoning_effort with reasoning context from historical messages retained via preserve_thinking.

Training Data Pre-training & Post-training

Benchmark Scores

Benchmark Score Date Standing
Terminal Bench 2.1
coding_agent
95.57%
18.08.2026
#7/44
better than 86%
SWE-bench Pro
coding_agent
75.47%
18.08.2026
#5/77
better than 95%
NL2Repo-Bench
coding_agent
63.87%
18.08.2026
#10/23
better than 59%
FrontierSWE
coding_agent
75.17%
18.08.2026
#5/12
better than 64%
MLS-Bench-Lite
coding_agent
61.64%
18.08.2026
#5/9
better than 50%
PaperBench
coding_agent
100.00%
18.08.2026
#1/10
better than 100%
AndroidBench
coding_agent
66.43%
18.08.2026
#2/5
better than 75%
QwenSWEBench
coding_agent
84.86%
18.08.2026
#3/9
better than 75%
QwenQoderBench
coding_agent
82.13%
18.08.2026
#3/5
better than 50%
QwenReactBench
coding_agent
80.17%
18.08.2026
#2/5
better than 75%

Showing 10 of 68 results

What is a good Terminal Bench 2.1 score? What is a good SWE-bench Pro score? What is a good NL2Repo-Bench score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 95.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 4800.0 GB 14.7 GB 5054.7 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 2400.0 GB 7.4 GB 2527.4 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 1200.0 GB 3.7 GB 1263.7 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 1200.0 GB 3.7 GB 1263.7 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 5054.7 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block ×92 input Embedding vocab 248K · d 8192 Linear / Recurrent Hybrid 64:4 · dₕ 256 ×69 Full Attention Hybrid 64:4 · dₕ 256 ×23 MoE FFN 512 experts · top-10 · dᴻ 2048 Final Norm LM Head vocab 248K output
Attention
Hybrid Attention (64:4)
MoE
512 experts · top-10 per token
Layers
92
Hidden size
8192
Context
262K tokens
Parameters
2400000M
Active params
95000M

Source: Hugging Face config.json · Qwen3_5MoeForCausalLM · exact layer pattern · model repo

Type: Causal Language Model
Attention: Hybrid (Gated DeltaNet + Gated Attention)
Decoder: Autoregressive
MoE: yes (512 experts)
Routing: Top-k router with 10 routed experts + 1 shared expert
Lm Output

248K

Token Embedding

248K

Training Pipeline

  1. 1
    pretraining

    Pre-training

    Pre-training stage: Causal Language Model trained on large-scale multilingual corpus with Multi-Token Prediction (MTP). 2.4T total parameters, 95B activated parameters, MoE architecture with 512 experts (10 routed + 1 shared).

  2. 2
    sft

    Post-training (SFT + RL)

    Post-training stage: The model is a post-trained version with reasoning capabilities. Features reasoning_effort control (xhigh/medium/low) and preserve_thinking for retaining reasoning context across messages. Thinking mode is always on and cannot be disabled.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Qwen3.8 pre-training corpus pretraining — —

Linked Resources

blog

Qwen3.8-Max: A New Bar for Coding and Cowork

https://qwen.ai/blog?id=qwen3.8

 citation

Qwen3.8-Max: A New Bar for Coding and Cowork

https://qwen.ai/blog?id=qwen3.8

 website

Qwen Cloud - Qwen3.8-Max

https://www.qwencloud.com/models/qwen3.8-max

 other

SGLang Qwen3.8 Cookbook

https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8

 other

vLLM Qwen3.8 Recipe

https://recipes.vllm.ai/Qwen/Qwen3.8-2.4T-A95B

 other

TokenSpeed Qwen3.8 Recipe

https://lightseek.org/tokenspeed/recipes/models#qwen3-8

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Qwen3.8-2.4T-A95B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON