Qwen3-30B-A3B

Qwen

Parameters

30.5B total / 3.3B active

MoE: total / active

Architecture

Mixture-of-Experts Transformer with Hybrid Thinking Mode

Released

27.04.2025

License

Apache License 2.0

Open Weights Commercial Use Multimodal bfloat16 Qwen3 English, Chinese, 119 languages and dialects

Input Modalities

text

Output Modalities

text

Context (native)

32,768 tokens

Context (extended)

131,072 tokens

Openness Index Score 100.0/100

About

Qwen3-30B-A3B is the mid-size sparse Mixture-of-Experts model of Alibaba's Qwen3 generation - a causal decoder-only LLM with 30.5B total parameters (29.9B non-embedding), 3.3B activated per token, and 48 layers. Each MoE layer routes each token to 8 of 128 experts (expert dim 768; dense FFN 6144), with Grouped-Query Attention (GQA) of 32 query heads and 4 KV heads (head dim 128), hidden size 2048, 152K vocabulary and RoPE theta 1M.

Its signature capability is seamless thinking-mode switching: one model serves both a thinking mode (for complex logical reasoning, math, and coding) and a non-thinking mode (for efficient general-purpose dialogue), set via the chat template (enable_thinking) or no_think in the prompt. Qwen3 delivers significantly enhanced reasoning (surpassing QwQ and Qwen2.5 in both modes), superior human-preference alignment for creative writing, role-play and multi-turn dialogue, and expertise in agentic tool integration in both modes. It supports 100+ languages and dialects with strong multilingual instruction following and translation.

Pre-trained on ~36 trillion tokens across 119 languages with a four-stage post-training pipeline (CoT cold start, reasoning RL, thinking-mode fusion, general RL), it has a native context of 32,768 tokens extendable to 131,072 with YaRN. Released April 27, 2025 under Apache 2.0.

Training Data Pretrained on ~36 trillion tokens across 119 languages; 4-stage post-training: CoT cold start, reasoning RL, thinking mode fusion, general RL. Supports both thinking and non-thinking modes.

Benchmark Scores

Benchmark Score Date Standing
MMLU-Pro
knowledge
59.68%
29.04.2025
#30/48
better than 38%
MMLU-Redux
knowledge
74.37%
29.04.2025
#17/30
better than 45%
GPQA Diamond
stem_reasoning
46.11%
29.04.2025
#68/80
better than 15%
SuperGPQA
knowledge
57.66%
29.04.2025
#18/28
better than 37%
HMMT Feb 25
stem_reasoning
49.80
29.04.2025
#17/17
better than 0%
LiveCodeBench v6
stem_reasoning
50.75%
29.04.2025
#40/49
better than 19%
OJBench
stem_reasoning
26.58%
29.04.2025
#16/22
better than 29%
IFEval
instruction_following
83.45%
29.04.2025
#22/26
better than 16%
MMLU-ProX
multilingual
14.19%
29.04.2025
#17/19
better than 11%
INCLUDE
multilingual
6.98%
29.04.2025
#9/11
better than 20%

Showing 10 of 12 results

What is a good MMLU-Pro score? What is a good MMLU-Redux score? What is a good GPQA Diamond score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.3B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 61.0 GB 0.5 GB 64.6 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 30.5 GB 0.3 GB 32.3 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 15.2 GB 0.1 GB 16.1 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 15.2 GB 0.1 GB 16.1 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 64.6 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

AMD Radeon RX 7900 XT · 20GB GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB

Architecture

Decoder Block input Embedding vocab 152K · d 2048 Full Attention GQA 32:4 · dₕ 128 ×48 MoE FFN 128 experts · top-8 · dᴻ 768 Final Norm LM Head vocab 152K output
Attention
Grouped Query Attention (32:4)
MoE
128 experts · top-8 per token
Layers
48
Hidden size
2048
Context
41K tokens
RoPE θ
1M
Parameters
30500M
Active params
3300M

Source: Hugging Face config.json · Qwen3MoeForCausalLM · model repo

Type: Sparse MoE decoder (Qwen3-30B-A3B): 128 experts, GQA
Attention: Grouped-Query Attention: 32 Q heads / 4 KV heads, head dim 128, RoPE theta 1M; no SWA
Decoder: MoE causal decoder-only (48 layers)
MoE: yes (128 experts)
Routing: Top-8 of 128 experts per token, no shared expert

Training Pipeline

  1. 1
    pretraining

    Pre-training (~36T tokens, 119 languages)

    Large-scale multilingual pre-training; 36T tokens.

  2. 2
    sft

    Post-training: CoT cold start + reasoning RL (SFT phases)

    Four-stage post-training: long-CoT cold start, reasoning-centric SFT, thinking-mode fusion, general RL (DB training_data_info).

  3. 3
    rl

    Thinking mode fusion + general RL

    Thinking-mode fusion stage aligning both modes in one checkpoint, followed by general-domain RL.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
~36T token pretraining corpus (119 languages) pretraining — —

Linked Resources

blog

Qwen3 launch blog (QwenLM)

https://qwenlm.github.io/blog/qwen3/

 github

QwenLM/Qwen3 (GitHub)

https://github.com/QwenLM/Qwen3

 website

Qwen documentation

https://qwen.readthedocs.io/en/latest/

 collection

Qwen3 (HuggingFace collection)

https://huggingface.co/collections/Qwen/qwen3-66dd1b639b283947e839b4ce

 demo

Qwen3 demo (chat.qwen.ai)

https://chat.qwen.ai

 citation

BibTeX citation (Qwen team)

https://huggingface.co/Qwen/Qwen3-30B-A3B

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Qwen3-30B-A3B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON