Qwen3-30B-A3B-Thinking-2507

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.3B are active per token

Only datacenter GPUs — FP16 needs 64.6 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

Architecture

Training Pipeline

1. 1
   pretraining

   Three-Stage Pretraining (36T tokens, 119 languages)

   Stage 1 (S1): Basic pretraining on over 30 trillion tokens with 4K context length. Stage 2 (S2): Improved dataset with increased proportion of knowledge-intensive data (STEM, coding, reasoning), 5 trillion additional tokens. Stage 3: Long-context extension to 32K tokens using high-quality long-context data. Total: ~36 trillion tokens across 119 languages.

2. 2
   sft

   Long CoT Cold Start

   Fine-tuned models using diverse long chain-of-thought (CoT) data covering mathematics, coding, logical reasoning, and STEM problems. This equipped the model with fundamental reasoning abilities.

3. 3
   rl

   Reasoning-Based Reinforcement Learning

   Scaled up computational resources for RL, utilizing rule-based rewards to enhance the model's exploration and exploitation capabilities for reasoning tasks.

4. 4
   sft

   Thinking Mode Fusion

   Integrated non-thinking capabilities into the thinking model by fine-tuning on a combination of long CoT data and instruction-tuning data generated by the enhanced thinking model from stage 2. This created a seamless blend of reasoning and quick response capabilities.

5. 5
   rl

   General Reinforcement Learning

   Applied RL across more than 20 general-domain tasks including instruction following, format following, and agent capabilities. This strengthened general capabilities and corrected undesired behaviors.

Training & Evaluation Datasets

Linked Resources