Qwen3-VL-30B-A3B

Qwen

Parameters

30.0B total / 3.0B active

MoE: total / active

Architecture

Mixture-of-Experts Vision-Language Transformer

Released

04.10.2025

License

Apache License 2.0

Open Weights Commercial Use Multimodal BF16 Qwen3-VL en zh multi

Input Modalities

text image video

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

1,000,000 tokens

Openness Index Score 100.0/100

About

Qwen3-VL-30B-A3B (Instruct) is the MoE vision-language model of the Qwen3-VL series, the most powerful VLM family the Qwen team (Alibaba) has released to date. It pairs a Qwen3-ViT vision encoder (initialized from SigLIP-2 SO-400M and continued on dynamic resolutions with 2D-RoPE) with the Qwen3-30B-A3B MoE LLM backbone (30B total, 3B activated per token, 128 experts with 8 routed per token), a two-layer MLP vision-language merger that compresses 2x2 visual features into one visual token, and DeepStack fusion of multi-level ViT features.

Its defining novelties are Interleaved-MRoPE (redesigned multimodal RoPE that interleaves temporal/height/width frequency components for long-horizon video reasoning) and Text-Timestamp Alignment, which moves beyond T-RoPE to precise, timestamp-grounded event localization for video temporal modeling. Key capabilities: a Visual Agent that operates PC/mobile GUIs, visual coding (Draw.io/HTML/CSS/JS from images and videos), advanced spatial perception with 2D/3D grounding, native 256K context expandable to 1M, hours-long video understanding with second-level indexing, OCR in 32 languages, and multimodal STEM/Math reasoning. Text understanding is on par with pure LLMs. Pre-training spans vision-language alignment (67B tokens), ~2T multimodal/long-context tokens and 100B ultra-long-context tokens; post-training uses SFT, strong-to-weak distillation and RL. Maintained by the Qwen team; Apache-2.0 weights on HuggingFace and Ollama; serves English, Chinese and more (32 OCR languages, multilingual benchmarks).

Training Data Pre-training: VL alignment (67B tokens) -> multimodal pre-training (~1T, 8K seq) -> long-context pre-training (~1T, 32K seq) -> ultra-long-context adaptation (100B, 256K seq). Post-training: SFT -> strong-to-weak distillation (text-only) -> RL (Reasoning RL + General RL).

Benchmark Scores

Benchmark	Score	Date	Standing
MMLU-Pro knowledge	57.42%	04.10.2025	#32/48 better than 34%
MMLU-Redux knowledge	69.75%	04.10.2025	#21/30 better than 31%
GPQA reasoning	89.07%	04.10.2025	#4/14 better than 77%
SuperGPQA knowledge	60.59%	04.10.2025	#16/28 better than 44%
HMMT Feb 25 stem_reasoning	1.78%	04.10.2025	#16/17 better than 6%
LiveBench 241125 reasoning	65.40	04.10.2025	#2/2 better than 0%
Arena-Hard v2 instruction_following	58.50	04.10.2025	#2/2 better than 0%
Creative Writing v3 instruction_following	100.00%	04.10.2025	#1/1 only tracked model
WritingBench instruction_following	82.60	04.10.2025	#2/2 better than 0%
LiveCodeBench v6 stem_reasoning	30.56%	04.10.2025	#45/49 better than 8%
BFCL-V3 general_agent	66.30	04.10.2025	#3/3 better than 0%
Multi-IF instruction_following	52.87%	04.10.2025	#9/13 better than 33%
MMLU-ProX multilingual	70.90	04.10.2025	#19/19 better than 0%
PolyMATH multilingual	44.30	04.10.2025	#11/11 better than 0%
INCLUDE multilingual	4.65%	04.10.2025	#10/11 better than 10%
AIME 2025 stem_reasoning	56.39%	04.10.2025	#18/24 better than 26%
IFEval instruction_following	82.32%	04.10.2025	#23/26 better than 12%

Showing 10 of 17 results

What is a good MMLU-Pro score? What is a good MMLU-Redux score? What is a good GPQA score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token

Quantization	Weights	KV Cache	Total VRAM	On your hardware
FP16 — 16-bit	60.0 GB	0.5 GB	63.5 GB	Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit	30.0 GB	0.2 GB	31.7 GB	Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit	15.0 GB	0.1 GB	15.9 GB	Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit	15.0 GB	0.1 GB	15.9 GB	Which GPUs can run it? Calculate on your GPU

Consumer GPUs that fit this model with a single card (INT4):

AMD Radeon RX 7900 XT · 20GB GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB

Architecture

Attention: Grouped Query Attention (32:4)
MoE: 128 experts · top-8 per token
Layers: 48
Hidden size: 2048
Context: 262K tokens
RoPE θ: 5M
Parameters: 30000M
Active params: 3000M

Source: Hugging Face config.json · Qwen3VLMoeForConditionalGeneration · model repo

Type: MoE Vision-Language Transformer (Qwen3-ViT + Qwen3 MoE LLM)

Attention: Qwen3 LLM backbone (32 Q heads, 4 KV heads) with Interleaved-MRoPE multimodal position encoding

Decoder: 48-layer MoE causal decoder

MoE: yes (128 experts)

Routing: Top-k routing 8 experts per token

Layers 48

Context length 262K

Extended context 1M

Experts 128

Experts per token 8

Attention heads 32

KV heads 4

Head dim 128

Hidden size 2048

Expert FFN dim 768

Vision Yes

RoPE θ 5M

Deepstack Indexes 8, 16, 24

Mrope Interleaved Yes

Mrope Section 24, 20, 20

MTP No

Patch Size 16

Spatial Merge Size 2

Vision Depth 27

Vision Heads 16

Vision Hidden 1152

Training Pipeline

1
other
S0: Vision-Language Alignment

Trains the merger on 67B tokens at sequence length 8,192 (Qwen3-VL TR Table 1).
2
pretraining
S1: Multimodal Pre-Training

All parameters on ~1T tokens at sequence length 8,192 (Qwen3-VL TR Table 1).
3
cpt
S2: Long-Context Pre-Training

All parameters on ~1T tokens at sequence length 32,768 (Qwen3-VL TR Table 1).
4
cpt
S3: Ultra-Long-Context Adaptation

All parameters on 100B tokens at sequence length 262,144 (Qwen3-VL TR Table 1).
5
sft
Supervised Fine-Tuning (32K then 256K context)

Instruction-following SFT in two phases: 32K context, then extension to 256K with long-document and long-video data; standard formats for non-thinking models, CoT formats for thinking models.
6
other
Strong-to-Weak Distillation

Knowledge distillation from a powerful teacher to the student models using text-only data to fine-tune the LLM backbone.
7
rl
RL: Reasoning RL + General RL

Large-scale RL across text and multimodal domains incl. math, OCR, grounding, instruction-following.

Training & Evaluation Datasets

Name	Role	Size	Collection
Image-caption pairs (Chinese-English web)	pretraining	—	—
Interleaved text-image documents	pretraining	—	—
OCR / document parsing data	pretraining	—	—
Video data	pretraining	—	—
Agent data	pretraining	—	—

Linked Resources

paper

Qwen3-VL Technical Report

https://arxiv.org/abs/2511.21631

paper

Qwen3 Technical Report

https://arxiv.org/abs/2505.09388

paper

Qwen2.5-VL Technical Report

https://arxiv.org/abs/2502.13923

github

QwenLM/Qwen3-VL repository

https://github.com/QwenLM/Qwen3-VL

collection

Qwen3-VL HuggingFace collection

https://huggingface.co/collections/Qwen/qwen3-vl

demo

Qwen Chat

https://chat.qwen.ai/

Trend Analysis

24h Change

+0.2%

7d Change

+1.6%

30d Change

+5.6%

Current

116,019

ollama

downloads

+0.0%

huggingface

likes

+0.2%

huggingface

downloads

-0.4%

huggingface

downloads_all_time

+0.0%

View raw metric history →

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Qwen3-VL-30B-A3B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON

Qwen3-VL-30B-A3B

About

Benchmark Scores

Hardware Requirements

Architecture

Training Pipeline

S0: Vision-Language Alignment

S1: Multimodal Pre-Training

S2: Long-Context Pre-Training

S3: Ultra-Long-Context Adaptation

Supervised Fine-Tuning (32K then 256K context)

Strong-to-Weak Distillation

RL: Reasoning RL + General RL

Training & Evaluation Datasets

Linked Resources

Trend Analysis

Related Models

Qwen3.6-27B

Muse Glimmer-30B

Gemma4-26B-A4B

Qwen3.5-27B

Comments (0)

No comments yet

What now?