Qwen3.5-4B

Qwen

Parameters

4.0B

Architecture

Causal Language Model with Vision Encoder - Hybrid Gated Delta Networks + Gated Attention (dense FFN)

Released

27.02.2026

License

Apache License 2.0

Open Weights Commercial Use Multimodal BF16 Qwen3.5 en zh 201 languages

Input Modalities

text image video

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

1,010,000 tokens

Openness Index Score 100.0/100

About

Qwen3.5-4B (Qwen/Qwen3.5-4B) is the small dense member of Alibaba's Qwen3.5 generation - a unified vision-language foundation model with 4B parameters handling text, image and video input in 201 languages and dialects, trained with Early fusion (multimodal pre-training) on multimodal tokens and post-trained with RL scaled across million-agent RL environments.

Its 32-layer hybrid layout repeats 8x (3x (Gated DeltaNet -> FFN) + 1x (Gated Attention -> FFN)): 24 linear-attention layers (Gated DeltaNet with 32 V / 16 QK heads, head dim 128) and 8 full-attention layers (Gated Attention with 16 Q / 4 KV heads, head dim 256, RoPE dim 64), with hidden size 2560, dense FFN 9216 (no MoE), 248K vocabulary tied to token embeddings, and Multi-Token Prediction (MTP) trained with multi-steps. Context is 262,144 tokens natively, extensible up to 1,010,000 tokens.

As a Qwen3.5 family member it delivers the generation's agentic capabilities (native function calling, thinking-mode control via enable_thinking) in a single-GPU-sized package, outperforming much larger Qwen3-VL models across reasoning, coding, agents and visual understanding. Released February 27, 2026 under Apache 2.0.

Training Data Pre-trained on multimodal tokens with early fusion. Post-trained with RL across million-agent environments. 201 languages and dialects.

Benchmark Scores

Benchmark	Score	Date	Standing
LVBench video_understanding	71.19%	01.02.2026	#8/12 better than 36%
ScreenSpot Pro general_agent	60.40%	01.02.2026	#9/11 better than 20%
OSWorld-Verified general_agent	35.60	01.02.2026	#25/25 better than 0%
AndroidWorld general_agent	58.60	01.02.2026	#13/13 better than 0%
TIR-Bench vision_language	40.62%	01.02.2026	#4/8 better than 57%
V-Star vision_language	64.57%	01.02.2026	#15/20 better than 26%
SLAKE vision_language	79.55%	01.02.2026	#4/8 better than 57%
PMC-VQA vision_language	71.11%	01.02.2026	#5/8 better than 43%
MedXpertQA-MM vision_language	44.29%	01.02.2026	#10/13 better than 25%
ParseBench Mean document_understanding	35.40	01.02.2026	#3/3 better than 0%
ParseBench Text Content document_understanding	88.90	01.02.2026	#3/3 better than 0%
ParseBench Text Formatting document_understanding	57.80	01.02.2026	#3/3 better than 0%
Ifstruct V1 instruction_following	36.25	01.02.2026	#3/3 better than 0%
MMLU-ProX multilingual	3.87%	01.02.2026	#18/19 better than 6%
NOVA-63 multilingual	35.82%	01.02.2026	#5/6 better than 20%
INCLUDE multilingual	71.00	01.02.2026	#11/11 better than 0%
OmniDocBench 1.5 document_understanding	94.09%	01.02.2026	#14/26 better than 48%
Global PIQA multilingual	78.90	01.02.2026	#6/6 better than 0%
CharXiv (RQ) document_understanding	18.97%	01.02.2026	#22/34 better than 36%
PolyMATH multilingual	25.28%	01.02.2026	#8/11 better than 30%
MMLongBench-Doc document_understanding	39.39%	01.02.2026	#6/7 better than 17%
WMT24++ (en→xx) multilingual	59.68%	01.02.2026	#13/17 better than 25%
CC-OCR document_understanding	61.87%	01.02.2026	#10/14 better than 31%
MAXIFE multilingual	78.00	01.02.2026	#6/6 better than 0%
MMMU vision_language	83.84%	01.02.2026	#14/19 better than 28%
MathVision vision_language	55.21%	01.02.2026	#16/27 better than 35%
CountBench spatial_intelligence	80.77%	01.02.2026	#6/13 better than 58%
We-Math vision_language	100.00%	01.02.2026	#1/2 better than 100%
RefSpatialBench spatial_intelligence	77.29%	01.02.2026	#8/12 better than 36%
ZEROBench vision_language	3.00	01.02.2026	#13/14 better than 0%
LingoQA spatial_intelligence	89.02%	01.02.2026	#4/8 better than 57%
ZEROBench_sub vision_language	2.94%	01.02.2026	#8/11 better than 10%
Hypersim spatial_intelligence	71.43%	01.02.2026	#4/6 better than 40%
VlmsAreBlind vision_language	79.25%	01.02.2026	#6/11 better than 50%
Nuscene spatial_intelligence	9.90	01.02.2026	#6/6 better than 0%
RealWorldQA vision_language	79.07%	01.02.2026	#12/23 better than 50%
MMStar vision_language	80.45%	01.02.2026	#9/16 better than 47%
MMLU-Pro knowledge	61.61%	01.02.2026	#29/48 better than 40%
MMLU-Redux knowledge	71.43%	01.02.2026	#20/30 better than 34%
SuperGPQA knowledge	60.14%	01.02.2026	#17/28 better than 41%
GPQA Diamond stem_reasoning	65.78%	01.02.2026	#63/80 better than 22%
IFEval instruction_following	88.78%	01.02.2026	#14/26 better than 44%
Multi-Challenge instruction_following	30.09%	01.02.2026	#17/18 better than 6%
LongBench v2 long_context	56.79%	01.02.2026	#11/22 better than 52%
HMMT Feb 25 stem_reasoning	53.78%	01.02.2026	#12/17 better than 31%
HMMT Nov 25 stem_reasoning	76.80	01.02.2026	#20/20 better than 0%
LiveCodeBench v6 stem_reasoning	48.57%	01.02.2026	#41/49 better than 17%
OJBench stem_reasoning	31.57%	01.02.2026	#13/22 better than 43%
BFCL-V4 general_agent	47.63%	01.02.2026	#20/27 better than 27%
TAU2-Bench general_agent	78.77%	01.02.2026	#4/21 better than 85%
VITA-Bench general_agent	45.60%	01.02.2026	#9/16 better than 47%
DeepPlanning general_agent	2.97%	01.02.2026	#8/10 better than 22%
MMMLU multilingual	37.99%	01.02.2026	#17/20 better than 16%
HallusionBench vision_language	41.80%	01.02.2026	#11/13 better than 17%
MMMU-Pro vision_language	69.15%	01.02.2026	#29/35 better than 18%
TAU3-Bench general_agent	8.12%	—	#17/21 better than 20%
Workspace Bench	57.87%	—	#3/5 better than 50%
BrowseComp general_agent	12.58%	—	#37/45 better than 18%
SWE-bench Pro coding_agent	32.78%	—	#65/77 better than 16%
SWE-bench Verified coding_agent	44.04%	—	#51/60 better than 15%
SWE-bench Multilingual coding_agent	26.86%	—	#44/48 better than 9%
Gaokao 2026	90.32%	—	#4/8 better than 57%
AIME 26 stem_reasoning	79.34%	—	#36/45 better than 20%
HMMT Feb 26 stem_reasoning	63.73%	—	#34/44 better than 23%
IMOAnswerBench stem_reasoning	65.20%	—	#25/30 better than 17%
Humanity's Last Exam stem_reasoning	12.31%	—	#61/71 better than 14%
GPQA reasoning	83.92%	—	#6/14 better than 62%
Multi-IF instruction_following	68.24%	—	#6/13 better than 58%
Terminal Bench 2.1 coding_agent	28.16%	—	#35/44 better than 21%
BrowseComp-zh general_agent	54.50%	—	#6/13 better than 58%
Gaia2 general_agent	83.76%	—	#2/12 better than 91%
NoLiMa long_context	63.61%	—	#2/9 better than 88%
GDPVal-AA v2 general_agent	0.64%	—	#31/38 better than 19%
Claw-Eval Avg coding_agent	61.36%	—	#31/40 better than 23%
WildClawBench coding_agent	18.38%	—	#14/20 better than 32%
LCB-Pro 25Q2 (Easy) stem_reasoning	83.19%	—	#2/9 better than 88%
LCB-Pro 25Q2 (Medium) stem_reasoning	40.00%	—	#2/9 better than 88%
MATH-500 math	100.00%	—	#1/11 better than 100%
C-Eval knowledge	26.61%	01.02.2026	#12/15 better than 21%
AI2D_TEST document_understanding	87.71%	01.02.2026	#5/15 better than 71%
MCPMark general_agent	11.06%	—	#14/15 better than 7%
DynaMath vision_language	50.56%	01.02.2026	#6/14 better than 62%
QwenClawBench coding_agent	66.67%	—	#10/17 better than 44%
LongBenchPro long_context	89.17%	—	#2/10 better than 89%
AIME 2025 stem_reasoning	69.89%	—	#15/24 better than 39%
SciCode	34.29%	—	#21/27 better than 23%
MCP-Atlas general_agent	36.81%	—	#48/54 better than 11%
RefCOCO (avg) spatial_intelligence	76.78%	01.02.2026	#10/11 better than 10%
MMBench EN-DEV v1.1 vision_language	82.22%	01.02.2026	#9/15 better than 43%
MMVU video_understanding	41.10%	01.02.2026	#9/10 better than 11%
VideoMME (w sub.) video_understanding	69.05%	01.02.2026	#13/17 better than 19%
SimpleVQA vision_language	19.03%	01.02.2026	#15/16 better than 7%
MLVU video_understanding	71.88%	01.02.2026	#10/13 better than 25%
ERQA spatial_intelligence	45.86%	01.02.2026	#11/21 better than 45%
OCRBench document_understanding	1.15%	01.02.2026	#10/12 better than 18%
IFBench instruction_following	59.35%	01.02.2026	#35/42 better than 17%
VideoMMMU video_understanding	63.97%	01.02.2026	#18/19 better than 6%
AA-LCR long_context	67.86%	01.02.2026	#31/42 better than 27%
MathVista (mini) vision_language	89.33%	01.02.2026	#8/14 better than 46%
VideoMME (w/o sub.) video_understanding	73.28%	01.02.2026	#8/11 better than 30%
EmbSpatialBench spatial_intelligence	76.51%	01.02.2026	#9/12 better than 27%
MVBench video_understanding	67.30%	01.02.2026	#9/12 better than 27%
BabyVision vision_language	4.17%	01.02.2026	#22/24 better than 9%

Showing 10 of 103 results

What is a good LVBench score? What is a good ScreenSpot Pro score? What is a good OSWorld-Verified score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context

Quantization	Weights	KV Cache	Total VRAM	On your hardware
FP16 — 16-bit	8.0 GB	0.6 GB	9.0 GB	Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit	4.0 GB	0.3 GB	4.5 GB	Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit	2.0 GB	0.2 GB	2.3 GB	Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit	2.0 GB	0.2 GB	2.3 GB	Which GPUs can run it? Calculate on your GPU

Consumer GPUs that fit this model with a single card (INT4):

Intel Arc A310 · 4GB GeForce GTX 1650 GDDR6 · 4GB GeForce GTX 1630 · 4GB AMD Radeon RX 6400 · 4GB GeForce GTX 1650 Super · 4GB GeForce GTX 1650 · 4GB AMD Radeon RX 6500 XT · 4GB GeForce GTX 1660 Ti · 6GB GeForce GTX 1660 Super · 6GB GeForce RTX 3050 6GB · 6GB Intel Arc A380 · 6GB GeForce GTX 1660 · 6GB

Architecture

Attention: Hybrid Attention (16:4)
Layers: 32
Hidden size: 2560
Context: 262K tokens
Parameters: 4000M

Source: Hugging Face config.json · Qwen3_5ForConditionalGeneration · exact layer pattern · model repo

Type: Hybrid Gated Delta Networks + Gated Attention (dense)

Attention: Gated Attention with RoPE (16 Q heads, 4 KV heads, head_dim=256, RoPE dim=64) + Gated DeltaNet linear attention (32 V heads, 16 QK heads, head_dim=128)

Decoder: Causal Language Model

Routing: sparse

Lm Output

248K

Lm Output Note

tied to token embedding

Token Embedding

248K

Token Embedding Note

padded

Training Pipeline

1
pretraining
Pre-training with Multimodal Early Fusion

Pre-training on multimodal tokens with early fusion. Near-100% multimodal training efficiency compared to text-only training. 201 languages and dialects supported.
2
rl
Reinforcement Learning Post-training

Post-training with reinforcement learning scaled across million-agent environments with progressively complex task distributions for robust real-world adaptability. Asynchronous RL frameworks supporting massive-scale agent scaffolds and environment orchestration.
3
other
Post-training (Thinking + Instruct Modes)

Post-training stage producing the final model weights compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers. Includes thinking mode and instruct mode capabilities.

Training & Evaluation Datasets

Name	Role	Size	Modalities	Collection
Early-fusion multimodal token corpus (text + image)	pretraining	—		—
Million-agent RL task environments	rl	—		—

Linked Resources

blog

Qwen3.5 Blog Post

https://qwen.ai/blog?id=qwen3.5

github

Qwen-Agent

https://github.com/QwenLM/Qwen-Agent

github

Qwen Code

https://github.com/QwenLM/qwen-code

citation

Qwen3.5: Towards Native Multimodal Agents

https://qwen.ai/blog?id=qwen3.5

demo

Qwen Chat

https://chat.qwen.ai/

collection

Qwen3.5 Collection

https://huggingface.co/collections/Qwen/qwen35

Trend Analysis

24h Change

+0.5%

7d Change

+2.4%

30d Change

+13.3%

Current

1,226

huggingface

followers

+0.2%

huggingface

downloads

-1.2%

ollama

downloads

+0.4%

huggingface

downloads_all_time

+0.2%

View raw metric history →

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Qwen3.5-4B — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON

Qwen3.5-4B

About

Benchmark Scores

Hardware Requirements

Architecture

Training Pipeline

Pre-training with Multimodal Early Fusion

Reinforcement Learning Post-training

Post-training (Thinking + Instruct Modes)

Training & Evaluation Datasets

Linked Resources

Trend Analysis

Related Models

Qwen3.5-2B

Clef-Flash

Qwen3.5-27B

Qwen3.5-35B-A3B

Comments (0)

No comments yet

What now?