Parameters
35.0B total / 3.0B active
MoE: total / active
Architecture
Hybrid Gated DeltaNet + Gated Attention MoE - Language World Model (qwen3_5_moe)
Released
22.06.2026
License
Apache License 2.0
Input Modalities
Output Modalities
Context (native)
262,144 tokens
Context (extended)
262,144 tokens
About
Qwen-AgentWorld-35B-A3B (Qwen/Qwen-AgentWorld-35B-A3B) is Alibaba's open Language World Model (LWM) - an agent foundation model built as a native world model rather than a post-hoc-adapted general LLM. A single model covers seven unified domains - MCP (tool calling), Search, Terminal, SWE, Android, Web and OS - spanning both text and GUI interaction environments. It generalizes zero-shot to out-of-distribution environments (e.g. OpenClaw), supports controllable perturbations, and can construct fictional training worlds that surpass real-environment training.
Built on Qwen3.5-35B-A3B-Base (35B total, 3B activated; 40 layers in a 10x (3x (Gated DeltaNet -> MoE) + 1x (Gated Attention -> MoE)) hybrid layout with 256 experts, 8 routed + 1 shared per token, hidden 2048), it is trained with a three-stage pipeline: Continual Pre-Training for environment knowledge injection, SFT with next-state-prediction reasoning, and RL with GSPO for simulation fidelity. LWM RL warm-up on single-turn, non-agentic trajectories transfers to multi-turn, tool-calling agentic tasks across 7 benchmarks including 3 entirely out-of-domain. A key disclaimer: no outputs from external API services are included in the training pipeline. Context length is 262,144 tokens; released June 22, 2026 under Apache 2.0. Paper: "Qwen-AgentWorld: Language World Models for General Agents" (arXiv 2606.24597).
Training Data Three-stage pipeline: CPT (environment knowledge injection) → SFT (next-state-prediction reasoning) → RL (GSPO, simulation fidelity). No external API service outputs included in training. Base model: Qwen3.5-35B-A3B-Base.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
AgentWorldBench MCP
general_agent
|
65.00%
|
24.06.2026 |
#8/14
better than 46%
|
|
AgentWorldBench Search
general_agent
|
92.88%
|
24.06.2026 |
#3/14
better than 85%
|
|
AgentWorldBench Terminal
general_agent
|
70.27%
|
24.06.2026 |
#6/14
better than 62%
|
|
AgentWorldBench SWE
general_agent
|
90.79%
|
24.06.2026 |
#3/14
better than 85%
|
|
AgentWorldBench Web
general_agent
|
32.41%
|
24.06.2026 |
#12/14
better than 15%
|
|
AgentWorldBench OS
general_agent
|
69.27%
|
24.06.2026 |
#6/14
better than 62%
|
|
AgentWorldBench Overall
general_agent
|
81.57%
|
24.06.2026 |
#5/14
better than 69%
|
|
AgentWorldBench Android
general_agent
|
61.78%
|
24.06.2026 |
#8/14
better than 46%
|
What is a good AgentWorldBench MCP score? What is a good AgentWorldBench Search score? What is a good AgentWorldBench Terminal score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 70.0 GB | 0.5 GB | 74.0 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 35.0 GB | 0.2 GB | 37.0 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 17.5 GB | 0.1 GB | 18.5 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 17.5 GB | 0.1 GB | 18.5 GB | Which GPUs can run it? Calculate on your GPU |
Only datacenter GPUs — FP16 needs 74.0 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Hybrid Attention (16:2)
- MoE
- 256 experts · top-8 per token
- Layers
- 40
- Hidden size
- 2048
- Context
- 262K tokens
- Parameters
- 35000M
- Active params
- 3000M
Source: Hugging Face config.json · Qwen3_5MoeForConditionalGeneration · exact layer pattern · model repo
248K
Training Pipeline
-
1
cpt
Continual Pre-Training - Environment Knowledge Injection
CPT stage injects environment knowledge into the base model (Qwen3.5-35B-A3B-Base). Environment modeling is the training objective from CPT onward, making Qwen-AgentWorld a native world model rather than a post-hoc adaptation.
-
2
sft
Supervised Fine-Tuning - Next-State-Prediction Reasoning
SFT stage activates next-state-prediction reasoning. The model learns to predict the next environment state given an agent's action and interaction history via long chain-of-thought reasoning.
-
3
rl
Reinforcement Learning - Simulation Fidelity (GSPO)
RL stage using GSPO sharpens simulation fidelity. LWM RL warm-up on single-turn non-agentic trajectories transfers to multi-turn tool-calling agentic tasks across 7 benchmarks.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Environment knowledge corpus (CPT injection) | pretraining | — | — | |
| Next-state-prediction SFT + GSPO RL agent trajectories | rl | — | — |
Linked Resources
Qwen-AgentWorld: Language World Models for General Agents (arXiv:2606.24597)
http://arxiv.org/abs/2606.24597
Qwen-AgentWorld Blog Post
https://qwen.ai/blog?id=qwen-agentworld
QwenLM/Qwen-AgentWorld GitHub Repository
https://github.com/QwenLM/Qwen-AgentWorld
Qwen-AgentWorld HuggingFace Collection
https://huggingface.co/collections/Qwen/qwen-agentworld
Qwen-AgentWorld on ModelScope
https://modelscope.cn/collections/Qwen/Qwen-AgentWorld
Qwen-AgentWorld Interactive Demo
https://qwen.ai/blog?id=qwen-agentworld#interactive-demo-interactive-demo
Trend Analysis
24h Change
-0.9%
7d Change
-2.8%
30d Change
-64.9%
Current
9,617
followers
+0.2%
likes
+0.0%
downloads_all_time
+0.1%
No comments yet
Be the first to share your thoughts!