Qwen-AgentWorld-35B-A3B

The model uses thinking mode by default (<think>...</think>) to reason about environment state transitions before producing the predicted observation.

Key behavior:

• Thinking mode is active by default — the model generates internal reasoning tokens enclosed in <think> tags before the final output.
• This enables the model to reason through complex environment simulation scenarios (e.g., predicting terminal output, web page state, or Android UI changes) before emitting the predicted observation.

Recommended sampling parameters when using thinking mode:

• temperature=0.6
• top_p=0.95
• top_k=20

• Native context length: 262,144 tokens (256K)
• Extended context length: 262,144 tokens (same as native)
• Recommendation: The model leverages extended context for multi-turn environment simulation. If OOM errors occur, consider reducing the context window, but maintain at least 128K tokens for optimal simulation performance.

Architecture note: The Hybrid Gated DeltaNet + Gated Attention architecture with 40 layers supports this context length natively. Rotary Position Embedding (RoPE) dimension is 64.

Three-stage training pipeline:

1. Continual Pre-Training (CPT): Environment knowledge injection. Base model: Qwen3.5-35B-A3B-Base. Environment modeling is the training objective from the CPT stage onward (native world model, not post-hoc adaptation).

2. Supervised Fine-Tuning (SFT): Next-state-prediction reasoning activation. The model learns to predict the next environment state given an agent's action and interaction history.

3. Reinforcement Learning (RL): GSPO (Group Sequence Policy Optimization) sharpens simulation fidelity. LWM RL warm-up on single-turn, non-agentic trajectories transfers to multi-turn, tool-calling agentic tasks.

Key constraint: No outputs from external API services are included in the training pipeline.

Base model: Qwen/Qwen3.5-35B-A3B-Base

Derived models:

• Adapters: 1 model
• Finetunes: 9 models
• Merges: 4 models
• Quantizations: 73 models (llama.cpp, LM Studio, Jan, Ollama)

Collection: Qwen-AgentWorld — 3 items

Spaces using this model: 6

If you find our work helpful, feel free to give us a cite.

@article{zuo2026qwen,
  title={Qwen-agentworld: language world models for general agents},
  author={Zuo, Yuxin and Xiao, Zikai and Sheng, Li and Huang, Fei and Tu, Jianhong and Liu, Yuxuan and Tang, Tianyi and Hu, Xiaomeng and Su, Yang and Lan, Qingfeng and others},
  journal={arXiv preprint arXiv:2606.24597},
  year={2026}
}

Links: 📑 Technical Report | 📖 Blog | 🤗 Hugging Face | 🤖 ModelScope | 💻 GitHub | 🖥️ Demo

AgentWorldBench evaluates language world models by scoring each predicted environment observation on 5 dimensions: Format, Factuality, Consistency, Realism, and Quality.

Setup

## Clone the evaluation repository
git clone https://github.com/QwenLM/Qwen-AgentWorld.git
cd Qwen-AgentWorld

## Download the benchmark
huggingface-cli download Qwen/AgentWorldBench --repo-type dataset --local-dir ./AgentWorldBench

## Install dependencies
pip install openai

Run Evaluation

The evaluation follows a three-step pipeline:

cd eval

## Step 1: Run world model inference
python eval.py infer \
    --data-dir ../AgentWorldBench \
    --model-base-url http://localhost:8000/v1 \
    --model-name Qwen/Qwen-AgentWorld-35B-A3B \
    --output-dir ./results

## Step 2: Run LLM judge scoring
export OPENAI_API_KEY="your-api-key"
python eval.py judge \
    --predictions ./results/predictions.jsonl \
    --judge-base-url https://api.openai.com/v1 \
    --judge-model gpt-5.2-2025-12-11 \
    --output-dir ./results

## Step 3: Aggregate and display scores
python eval.py score --predictions ./results/judged.jsonl

1. Sampling Parameters: We recommend temperature=0.6, top_p=0.95, top_k=20 for world model inference. The model uses thinking mode by default (<think>...</think>) to reason about environment state transitions before producing the predicted observation.

2. Adequate Output Length: We recommend an output length of 32,768 tokens for most queries. For long, multi-step trajectories, you may increase the max output length to accommodate detailed environment observations.

3. Domain-Specific System Prompts: For optimal simulation fidelity, use the domain-specific system prompts provided in the prompts/ directory of the GitHub repository.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

## Terminal domain example
messages = [
    {
        "role": "system",
        "content": "You are a language world model simulating a Linux terminal environment. "
                   "Given the user's command, predict the terminal output."
    },
    {
        "role": "user",
        "content": "Action: execute_bash\nCommand: ls -la /home/user/project/"
    }
]

response = client.chat.completions.create(
    model="Qwen/Qwen-AgentWorld-35B-A3B",
    messages=messages,
    max_tokens=32768,
    temperature=0.6,
)
print(response.choices[0].message.content)

Domain-specific world model system prompt templates are provided in prompts/ of the GitHub repository for all 7 domains. Each domain folder contains a system_prompt.txt (world model system prompt) and a judge_system_prompt.txt (evaluation prompt).

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen-AgentWorld-35B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "You are a language world model simulating a Linux terminal environment. "
                   "Given the user's command, predict the terminal output."
    },
    {
        "role": "user",
        "content": "Action: execute_bash\nCommand: ls -la /home/user/project/"
    }
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
print(response)

vLLM is a high-throughput and memory-efficient inference engine for LLMs.

vllm serve Qwen/Qwen-AgentWorld-35B-A3B \
    --port 8000 \
    --tensor-parallel-size 4 \
    --max-model-len 262144 \
    --reasoning-parser qwen3 \
    --language-model-only \
    --trust-remote-code

The --language-model-only flag is required because the model architecture includes visual component definitions but the checkpoint only contains language model weights. Without this flag, vLLM will attempt to initialize visual modules and fail.

An OpenAI-compatible API will be available at http://localhost:8000/v1.

SGLang is a fast serving framework for large language models.

python -m sglang.launch_server \
    --model-path Qwen/Qwen-AgentWorld-35B-A3B \
    --port 8000 \
    --tp-size 4 \
    --context-length 262144 \
    --reasoning-parser qwen3

An OpenAI-compatible API will be available at http://localhost:8000/v1.

Five-dimensional rubric mean per domain, normalized to 0-100 scale.

Source: HuggingFace Model Card

• Type: Causal Language Model (Language World Model)
• Base Model: Qwen3.5-35B-A3B-Base
• Training Stage: Continual Pre-Training (CPT) → Supervised Fine-Tuning (SFT) → Reinforcement Learning (RL, GSPO)
• Number of Parameters: 35B in total and 3B activated
• Hidden Dimension: 2048
• Token Embedding: 248320 (Padded)
• Number of Layers: 40
• Hidden Layout: 10 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))

Gated DeltaNet:

• Number of Linear Attention Heads: 32 for V and 16 for QK
• Head Dimension: 128

Gated Attention:

• Number of Attention Heads: 16 for Q and 2 for KV
• Head Dimension: 256
• Rotary Position Embedding Dimension: 64

Mixture of Experts:

• Number of Experts: 256

• Number of Activated Experts: 8 Routed + 1 Shared

• Expert Intermediate Dimension: 512

• Context Length: 262,144 tokens

• Disclaimer: No outputs from external API services are included in the training pipeline.

• Seven Unified Domains. A single model covers MCP (tool calling), Search, Terminal, SWE (software engineering), Android, Web, and OS — spanning both text and GUI interaction environments.
• Native World Model. Environment modeling from CPT onward, not post-hoc adaptation on a general-purpose LLM.
• Generalizable, Scalable & Controllable Simulator. Zero-shot generalization to OOD environments (e.g., OpenClaw); controllable perturbations and fictional-world construction surpass real-environment training.
• Agent Foundation Model. LWM RL warm-up on single-turn, non-agentic trajectories transfers to multi-turn, tool-calling agentic tasks across 7 benchmarks, including 3 entirely out-of-domain.

Qwen-AgentWorld is the first language world model to cover seven agent interaction domains within a single model. It simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state given an agent's action and interaction history.

Trained through a three-stage pipeline — CPT injects environment knowledge, SFT activates next-state-prediction reasoning, RL sharpens simulation fidelity — Qwen-AgentWorld is a native world model: environment modeling is the training objective from the CPT stage onward, not a post-hoc add-on.

This repository contains the model weights and configuration files for Qwen-AgentWorld-35B-A3B, a native language world model trained for agentic environment simulation. These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, etc.