Parameters
31.2B / 4.0B active
MoE total / active
Context
262,144 tokens
native · extended 524,288
Modalities
text text
input → output
Benchmarks
#15/131
better than 89% of all scored models
Size class
#2/22
better than 91% of similar-size models
24h trend
+0.5%
Interest delta, 24h
License
Apache License 2.0
About
Xing4.0-29B-A4B (XingChen-AGI/Xing4.0-29B-A4B) is the next-generation open agentic large language model of the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. and open-sourced on Hugging Face and ModelScope on September 16, 2026 under Apache 2.0. It is a sparse Mixture-of-Experts model with 29B total parameters and only 4B activated per token — the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework — and is deeply optimized for complex engineering (agentic coding) tasks.
Its agent-oriented architecture combines mHC (multi-head hyper-connections) with MLA (Multi-head Latent Attention) and MTP (Multi-Token Prediction): 64 routed experts with 4 active per token plus 1 shared expert (sigmoid scoring, noaux_tc Top-K routing), 2 first-k dense layers, 40 layers, hidden size 3584, dense FFN intermediate size 9216, expert intermediate size 1024, and a 131,072-token vocabulary. MLA compresses the KV cache into a low-rank latent (kv_lora_rank 512, q_lora_rank 768, qk_rope_head_dim 64, qk_nope_head_dim 128, v_head_dim 128); MTP adds one nextn predict layer (num_nextn_predict_layers 1) to accelerate decoding. The mHC hyper-connection mechanism (hc_mult 4, hc_sinkhorn_iters 20) multiplies residual-stream connectivity for stable training and better long-context coherence. A native 256K (262,144-token) context is natively supported and extensible to 512K via YaRN RoPE scaling (factor 64 from original_max_position_embeddings 4096, beta_fast 32 / beta_slow 1).
Training was co-optimized with Ascend NPU (Ascend 910C clusters, MindSpore/MindFormers): fine-grained MoE communication optimization, selective recomputation, DVM automatic graph-operator fusion, and Ascend C mHC fused operators lifted overall training throughput by approximately 96% over out-of-the-box performance. Fine-tuning is supported through LLaMA-Factory and MindFormers; inference through vLLM, SGLang, and KTransformers; agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes are explicitly adapted. An OpenAI-compatible API with enable_thinking toggle is documented; recommended sampling is temperature 1.0 / top_p 0.95 for reasoning and 0.8 / 0.95 for coding/agent tasks (repetition_penalty 1.05).
On benchmarks it scores 75.0 on SWE-bench Verified (SWE-agent harness, 210K context), 57.5 on Terminal-Bench 2.1 (terminus-2 harness, avg of 3 runs), 76.55 on Claw-Eval (avg of 3 runs), 64.63 on Tau3-Bench (avg pass^1 over 4 runs), 90.0 on AIME2026 (avg of 5 runs), 69.67 on IFBench, 61.0 on AA-LCR, 66.0 on SWE-bench Multilingual, and 60.8 on DeepresearchBII (OpenCode harness, Exa MCP enabled) — surpassing Gemma4-26B-A4B on most agentic benchmarks at roughly the same active-parameter budget. Weights are distributed in BF16 (~31.2B safetensors parameters, including F32 mHC scaling constants); 4-bit quantization runs locally on an RTX 3090/4090 (~15GB VRAM); community finetunes and quantized GGUF builds exist.
Training Data First model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework (Ascend 910C clusters, MindFormers); deeply optimized for complex engineering tasks with ~96% training throughput improvement via fine-grained MoE communication optimization, selective recomputation, DVM graph-operator fusion, and Ascend C mHC fused operators.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
Claw-Eval Avg
coding_agent
|
92.66%
|
16.09.2026 |
#4/40
better than 92%
|
|
SWE-bench Multilingual
coding_agent
|
72.19%
|
16.09.2026 |
#34/49
better than 31%
|
|
Terminal-Bench 2.1 (Terminus-2)
coding_agent
|
53.83%
|
16.09.2026 |
#34/50
better than 33%
|
|
DeepresearchBII
general_agent
|
100.00%
|
16.09.2026 |
#1/1
only tracked model
|
|
SWE-bench Verified
coding_agent
|
85.55%
|
16.09.2026 |
#26/60
better than 56%
|
|
TAU3-Bench
general_agent
|
93.69%
|
16.09.2026 |
#6/22
better than 76%
|
|
IFBench
instruction_following
|
76.37%
|
16.09.2026 |
#29/44
better than 35%
|
|
AIME 26
stem_reasoning
|
88.27%
|
16.09.2026 |
#30/45
better than 34%
|
|
AA-LCR
long_context
|
72.62%
|
16.09.2026 |
#32/47
better than 33%
|
What is a good Claw-Eval Avg score? What is a good SWE-bench Multilingual score? What is a good Terminal-Bench 2.1 (Terminus-2) score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 4.0B are active per token
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 62.4 GB | 0.6 GB | 66.2 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 31.2 GB | 0.3 GB | 33.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 15.6 GB | 0.2 GB | 16.5 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 15.6 GB | 0.2 GB | 16.5 GB | Which GPUs can run it? Calculate on your GPU |
Only datacenter GPUs — FP16 needs 66.2 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.
Consumer GPUs that fit this model with a single card (INT4):
Architecture
decoder-only transformer
- Parameters
- 31.2B
- Active params
- 4B
- Layers
- 40 40× Full Attention
- Attention
- MLA Multi-head Latent Attention
- Experts
- 64 top-4 + 1 shared
- Hidden
- 3584
- Vocabulary
- 131K
- Context
- 262K → 524K extended
Source: Hugging Face config.json · Xing4_0ForCausalLM · model repo
Linked Resources
Training Report of TeleChat3-MoE
https://arxiv.org/abs/2512.24157
Technical Report of TeleChat2, TeleChat2.5 and T1
https://arxiv.org/abs/2507.18013
XingChen-AGI/Xing4.0-29B-A4B
https://github.com/XingChen-AGI/Xing4.0-29B-A4B
Tele-AI/TeleChat3
https://github.com/Tele-AI/TeleChat3
vllm-project/vllm
https://github.com/vllm-project/vllm
sgl-project/sglang
https://github.com/sgl-project/sglang
kvcache-ai/ktransformers
https://github.com/kvcache-ai/ktransformers
claw-eval/claw-eval
https://github.com/claw-eval/claw-eval
sierra-research/tau-bench
https://github.com/sierra-research/tau-bench
Citation (BibTeX)
https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B#citation
Xing4.0-29B-A4B Collection
https://huggingface.co/collections/XingChen-AGI/xing40-29b-a4b
Trend Analysis
24h Change
+0.5%
7d Change
+1.9%
Current
372
downloads_all_time
+0.3%
downloads
+0.2%
likes
+0.0%
No comments yet
Be the first to share your thoughts!