Xing4.0-29B-A4B

China Telecom AI

Sparse MoE Transformer with mHC + MLA + MTP

Open Weights Commercial Use Multimodal BF16,F32 Xing4.0 English Chinese

Parameters

31.2B / 4.0B active

MoE total / active

Context

262,144 tokens

native · extended 524,288

Modalities

text text

input → output

Benchmarks

#15/131

better than 89% of all scored models

Size class

#2/22

better than 91% of similar-size models

24h trend

+0.5%

Interest delta, 24h

License

Apache License 2.0

About

Xing4.0-29B-A4B (XingChen-AGI/Xing4.0-29B-A4B) is the next-generation open agentic large language model of the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. and open-sourced on Hugging Face and ModelScope on September 16, 2026 under Apache 2.0. It is a sparse Mixture-of-Experts model with 29B total parameters and only 4B activated per token — the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework — and is deeply optimized for complex engineering (agentic coding) tasks.

Its agent-oriented architecture combines mHC (multi-head hyper-connections) with MLA (Multi-head Latent Attention) and MTP (Multi-Token Prediction): 64 routed experts with 4 active per token plus 1 shared expert (sigmoid scoring, noaux_tc Top-K routing), 2 first-k dense layers, 40 layers, hidden size 3584, dense FFN intermediate size 9216, expert intermediate size 1024, and a 131,072-token vocabulary. MLA compresses the KV cache into a low-rank latent (kv_lora_rank 512, q_lora_rank 768, qk_rope_head_dim 64, qk_nope_head_dim 128, v_head_dim 128); MTP adds one nextn predict layer (num_nextn_predict_layers 1) to accelerate decoding. The mHC hyper-connection mechanism (hc_mult 4, hc_sinkhorn_iters 20) multiplies residual-stream connectivity for stable training and better long-context coherence. A native 256K (262,144-token) context is natively supported and extensible to 512K via YaRN RoPE scaling (factor 64 from original_max_position_embeddings 4096, beta_fast 32 / beta_slow 1).

Training was co-optimized with Ascend NPU (Ascend 910C clusters, MindSpore/MindFormers): fine-grained MoE communication optimization, selective recomputation, DVM automatic graph-operator fusion, and Ascend C mHC fused operators lifted overall training throughput by approximately 96% over out-of-the-box performance. Fine-tuning is supported through LLaMA-Factory and MindFormers; inference through vLLM, SGLang, and KTransformers; agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes are explicitly adapted. An OpenAI-compatible API with enable_thinking toggle is documented; recommended sampling is temperature 1.0 / top_p 0.95 for reasoning and 0.8 / 0.95 for coding/agent tasks (repetition_penalty 1.05).

On benchmarks it scores 75.0 on SWE-bench Verified (SWE-agent harness, 210K context), 57.5 on Terminal-Bench 2.1 (terminus-2 harness, avg of 3 runs), 76.55 on Claw-Eval (avg of 3 runs), 64.63 on Tau3-Bench (avg pass^1 over 4 runs), 90.0 on AIME2026 (avg of 5 runs), 69.67 on IFBench, 61.0 on AA-LCR, 66.0 on SWE-bench Multilingual, and 60.8 on DeepresearchBII (OpenCode harness, Exa MCP enabled) — surpassing Gemma4-26B-A4B on most agentic benchmarks at roughly the same active-parameter budget. Weights are distributed in BF16 (~31.2B safetensors parameters, including F32 mHC scaling constants); 4-bit quantization runs locally on an RTX 3090/4090 (~15GB VRAM); community finetunes and quantized GGUF builds exist.

Training Data First model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework (Ascend 910C clusters, MindFormers); deeply optimized for complex engineering tasks with ~96% training throughput improvement via fine-grained MoE communication optimization, selective recomputation, DVM graph-operator fusion, and Ascend C mHC fused operators.

Benchmark Scores

Benchmark Score Date Standing
Claw-Eval Avg
coding_agent
92.66%
16.09.2026
#4/40
better than 92%
SWE-bench Multilingual
coding_agent
72.19%
16.09.2026
#34/49
better than 31%
Terminal-Bench 2.1 (Terminus-2)
coding_agent
53.83%
16.09.2026
#34/50
better than 33%
DeepresearchBII
general_agent
100.00%
16.09.2026
#1/1
only tracked model
SWE-bench Verified
coding_agent
85.55%
16.09.2026
#26/60
better than 56%
TAU3-Bench
general_agent
93.69%
16.09.2026
#6/22
better than 76%
IFBench
instruction_following
76.37%
16.09.2026
#29/44
better than 35%
AIME 26
stem_reasoning
88.27%
16.09.2026
#30/45
better than 34%
AA-LCR
long_context
72.62%
16.09.2026
#32/47
better than 33%

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 4.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 62.4 GB 0.6 GB 66.2 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 31.2 GB 0.3 GB 33.1 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 15.6 GB 0.2 GB 16.5 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 15.6 GB 0.2 GB 16.5 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 66.2 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

Architecture

Decoder Block input Embedding vocab 131K · d 3584 Full Attention MLA · 32 heads ×40 MoE FFN 64 experts · top-4 · +1 shared · dᴻ 1024 MTP Head ×1 speculative layer Final Norm LM Head vocab 131K output

decoder-only transformer

MoE
Parameters
31.2B
Active params
4B
Layers
40 40× Full Attention
Attention
MLA Multi-head Latent Attention
Experts
64 top-4 + 1 shared
Hidden
3584
Vocabulary
131K
Context
262K → 524K extended
silu BF16 RoPE θ 10K MTP ×1 First 2 dense top-k routing with sigmoid…

Source: Hugging Face config.json · Xing4_0ForCausalLM · model repo

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Xing4.0-29B-A4B — here is where to go next.