Kimi K2.5

About

Kimi K2.5 (moonshotai/Kimi-K2.5) is Moonshot AI's open-source native multimodal agentic model, built through continual pretraining on approximately 15 trillion mixed visual and text tokens atop Kimi-K2-Base. It integrates vision and language understanding with advanced agentic capabilities, instant and thinking modes, and conversational and agentic paradigms.

The architecture is a trillion-parameter Mixture-of-Experts with 32B activated parameters: 61 layers (1 dense), hidden size 7168, 384 routed experts with 8 selected per token plus 1 shared expert (MoE expert dim 2048), attention based on Multi-head Latent Attention (MLA) (kv-lora rank 512, q-lora rank 1536, 128 nope + 64 rope head dims, SwiGLU activation), 160K vocabulary and a 256K context window. Vision comes from MoonViT, a 400M-parameter encoder (27 layers, patch 14, patchmerger projector, spatial-temporal video attention). The checkpoint ships with native INT4 quantization (compressed-tensors).

Key features: native multimodality (visual knowledge, cross-modal reasoning, agentic tool use grounded in visual inputs), coding with vision (generates code from UI designs/video workflows and orchestrates visual data tools), and Agent Swarm - a self-directed, coordinated swarm-like execution scheme that decomposes complex tasks into parallel sub-tasks executed by dynamically instantiated, domain-specific agents.

Training Data Continual pretraining on approx. 15 trillion mixed visual and text tokens atop Kimi-K2-Base; vision-language agentic training (card Model Introduction).