Kimi K3.1 Pro

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 104.0B are active per token

Only datacenter GPUs — FP16 needs 5896.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Training Pipeline

1. 1
   pretraining

   Pretraining (inherited Kimi K3 base)

   The HF model card states Kimi K3.1 Pro is "post-trained from Kimi K3", so the pretrained base checkpoint is inherited from the Kimi K3 release rather than trained anew for K3.1 Pro; the sibling Kimi K3 record documents that Kimi K3 itself was pretrained from scratch. No new pretraining evidence for K3.1 Pro appears on this card.

2. 2
   cpt

   Mid-training (continued pretraining)

   The tech blog documents a mid-training stage in K3.1 Pro's training pipeline: cyber-related content was embedded within the general mid-training corpus (inherited from the K3 lineage's broad data), and for the open-weight release Moonshot identified and removed cyber-related data at the mid-training stage while retaining the general-purpose coding tasks needed for usefulness. An internal checkpoint trained with no mid-training data removal scored 90.5% vs the released model's 81.0% on xdev-bench-hard (pass@3).

3. 3
   rl

   Large-scale joint RL post-training

   Per the HF card, K3.1 Pro is "post-trained from Kimi K3 with large-scale joint reinforcement learning across hundreds of downstream domains". The tech blog adds that K3.1 Pro is a post-trained version of Kimi K3 with substantially larger post-training compute, and that to make large-scale joint RL stable the team optimized every layer of the stack (RL framework, long-context inference, training efficiency, sandbox infrastructure, rollout scheduling, environment quality checks). For the open-weight release, cyber-related tasks were also filtered from the RL environment mix.

Training & Evaluation Datasets

Linked Resources

Usage & Social Metrics

View full metric history →