Kimi K3

Moonshot AI

Parameters

2.8T total / 104.0B active

MoE: total / active

Architecture

Mixture-of-Experts (MoE)

Released

13.06.2026

License

Kimi K3 License

Open Weights Commercial Use Multimodal MXFP4/MXFP8 Kimi English Chinese

Input Modalities

text image video

Output Modalities

text

Context (native)

1,048,576 tokens

Context (extended)

1,048,576 tokens

Openness Index Score 70.0/100

About

Kimi K3 (moonshotai/Kimi-K3, released June 13, 2026 under the Kimi K3 License) is Moonshot AI's open-weight, native multimodal agentic model and its most capable model to date - a 2.8T-parameter MoE model (104B activated per token, 93 layers: 1 dense + 69 KDA + 24 Gated MLA attention layers; activates 16 of 896 experts), the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work and reasoning.

It is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), and scales up MoE sparsity with a Stable LatentMoE framework yielding an approximate 2.5x improvement in overall scaling efficiency over Kimi K2. Kimi K3 understands text, images and video in the same model, with a 1M-token context window, a MoonViT-V2 vision encoder (401M parameters), SiTU-GLU activation, and native MXFP4 quantization (quantization-aware training from the SFT stage onward: MXFP4 weights / MXFP8 activations for broad hardware compatibility). It sustains long engineering sessions with minimal oversight - navigating massive repositories and orchestrating terminal tools from GPU kernels and compilers to vision-in-the-loop game dev, CAD and chip design - and advances end-to-end agentic knowledge work with interactive visualizations and deep research. Full weights are open for research, deployment and further innovation.

Training Data Quantization-aware training from SFT stage onward using MXFP4 weights with MXFP8 activations

Benchmark Scores

Benchmark Score Date Standing
ExploitGym (6h)
cybersecurity
18.11%
28.08.2026
#5/8
better than 43%
GPQA Diamond
stem_reasoning
98.49%
18.08.2026
#6/80
better than 94%
CritPt (no tools)
stem_reasoning
71.92%
18.08.2026
#4/21
better than 85%
FrontierSWE
coding_agent
88.18%
18.08.2026
#2/12
better than 91%
MLS-Bench-Lite
coding_agent
93.10%
18.08.2026
#2/9
better than 88%
PostTrainBench
coding_agent
80.20%
18.08.2026
#8/22
better than 67%
Terminal Bench 2.1
coding_agent
97.45%
18.08.2026
#3/44
better than 95%
SciCode (subtask)
stem_reasoning
99.03%
18.08.2026
#2/15
better than 93%
WildClawBench
coding_agent
73.53%
18.08.2026
#5/20
better than 79%
DeepSearch QA
general_agent
100.00%
18.08.2026
#1/9
better than 100%

Showing 10 of 78 results

What is a good ExploitGym (6h) score? What is a good GPQA Diamond score? What is a good CritPt (no tools) score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 104.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 5600.0 GB 16.1 GB 5896.1 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 2800.0 GB 8.1 GB 2948.1 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 1400.0 GB 4.0 GB 1474.0 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 1400.0 GB 4.0 GB 1474.0 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 5896.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block input Embedding vocab 164K · d 7168 Full Attention MLA · 96 heads ×93 MoE FFN 896 experts · +2 shared · dᴻ 3072 Final Norm LM Head vocab 164K output
Attention
Multi-head Latent Attention
MoE
896 experts
Layers
93
Hidden size
7168
Context
1M tokens
Parameters
2800000M
Active params
104000M

Source: Hugging Face config.json · KimiK3ForConditionalGeneration · model repo

Type: Mixture-of-Experts (MoE)
Attention: KDA & Gated MLA
Decoder: Kimi Delta Attention (KDA) + Attention Residuals (AttnRes)
MoE: yes (896 experts)
Routing: Stable LatentMoE - activates 16 out of 896 experts with 2 shared experts

Training Pipeline

  1. 1
    pretraining

    Pretraining

    2.8T parameter MoE model with Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). Uses Stable LatentMoE framework activating 16 out of 896 experts. Native MXFP4 quantization-aware training from SFT stage onward.

  2. 2
    sft

    Supervised Fine-Tuning (SFT)

    Quantization-aware training applied from SFT stage onward using MXFP4 weights with MXFP8 activations for broad hardware compatibility.

  3. 3
    rl

    Reinforcement Learning (RL)

    Trained in preserved thinking history mode. Thinking effort supports low, high, and max settings.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
DeepSWE evaluation — —
GPQA Diamond evaluation — —
WildClawBench evaluation — —
MDPBench evaluation — —

Linked Resources

tech_report

Kimi K3 Tech Report

https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf

 blog

Kimi K3 Tech Blog

https://www.kimi.com/blog/kimi-k3

 github

Kimi-K3 GitHub Repository

https://github.com/MoonshotAI/Kimi-K3

 api_docs

Kimi K3 Quickstart Guide

https://platform.kimi.ai/docs/guide/kimi-k3-quickstart

 api_docs

Thinking Effort Guide

https://platform.kimi.ai/docs/guide/use-thinking-effort

 website

Kimi Platform

https://platform.kimi.ai

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Kimi K3 — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON