Parameters
2.5B
Architecture
LlamaForCausalLM
Released
08.09.2026
License
Apache License 2.0
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
MiniCPM5-2B (openbmb/MiniCPM5-2B, released September 8, 2026 under the Apache 2.0 License) is the second model in the MiniCPM5 series - a dense 2B Transformer (2.52B parameters) built for on-device, local deployment and resource-constrained scenarios, reaching 2B-class open-source SOTA (average score 53.9 in its comparison set, exceeding even the larger models listed, highest of which scores 51.1). It stays competitive with 4B-class models overall, with advantages in coding, mathematics, long-context understanding, tool use and agentic tasks, and provides native long-context support (128K context).
Its training is a full-stack practice of UltraData Tiered Data Management (arXiv 2602.09003): base training (stable + decay phases) and mid-training build core language and target capabilities on the open Ultra-FineWeb/UltraX/UltraData-Code (L0-L3 tiered)/UltraData-Math corpora; post-training runs SFT (400B tokens of deep-thinking SFT), RL with the critic-based algorithm of JustRL II (scaling small LLMs to 128K reasoning), and On-Policy Distillation (OPD) merging 16 expert models produced by RL (including 5 agentic experts) into one release model via full-vocabulary reverse-KL advantage estimates - improving reasoning/general capabilities by an average of +10.96 points and agentic capabilities by +6.96. All datasets are released as part of the UltraData family.
Training Data UltraData family: UltraX, UltraData-Code (L0-L3 tiered), UltraData-SFT-2605, UltraData-SFT-Agent-2609, UltraData-RL-2609, Ultra-FineWeb, Ultra-FineWeb-L3, UltraData-Math. 400B tokens of deep-thinking SFT.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
Humanity's Last Exam
stem_reasoning
|
12.88%
|
— |
#59/71
better than 17%
|
|
MMLU-Pro
knowledge
|
34.84%
|
— |
#37/48
better than 23%
|
|
BFCL-V4
general_agent
|
86.26%
|
— |
#7/27
better than 77%
|
|
SWE-bench Verified
coding_agent
|
52.75%
|
— |
#48/60
better than 20%
|
|
GPQA Diamond
stem_reasoning
|
54.43%
|
— |
#67/80
better than 16%
|
|
BrowseComp-zh
general_agent
|
60.36%
|
— |
#5/13
better than 67%
|
|
SuperGPQA
knowledge
|
32.88%
|
— |
#21/28
better than 26%
|
|
BrowseComp
general_agent
|
41.12%
|
— |
#33/45
better than 27%
|
|
AA-LCR
long_context
|
70.24%
|
— |
#29/42
better than 32%
|
|
Gaia2
general_agent
|
100.00%
|
— |
#1/12
better than 100%
|
|
LongBenchPro
long_context
|
65.01%
|
— |
#4/10
better than 67%
|
|
GDPVal-AA v2
general_agent
|
1.07%
|
— |
#30/38
better than 22%
|
|
LongBench v2
long_context
|
42.53%
|
— |
#14/22
better than 38%
|
|
TAU3-Bench
general_agent
|
28.95%
|
— |
#10/21
better than 55%
|
|
WildClawBench
coding_agent
|
28.53%
|
— |
#12/20
better than 42%
|
|
TAU2-Bench
general_agent
|
100.00%
|
— |
#1/21
better than 100%
|
|
QwenClawBench
coding_agent
|
78.53%
|
— |
#7/17
better than 62%
|
|
LiveCodeBench v6
stem_reasoning
|
66.71%
|
— |
#35/49
better than 29%
|
|
LCB-Pro 25Q2 (Easy)
stem_reasoning
|
100.00%
|
— |
#1/9
better than 100%
|
|
LCB-Pro 25Q2 (Medium)
stem_reasoning
|
100.00%
|
— |
#1/9
better than 100%
|
|
SciCode
|
38.67%
|
— |
#19/27
better than 31%
|
|
AIME 2025
stem_reasoning
|
80.82%
|
— |
#11/24
better than 57%
|
|
AIME 26
stem_reasoning
|
83.80%
|
— |
#34/45
better than 25%
|
|
HMMT Feb 26
stem_reasoning
|
56.09%
|
— |
#36/44
better than 19%
|
|
MATH-500
math
|
67.65%
|
— |
#5/11
better than 60%
|
|
IFBench
instruction_following
|
70.89%
|
— |
#29/42
better than 29%
|
|
Multi-IF
instruction_following
|
64.55%
|
— |
#7/13
better than 50%
|
|
OJBench
stem_reasoning
|
43.91%
|
— |
#10/22
better than 57%
|
|
MMLU-Redux
knowledge
|
54.20%
|
— |
#23/30
better than 24%
|
|
Claw-Eval Avg
coding_agent
|
70.89%
|
— |
#22/40
better than 46%
|
|
NoLiMa
long_context
|
100.00%
|
— |
#1/9
better than 100%
|
|
Terminal Bench 2.1
coding_agent
|
9.09%
|
— |
#38/44
better than 14%
|
|
IFEval
instruction_following
|
83.78%
|
— |
#21/26
better than 20%
|
|
SWE-bench Pro
coding_agent
|
16.05%
|
— |
#67/77
better than 13%
|
What is a good Humanity's Last Exam score? What is a good MMLU-Pro score? What is a good BFCL-V4 score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 5.0 GB | 0.4 GB | 5.7 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 2.5 GB | 0.2 GB | 2.8 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 1.3 GB | 0.1 GB | 1.4 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 1.3 GB | 0.1 GB | 1.4 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Grouped Query Attention (16:2)
- Layers
- 42
- Hidden size
- 2048
- Context
- 131K tokens
- RoPE θ
- 5M
- Parameters
- 2516.8M
Source: Hugging Face config.json · LlamaForCausalLM · model repo
Training Pipeline
-
1
pretraining
Base Training (Stable + Decay)
Base training with stable training and decay training phases to build core language capability and training stability. Uses Ultra-FineWeb, Ultra-FineWeb-L3, UltraX, UltraData-Code, and UltraData-Math corpora.
-
2
sft
Supervised Fine-Tuning (Deep-Thinking SFT)
400B tokens of deep-thinking SFT to establish deep-thinking and general chat abilities. Uses UltraData-SFT-2605 and UltraData-SFT-Agent-2609 (500K agent samples).
-
3
rl
RL + On-Policy Distillation (OPD)
RL training using critic-based algorithm (JustRL II) for improved stability. Specialized RL teachers for math, code, agentic tasks, writing. OPD merges 16 expert models (5 agentic) using full-vocabulary reverse KL divergence. Uses UltraData-RL-2609 (80K+ samples). RL + OPD improves reasoning by avg +10.96 points and agentic capabilities by +6.96 points.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| UltraX-Preview | pretraining | — | text | — |
| UltraData-Code | pretraining | — | text, code | — |
| UltraData-SFT-2605 | finetune | 400B tokens | text | — |
| UltraData-SFT-Agent-2609 | finetune | 500K samples | text | — |
| UltraData-RL-2609 | rl | 80K+ samples | text | — |
| Ultra-FineWeb | pretraining | — | text | — |
Linked Resources
MiniCPM Tech Report
https://arxiv.org/pdf/2506.07900
OpenBMB/MiniCPM GitHub Repository
https://github.com/OpenBMB/MiniCPM
MiniCPM5-2B Online Demo
https://huggingface.co/spaces/openbmb/MiniCPM5-2B-Demo
UltraData
https://ultradata.openbmb.cn/
MiniCPM4: Ultra-efficient LLMs on end devices
https://arxiv.org/abs/2506.07900
UltraData Tiered Data Management
https://arxiv.org/pdf/2602.09003
Trend Analysis
24h Change
+0.7%
7d Change
+6.5%
Current
14,700 pulls
downloads_all_time
+1.6%
followers
+0.2%
downloads
-0.7%
likes
+0.2%
No comments yet
Be the first to share your thoughts!