DeepSeek-V4-Pro-0813

DeepSeek

Parameters

1.7T total / 49.0B active

MoE: total / active

Architecture

MoE with DSpark speculative decoding module

Released

13.08.2026

License

MIT License

Open Weights Commercial Use Multimodal BF16, I64, F32, F8_E4M3, I8 DeepSeek en zh

Input Modalities

text

Output Modalities

text

Context (native)

1,000,000 tokens

Context (extended)

1,000,000 tokens

Openness Index Score 100.0/100

About

DeepSeek-V4-Pro-0813 (deepseek-ai/DeepSeek-V4-Pro-0813) is the official release of DeepSeek-V4-Pro, superseding the preview version with greatly enhanced agentic capabilities and performance improvements especially pronounced in production environments. It is built on the DeepSeek-V4-Pro (Preview) model structure with a DSpark speculative decoding module attached - 1.7T total parameters, 49B activated per token, one-million-token context, released August 13, 2026 under the MIT License.

The V4-Pro stack combines Compressed Sparse Attention (CSA) with Heavily Compressed Attention for long-context efficiency (at 1M context, 27% of single-token FLOPs and 10% of KV cache versus DeepSeek-V3.2) and Manifold-Constrained Hyper-Connections (mHC) for stable deep-stack signal propagation, trained with the Muon optimizer. It outperforms DeepSeek-V4-Pro (Preview) across the card's agentic benchmarks (HLE, Terminal Bench 2.1, NL2Repo, Cybergym, DeepSWE, Toolathlon-Verified, Agents' Last Exam, AutomationBench, DSBench) and is broadly competitive with the strongest proprietary models. Pre-trained on 32T+ diverse tokens; post-training follows the V4 two-stage paradigm - domain-specific expert cultivation (SFT + RL with GRPO) then consolidation via on-policy distillation. Supports three reasoning effort modes: low, high, max.

Training Data Official release of DeepSeek-V4-Pro, superseding the preview version with greatly enhanced agentic capabilities. Built on DeepSeek-V4-Pro (Preview) model structure with DSpark speculative decoding module attached. Pre-trained on 32T+ diverse tokens. Post-training: two-stage paradigm with independent domain-specific expert cultivation (SFT + RL with GRPO), then unified model consolidation via on-policy distillation. Supports three reasoning effort modes: low, high, max.

Benchmark Scores

Benchmark Score Date Standing
HLE (with tools)
stem_reasoning
82.17%
13.08.2026
#7/49
better than 85%
Humanity's Last Exam
stem_reasoning
76.89%
13.08.2026
#12/71
better than 83%
Terminal Bench 2.1
coding_agent
97.01%
13.08.2026
#5/44
better than 91%
NL2Repo
coding_agent
78.77%
13.08.2026
#4/41
better than 90%
Cybergym
general_agent
90.28%
13.08.2026
#4/17
better than 75%
DeepSWE
coding_agent
86.24%
13.08.2026
#8/21
better than 60%
Toolathlon Verified
general_agent
92.42%
13.08.2026
#8/35
better than 74%
Agents' Last Exam
general_agent
46.32%
13.08.2026
#14/29
better than 50%
Automation-Bench
general_agent
40.38%
13.08.2026
#12/27
better than 58%
DSBench-FullStack
coding_agent
84.83%
13.08.2026
#4/8
better than 57%

Showing 10 of 14 results

What is a good HLE (with tools) score? What is a good Humanity's Last Exam score? What is a good Terminal Bench 2.1 score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 49.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 3400.0 GB 7.6 GB 3577.6 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 1700.0 GB 3.8 GB 1788.8 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 850.0 GB 1.9 GB 894.4 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 850.0 GB 1.9 GB 894.4 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 3577.6 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block input Embedding vocab 129K · d 7168 Full Attention Sparse Attn 128:1 · dₕ 512 · win 128 ×61 MoE FFN 384 experts · top-6 · +1 shared · dᴻ 3072 MTP Head ×1 speculative layer Final Norm LM Head vocab 129K output
Attention
Sparse Attention (128:1)
MoE
384 experts · top-6 per token
Layers
61
Hidden size
7168
Context
1M tokens
RoPE θ
10K
Parameters
1700000M
Active params
49000M

Source: Hugging Face config.json · DeepseekV4ForCausalLM · model repo

Type: MoE
Attention: Hybrid Attention (Compressed Sparse Attention + Heavily Compressed Attention)
Decoder: Autoregressive
MoE: yes (? experts)
Routing: Expert routing
Speculative Decoding

DSpark (num_speculative_tokens=7)

Training Pipeline

  1. 1
    pretraining

    Pre-training on 32T+ tokens

    Pre-trained on more than 32T diverse and high-quality tokens. Uses Muon optimizer for faster convergence and greater training stability. MoE architecture with 1.7T total params (49B activated). DSpark speculative decoding module attached.

  2. 2
    sft

    Domain-specific expert SFT

    Independent cultivation of domain-specific experts through SFT. Part of first stage of two-stage post-training paradigm.

  3. 3
    rl

    Domain-specific expert RL (GRPO)

    Independent cultivation of domain-specific experts through RL with GRPO. Part of first stage of two-stage post-training paradigm.

  4. 4
    other

    On-policy distillation consolidation

    Unified model consolidation via on-policy distillation, integrating distinct proficiencies across diverse domains into a single model. Second stage of two-stage post-training paradigm.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
32T+ token pre-training corpus (diverse, high-quality) pretraining — —
Domain-specific SFT + GRPO RL corpora rl — —

Linked Resources

arxiv

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

https://arxiv.org/abs/2606.19348

 website

DeepSeek Homepage

https://www.deepseek.com/

 demo

DeepSeek Chat

https://chat.deepseek.com/

 other

vLLM Recipe

https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Pro

 other

SGLang Cookbook

https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4#hw=gb300&variant=flash-official&quant=fp4&strategy=low-latency&nodes=single

 citation

DeepSeek-V4 Citation

https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for DeepSeek-V4-Pro-0813 — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON