DeepSeek-V4-Pro

Model tree for deepseek-ai/DeepSeek-V4-Pro

Finetunes

15 models

Quantizations

27 models

Spaces using deepseek-ai/DeepSeek-V4-Pro 100

Collection including deepseek-ai/DeepSeek-V4-Pro

[

DeepSeek-V4

Collection

10 items • Updated 1 day ago • 900

](https://huggingface.co/collections/deepseek-ai/deepseek-v4)

Paper for deepseek-ai/DeepSeek-V4-Pro

[

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

Paper • 2606.19348 • Published Apr 26 • 43

](https://huggingface.co/papers/2606.19348)

Evaluation results

• openai/gsm8k · Gsm8k leaderboard

  92.6

• SWE-bench/SWE-bench_Multilingual · Swe Bench Multilingual Resolved leaderboard

  76.2

• Idavidrein/gpqa · Diamond leaderboard

  90.1

• ScaleAI/SWE-bench_Pro · SWE Bench Pro [leaderboard](https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?eval_result=deepseek-ai/DeepSeek-V4-Pro&leaderboard_task

(section continues in the model card)

Citation

@misc{deepseekai2026deepseekv4,
      title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
      author={DeepSeek-AI},
      year={2026},
}

License

This repository and the model weights are licensed under the MIT License.

How to Run Locally

Please refer to the inference folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.

For local deployment, we recommend setting the sampling parameters to temperature = 1.0, top_p = 1.0. For the Think Max reasoning mode, we recommend setting the context window to at least 384K tokens.

Chat Template

This release does not include a Jinja-format chat template. Instead, we provide a dedicated encoding folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the encoding folder for full documentation.

A brief example:

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]

## messages -> string
prompt = encode_messages(messages, thinking_mode="thinking")

## string -> tokens
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Pro")
tokens = tokenizer.encode(prompt)

Instruct Model

DeepSeek-V4-Pro and DeepSeek-V4-Flash both support three reasoning effort modes:

DeepSeek-V4-Pro-Max vs Frontier Models

Comparison across Modes

Evaluation Results

Base Model

Model Downloads

*FP4 + FP8 Mixed: MoE expert parameters use FP4 precision; most other parameters use FP8.

Introduction

We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models — DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSeek-V4-Flash with 284B parameters (13B activated) — both supporting a context length of one million tokens.

DeepSeek-V4 series incorporate several key upgrades in architecture and optimization:

1. Hybrid Attention Architecture: We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only 27% of single-token inference FLOPs and 10% of KV cache compared with DeepSeek-V3.2.
2. Manifold-Constrained Hyper-Connections (mHC): We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
3. Muon Optimizer: We employ the Muon optimizer for faster convergence and greater training stability.

We pre-train both models on more than 32T diverse and high-quality tokens, followed by a comprehensive post-training pipeline. The post-training features a two-stage paradigm: independent cultivation of domain-specific experts (through SFT and RL with GRPO), followed by unified model consolidation via on-policy distillation, integrating distinct proficiencies across diverse domains into a single model.

DeepSeek-V4-Pro-Max, the maximum reasoning effort mode of DeepSeek-V4-Pro, significantly advances the knowledge capabilities of open-source models, firmly establishing itself as the best open-source model available today. It achieves top-tier performance in coding benchmarks and significantly bridges the gap with leading closed-source models on reasoning and agentic tasks. Meanwhile, DeepSeek-V4-Flash-Max achieves comparable reasoning performance to the Pro version when given a larger thinking budget, though its smaller parameter scale naturally places it slightly behind on pure knowledge tasks and the most complex agentic workflows.

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

Technical Report👁️