DeepSeek-V4-Pro

About

DeepSeek-V4-Pro (deepseek-ai/DeepSeek-V4-Pro, released June 26, 2026 under the MIT License) is the preview flagship of the DeepSeek-V4 series - a 1.6T-parameter (49B activated) Mixture-of-Experts model with a one-million-token context, presented alongside DeepSeek-V4-Flash (284B, 13B activated). DeepSeek-V4-Pro-Max, its maximum reasoning effort mode, significantly advances the knowledge capabilities of open-source models - top-tier in coding benchmarks and significantly bridging the gap with leading closed-source models on reasoning and agentic tasks.

Architecture and optimization upgrades: a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) dramatically improves long-context efficiency - at 1M-token context, DeepSeek-V4-Pro requires only 27% of single-token inference FLOPs and 10% of KV cache compared with DeepSeek-V3.2; Manifold-Constrained Hyper-Connections (mHC) strengthen residual connections for stable cross-layer signal propagation; the Muon optimizer provides faster convergence and training stability. Both models are pre-trained on more than 32T diverse, high-quality tokens, followed by a two-stage post-training paradigm: independent cultivation of domain-specific experts (SFT and RL with GRPO), then unified model consolidation via on-policy distillation into a single model. Technical report: arXiv 2606.19348.

Training Data Preview version of DeepSeek-V4-Pro (duplicate record - same as model_id 3616c73d). Pre-trained on 32T+ diverse tokens with MoE architecture.