DeepSeek-V4-Flash

DeepSeek

Parameters

291.0B total / 13.0B active

MoE: total / active

Architecture

MoE

Released

26.06.2026

License

MIT License

Open Weights Commercial Use Multimodal BF16, I64, F32, F8_E4M3, I8 DeepSeek en zh

Input Modalities

text

Output Modalities

text

Context (native)

1,000,000 tokens

Context (extended)

1,000,000 tokens

Openness Index Score 100.0/100

About

DeepSeek-V4-Flash (deepseek-ai/DeepSeek-V4-Flash) is the preview release of the smaller DeepSeek-V4 MoE model - 284B total parameters with 13B activated per token (card figure; DB stores 291B total) - supporting a one-million-token context, released June 26, 2026 under the MIT License. It supports three reasoning modes: Non-think, Think High, Think Max.

The DeepSeek-V4 architecture combines Compressed Sparse Attention (CSA) with Heavily Compressed Attention (HCA) in a hybrid stack that dramatically improves long-context efficiency - at 1M context the series needs only 27% of single-token inference FLOPs and 10% of the KV cache compared with DeepSeek-V3.2 - and strengthens residual connections with Manifold-Constrained Hyper-Connections (mHC) for stable cross-layer signal propagation. Training uses the Muon optimizer for faster convergence.

Pre-trained on 32T+ diverse high-quality tokens, its post-training runs a two-stage paradigm: independent cultivation of domain-specific experts via SFT and RL with GRPO, then unified model consolidation via on-policy distillation into a single model. DeepSeek-V4-Flash-Max (maximum reasoning effort) achieves reasoning performance comparable to the Pro version given a larger thinking budget, though its smaller scale places it slightly behind on pure knowledge and the most complex agentic workflows.

Training Data Preview version of DeepSeek-V4-Flash. Pre-trained on 32T+ diverse tokens. Post-training: two-stage paradigm with independent domain-specific expert cultivation (SFT + RL with GRPO), then unified model consolidation via on-policy distillation. Supports three reasoning modes: Non-think, Think High, Think Max.

Benchmark Scores

Benchmark Score Date Standing
Vals.ai Financial Agent 1.1 (without web search)
general_agent
71.00%
04.06.2026
#5/7
better than 33%
Terminal-Bench 2.1 (Terminus-2)
coding_agent
48.97%
04.06.2026
#33/44
better than 26%
Vals.ai Financial Agent 1.1 (with web search)
general_agent
81.36%
04.06.2026
#3/7
better than 67%
CritPt (no tools)
stem_reasoning
31.55%
04.06.2026
#14/21
better than 35%
LiveCodeBench v6
stem_reasoning
97.41%
26.06.2026
#5/49
better than 92%
IMOAnswerBench
stem_reasoning
93.47%
26.06.2026
#8/30
better than 76%
SWE-bench Multilingual
coding_agent
80.83%
26.06.2026
#21/48
better than 57%
ProfBench (Search)
general_agent
79.14%
04.06.2026
#2/7
better than 83%
IMOAnswerBench (with tools)
stem_reasoning
77.92%
04.06.2026
#4/7
better than 50%
OmniScience Non-Hallucination
knowledge
2.80
04.06.2026
#10/10
better than 0%

Showing 10 of 56 results

What is a good Vals.ai Financial Agent 1.1 (without web search) score? What is a good Terminal-Bench 2.1 (Terminus-2) score? What is a good Vals.ai Financial Agent 1.1 (with web search) score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 13.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 582.0 GB 2.0 GB 613.1 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 291.0 GB 1.0 GB 306.6 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 145.5 GB 0.5 GB 153.3 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 145.5 GB 0.5 GB 153.3 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 613.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block input Embedding vocab 129K · d 4096 Full Attention Sparse Attn 64:1 · dₕ 512 · win 128 ×43 MoE FFN 256 experts · top-6 · +1 shared · dᴻ 2048 MTP Head ×1 speculative layer Final Norm LM Head vocab 129K output
Attention
Sparse Attention (64:1)
MoE
256 experts · top-6 per token
Layers
43
Hidden size
4096
Context
1M tokens
RoPE θ
10K
Parameters
291000M
Active params
13000M

Source: Hugging Face config.json · DeepseekV4ForCausalLM · model repo

Type: MoE
Attention: Hybrid Attention (Compressed Sparse Attention + Heavily Compressed Attention)
Decoder: Autoregressive
MoE: yes (? experts)
Routing: Expert routing

Training Pipeline

  1. 1
    pretraining

    Pre-training on 32T+ tokens

    Pre-trained on more than 32T diverse and high-quality tokens. Uses Muon optimizer for faster convergence and greater training stability. MoE architecture with 284B total params (13B activated).

  2. 2
    sft

    Domain-specific expert SFT

    Independent cultivation of domain-specific experts through SFT. Part of first stage of two-stage post-training paradigm.

  3. 3
    rl

    Domain-specific expert RL (GRPO)

    Independent cultivation of domain-specific experts through RL with GRPO. Part of first stage of two-stage post-training paradigm.

  4. 4
    other

    On-policy distillation consolidation

    Unified model consolidation via on-policy distillation, integrating distinct proficiencies across diverse domains into a single model. Second stage of two-stage post-training paradigm.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
32T+ token pre-training corpus (diverse, high-quality) pretraining — —
Domain-specific SFT + GRPO RL corpora rl — —

Linked Resources

arxiv

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

https://arxiv.org/abs/2606.19348

 website

DeepSeek Homepage

https://www.deepseek.com/

 demo

DeepSeek Chat

https://chat.deepseek.com/

 citation

DeepSeek-V4 Citation

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash

 collection

DeepSeek-V4 Collection

https://huggingface.co/collections/deepseek-ai/deepseek-v4

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for DeepSeek-V4-Flash — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON