DeepSeek-V4-Flash-0731

DeepSeek

Parameters

304.0B total / 13.0B active

MoE: total / active

Architecture

MoE with DSpark speculative decoding module

Released

31.07.2026

License

MIT License

Open Weights Commercial Use Multimodal BF16, F32, F8_E4M3, I8 DeepSeek en zh

Input Modalities

text

Output Modalities

text

Context (native)

1,000,000 tokens

Context (extended)

1,000,000 tokens

Openness Index Score 100.0/100

About

DeepSeek-V4-Flash-0731 (deepseek-ai/DeepSeek-V4-Flash-0731) is the official release of DeepSeek-V4-Flash, superseding the June preview with substantially enhanced agentic capabilities - released July 31, 2026 under the MIT License with 304B total parameters (13B activated) and a one-million-token context. It has the same model structure as DeepSeek-V4-Flash-DSpark: the V4 hybrid stack (Compressed Sparse Attention (CSA) + Heavily Compressed Attention, Manifold-Constrained Hyper-Connections (mHC), trained with the Muon optimizer) with a DSpark speculative decoding module attached.

Despite its far smaller activated parameter count it outperforms DeepSeek-V4-Pro (Preview) on the card's agentic benchmarks (Terminal Bench 2.1, NL2Repo, Cybergym, DeepSWE, Toolathlon-Verified, Agents' Last Exam, AutomationBench, DSBench) and is broadly competitive with the strongest proprietary models. Post-training follows the V4 two-stage paradigm - domain-specific expert cultivation (SFT + RL with GRPO) then unified consolidation via on-policy distillation. Technical report: arXiv 2606.19348.

Training Data Official release of DeepSeek-V4-Flash, superseding the preview version with substantially enhanced agentic capabilities. Same model structure as DeepSeek-V4-Flash-DSpark with speculative decoding module attached.

Benchmark Scores

Benchmark Score Date Standing
Terminal Bench 2.1
coding_agent
91.24%
31.07.2026
#15/44
better than 67%
Cybergym
general_agent
76.92%
31.07.2026
#13/17
better than 25%
DeepSWE
coding_agent
74.83%
31.07.2026
#13/21
better than 40%
Toolathlon Verified
general_agent
84.83%
31.07.2026
#16/35
better than 56%
Agents' Last Exam
general_agent
44.79%
31.07.2026
#16/29
better than 46%
DSBench-FullStack
coding_agent
78.86%
31.07.2026
#5/8
better than 43%
DSBench-Hard
coding_agent
73.64%
31.07.2026
#6/9
better than 38%
HLE (with tools)
stem_reasoning
67.02%
13.08.2026
#26/49
better than 48%
Humanity's Last Exam
stem_reasoning
67.61%
13.08.2026
#23/71
better than 69%
CoWorkBench
general_agent
45.10
26.08.2026
#12/12
better than 0%

Showing 10 of 20 results

What is a good Terminal Bench 2.1 score? What is a good Cybergym score? What is a good DeepSWE score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 13.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 608.0 GB 2.0 GB 640.4 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 304.0 GB 1.0 GB 320.2 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 152.0 GB 0.5 GB 160.1 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 152.0 GB 0.5 GB 160.1 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 640.4 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block input Embedding vocab 129K · d 4096 Full Attention Sparse Attn 64:1 · dₕ 512 · win 128 ×43 MoE FFN 256 experts · top-6 · +1 shared · dᴻ 2048 MTP Head ×1 speculative layer Final Norm LM Head vocab 129K output
Attention
Sparse Attention (64:1)
MoE
256 experts · top-6 per token
Layers
43
Hidden size
4096
Context
1M tokens
RoPE θ
10K
Parameters
304000M
Active params
13000M

Source: Hugging Face config.json · DeepseekV4ForCausalLM · model repo

Type: MoE
Attention: Multi-head attention
Decoder: Transformer decoder
MoE: yes (? experts)
Routing: Mixture of Experts with DSpark speculative decoding
Inference Frameworks

vLLM, SGLang

Reasoning Effort Levels

low, high, max

Speculative Decoding

DSpark

Speculative Decoding Tokens

7

Training Pipeline

  1. 1
    pretraining

    Pre-training

    32T+ diverse high-quality tokens (V4 family recipe).

  2. 2
    sft

    Domain-specific expert SFT

    Independent cultivation of domain-specific experts through SFT (V4 two-stage post-training paradigm).

  3. 3
    rl

    GRPO RL expert cultivation

    RL with GRPO applied to the domain-specific experts.

  4. 4
    other

    Consolidation + DSpark attachment

    Unified model consolidation via on-policy distillation, integrating distinct proficiencies; DSpark speculative decoding module attached (card Introduction).

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
32T+ token pre-training corpus (diverse, high-quality) pretraining — —
Domain-specific SFT + GRPO RL corpora rl — —

Linked Resources

arxiv

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

https://arxiv.org/abs/2606.19348

 paper

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

https://huggingface.co/papers/2606.19348

 website

DeepSeek Official Website

https://www.deepseek.com/

 website

DeepSeek Chat

https://chat.deepseek.com/

 collection

DeepSeek-V4 Collection

https://huggingface.co/collections/deepseek-ai/deepseek-v4

 citation

DeepSeek-V4 Citation

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for DeepSeek-V4-Flash-0731 — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON