DeepSeek-V4.1-Flash

About

DeepSeek-V4.1-Flash is DeepSeek's MIT-licensed multimodal Mixture-of-Experts model with 552B backbone parameters (763B total) and a 1M-token context window. Maintained by DeepSeek, it natively processes images and text (DeepSeek-ViT vision encoder plus a two-layer MLP projector) and generates text autoregressively, with English and Chinese as primary languages.

Its defining novelty is extreme KV-cache compression for agentic workloads: the global KV cache shrinks to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash and 1/437 of DeepSeek-V1 — while activating only 8B parameters per token during prefill and 16B during decode.

• Causal Encoder-Decoder (CED): a 40-layer Transformer split into a 20-layer causal encoder and a 20-layer decoder; the decoder's global KV cache is projected from the final encoder hidden states rather than from each decoder layer's own states.
• Compressed Sparse Attention 2 (CSA2): every attention layer runs in one of three static modes (Full, Reindex, Reuse), sharing main KV and indexer K across layers and reusing Top-K sparse-attention indices; a Hierarchical Sparse Indexer bounds deeper indexer cost independently of context length.
• FP4 main KV caching (E2M1 format, one E4M3 scale per 16 channels) and SWA Bounded Replay, which reconstructs missing sliding-window-attention KV states by replaying only the most recent n_win tokens — cutting the persistent KV footprint to about 1/8 of DeepSeek-V4-Flash without persisting SWA KV to SSD.
• Sparse MoE: 1 shared expert plus 384 routed experts per layer with 6 routed experts activated per token; Engram conditional memory (196B parameters, token-based lookup), DSpark speculative decoding (semi-autoregressive drafts with confidence-scheduled verification) and Single-Pass mHC with an efficient Mega-mHC kernel.

Pre-training used a 45T-token multimodal corpus with sparse attention trained at 64K sequence length, extending context to 1M tokens at 34T tokens. Post-training follows the SFT → RL → on-policy distillation (OPD) paradigm, with large-scale automated synthesis of agent tasks and environments; a continuously controllable reasoning_effort setting (integer 1–100) trades inference cost for accuracy.

At maximum reasoning effort the model sets records on DeepSWE v1.1 (74.2) and Terminal-Bench 2.1 (90.6) and leads on CyberGym (88.1), AutomationBench (54.8) and Agent's Last Exam (31.8). Recommended sampling: temperature 1.0, top_p 0.95, context window 1M tokens, max_tokens ≥ 256K.

Training Data Trained from scratch on a 45T-token multimodal corpus; sparse attention trained at 64K sequence length, context extended to 1M tokens at 34T tokens. Post-training: SFT -> RL -> on-policy distillation (OPD) with large-scale automated synthesis of agent tasks and environments.