DeepSeek-V4-Flash-Vision-Exp

About

DeepSeek-V4-Flash-Vision-Exp (deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) is DeepSeek's first experimental multimodal model in the DeepSeek-V4 family - a 305B-parameter (13B active) sparse Mixture-of-Experts agent model handling text and image input, released September 8, 2026. It builds on the DeepSeek-V4-Flash architecture by incorporating a vision encoder and aligner (32-layer ViT, dim 1024, patch 14, downsample 3, up to 384 tokens per image) and undergoing continued training to unlock visual understanding.

The DeepseekV4 backbone runs DFlash attention (64 heads of dim 512 with 1 KV head, index selection with 64 index heads of dim 128 and top-512, q/o LoRA ranks 1024 in 8 output groups, 3 hash layers) mixed with sparse MoE: 256 routed + 1 shared expert, 6 routed experts per token (expert FFN 2048, fp4 expert dtype, sqrtsoftplus scoring, noaux-tc top-k), 43 layers with per-layer token compression (ratios 4/128, compress RoPE theta 160k), Hyper-Connections (hc_mult 4), a DSpark forward path (block size 5, Markov rank 256, target layers 40-42, noise tokens), 3 Multi-Token Prediction (MTP) layers, native FP8 quantization, sliding window 128, SwiGLU limit 10.0, and a 1,048,576-token context (YaRN factor 16 from 64K).

Compared to DeepSeek-V4-Flash-0731 it achieves substantial improvements on multimodal agent capabilities (ApexBench 36.5 vs 26.2, Agents' Last Exam 27.3 vs 25.2, Chartography 64.3, ZeroBench 35.0) while maintaining comparable text-only agent performance (Terminal Bench 2.1: 83.9). Released under the MIT License.

Training Data First experimental multimodal model in the DeepSeek-V4 family. Builds on the DeepSeek-V4-Flash architecture by incorporating visual modules and undergoing continued training to unlock visual understanding capabilities. Compared to DeepSeek-V4-Flash-0731, achieves substantial improvements on multimodal agent capabilities while maintaining comparable performance on text-only agent tasks.