DeepSeek just released V4, and it changes the math on long-context inference. The new DeepSeek-V4-Pro packs 1.6 trillion parameters (49B activated) with a million-token context window — while using only 27% of the per-token FLOPs and 10% of the KV cache of its predecessor, DeepSeek-V3.2.
What Changed Under the Hood
The efficiency gains come from three architectural shifts:
- Hybrid Attention: DeepSeek combined Compressed Sparse Attention (CSA) with Heavily Compressed Attention (HCA) to dramatically cut memory and compute costs at long context lengths.
- Manifold-Constrained Hyper-Connections (mHC): A new take on residual connections that improves signal propagation stability across deep layers without sacrificing expressiveness.
- Muon Optimizer: Replaces the standard Adam-family optimizer for faster convergence during pre-training.
Both V4-Pro and the smaller V4-Flash (284B total, 13B activated) were pre-trained on over 32 trillion tokens, followed by a two-stage post-training pipeline: domain-specific expert cultivation (SFT + RL with GRPO), then unified model consolidation through on-policy distillation.
Benchmark Performance
DeepSeek-V4-Pro-Max (maximum reasoning effort mode) claims the title of best open-source model currently available:
- Top-tier performance on coding benchmarks
- Significantly closes the gap with closed-source leaders on reasoning and math
- The million-token context isn't just a marketing number — the hybrid attention mechanism means it's actually usable in production
What This Means for Developers
The 10x reduction in KV cache is the standout number. Long-context models have been expensive to serve because KV cache grows linearly with sequence length. DeepSeek-V4's compression approach means:
- Lower inference cost at production scale
- Faster time-to-first-token for long documents
- More concurrent requests per GPU
The MIT license on both model weights and code is notable — this is fully open, no restrictions.
Two Variants
| DeepSeek-V4-Pro | DeepSeek-V4-Flash | |
|---|---|---|
| Total Parameters | 1.6T | 284B |
| Activated Parameters | 49B | 13B |
| Context Length | 1M tokens | 1M tokens |
| Best For | Complex reasoning, coding, analysis | Fast inference, chat, lightweight tasks |
| License | MIT | MIT |
Should You Use It?
If you're working with long documents — codebases, legal texts, research papers, multi-turn conversations — V4-Pro is worth serious evaluation. The efficiency gains over V3.2 are substantial enough to change deployment economics.
For most general-purpose applications, V4-Flash at 13B activated parameters could be the sweet spot: million-token context in a package small enough to run on a single consumer GPU.
The model is available now on HuggingFace at deepseek-ai/DeepSeek-V4-Pro and deepseek-ai/DeepSeek-V4-Flash.