Verstehen
DeepSeek-V4.1 Flash: Architecture Revamp and Extreme Optimization
Überblick
DeepSeek-V4.1 Flash introduces aggressive architectural optimizations to make long-context LLMs and agentic workloads dramatically more affordable. Growing to 552 billion parameters while significantly reducing its KV cache footprint through techniques like Compressed Sparse Attention 2 (CSA2) and Causal Encoder Decoder (CED) architecture, the model delivers strong performance at a fraction of the cost of competing frontier models.
Wichtige Punkte
- DeepSeek-V4.1 Flash grows to 552 billion parameters while compressing the KV cache size per token by 54 times over the last 9 months.
- Compressed Sparse Attention 2 (CSA2) introduces layer-dimension compression by dividing attention layers into full, reindex, and reuse modes.
- The Causal Encoder Decoder (CED) architecture splits processing so that input tokens during prefill traverse only a smaller encoder portion, significantly cutting prefill compute costs.
- SWA bounded replay replaces the dependency chain of the local sliding window attention cache by replaying only the final 128 tokens.
- The model is directly trained with sparse attention from scratch at 64k context before scaling up to a 1 million token context across 45 trillion multimodal tokens.