A 200-billion-parameter model with a KV cache 15 times smaller than a standard model, generating tokens nearly 6 times faster — that is not a theoretical benchmark. Researchers at PyImageSearch and Towards AI, analyzing DeepSeek-V3's architecture in detail, confirmed both figures independently. The mechanism responsible is called Multi-head Latent Attention, or MLA. It did not just optimize DeepSeek. It redefined what memory efficiency in a large language model can look like. While Lecture 2 focused on sparse MoE activation, MLA offers a distinct approach to efficiency by addressing memory bandwidth bottlenecks. Traditional multi-head attention is expensive for a specific reason: it projects queries, keys, and values separately for every single attention head, generating independent weight matrices W-K and W-V per head. Memory usage scales with every head, every layer, every token in the context window. That cost compounds fast. MLA attacks this directly by introducing a latent representation space. Instead of computing full key and value matrices per head, it first compresses the input into a shared latent vector using a compression matrix called C-KV. That single compressed vector is then shared across both key and value projections for all heads simultaneously. The multi-head structure — the actual per-head differentiation — only reappears in the decompression phase, through matrices W-Q-U, W-K-U, and W-V-U. Compute once, share everywhere. The compression matrices C-Q and C-KV are calculated a single time and reused across all heads, slashing redundant computation. Causal masking is applied after decompression, preserving autoregressive properties so position i only attends to positions at or before i — standard behavior, maintained cleanly. Here is what makes this counterintuitive, Yunying: reducing data transfer — moving less information between memory and compute — actually improves performance. More bandwidth consumed means more latency, more heat, more hardware strain. By compressing KV representations rather than sharing heads the way older methods like GQA did, MLA keeps the full expressive power of multi-head attention while radically shrinking the memory footprint. Attention scores still incorporate both content similarity through latent vectors and positional information via RoPE. Nothing meaningful is lost. The 15x cache reduction and 5.7x token generation speedup are the direct numerical result. This is the insight to carry forward, Yunying: MLA solved the memory bandwidth bottleneck — the quiet constraint that limits how large a context window you can run and how fast tokens can flow — without sacrificing modeling accuracy. DeepSeek-V3 can handle larger datasets, longer contexts, and faster inference not because it got more hardware, but because it moved less data. This architectural discipline highlights MLA's pivotal role in enhancing memory efficiency and token generation speed, central to DeepSeek's success.