How DeepSeek's Multi-Head Latent Attention Works
MLA caches one 512-dimensional latent per token instead of 128 heads’ worth of keys and values, and matrix absorption means it never has to decompress them. We derive it step by step: joint compression, absorption, why RoPE breaks it, and the decoupled RoPE branch that fixes it.