https://zhangchitc.github.io/tags/deepseek/Recent content in Deepseek on Chi's BlogHugoen-usTue, 27 Oct 2026 01:44:00 +0800https://zhangchitc.github.io/posts/multi-head-latent-attention/Tue, 27 Oct 2026 01:44:00 +0800https://zhangchitc.github.io/posts/multi-head-latent-attention/MLA caches one 512-dimensional latent per token instead of 128 heads’ worth of keys and values, and matrix absorption means it never has to decompress them. We derive it step by step: joint compression, absorption, why RoPE breaks it, and the decoupled RoPE branch that fixes it.