https://zhangchitc.github.io/posts/Recent content in Posts on Chi's BlogHugoen-usThu, 05 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/mixture-of-experts/Thu, 05 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/mixture-of-experts/Mixture of Experts lets a model’s parameter count grow tens of times over while the compute per token barely moves. We build it up step by step: copying the FFN into experts, the router, dispatch and combine, top-K routing, and the auxiliary loss that keeps experts balanced.https://zhangchitc.github.io/posts/flash-attention/Wed, 04 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/flash-attention/FlashAttention does every multiplication standard attention does, yet runs several times faster. We build it up step by step from the GPU memory hierarchy: kernel fusion, safe softmax, online softmax, and tiling.https://zhangchitc.github.io/posts/paged-attention/Tue, 03 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/paged-attention/Why serving LLMs fragments GPU memory, and how vLLM’s PagedAttention fixes it with OS-style paging: physical blocks, a block table, chained block hashes, and a walkthrough of how blocks are allocated, shared, freed and evicted.https://zhangchitc.github.io/posts/kv-cache-memory/Mon, 02 Nov 2026 17:00:00 +0800https://zhangchitc.github.io/posts/kv-cache-memory/We build the KV cache size formula one factor at a time, from single-head attention to multi-head, stacked layers and MQA/GQA, then check it against a real calculator: 13.73 GiB for 100,000 tokens on Qwen3-8B.https://zhangchitc.github.io/posts/kv-cache-vs-prompt-cache/Mon, 02 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/kv-cache-vs-prompt-cache/A prompt cache doesn’t store your last answer. We walk through prefill, decode and attention to see what the KV cache keeps, and how a prompt cache reuses it across requests.https://zhangchitc.github.io/posts/positional-encoding/Sun, 01 Nov 2026 00:00:00 +0000https://zhangchitc.github.io/posts/positional-encoding/Attention can’t tell “dog bites man” from “man bites dog”. We try integer, fractional and binary encodings, see where each one breaks, and arrive at sines and cosines: a set of clock hands spinning at different speeds.