Why FlashAttention is So Fast
FlashAttention does every multiplication standard attention does, yet runs several times faster. We build it up step by step from the GPU memory hierarchy: kernel fusion, safe softmax, online softmax, and tiling.
Inference
How PagedAttention Works: Paging the KV Cache Like an Operating System
Why serving LLMs fragments GPU memory, and how vLLM’s PagedAttention fixes it with OS-style paging: physical blocks, a block table, chained block hashes, and a walkthrough of how blocks are allocated, shared, freed and evicted.
How Much GPU Memory Does the KV Cache Really Take?
We build the KV cache size formula one factor at a time, from single-head attention to multi-head, stacked layers and MQA/GQA, then check it against a real calculator: 13.73 GiB for 100,000 tokens on Qwen3-8B.
KV Cache vs. Prompt Cache: What's the Difference?
A prompt cache doesn’t store your last answer. We walk through prefill, decode and attention to see what the KV cache keeps, and how a prompt cache reuses it across requests.