https://zhangchitc.github.io/tags/gpu/Recent content in Gpu on Chi's BlogHugoen-usWed, 04 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/flash-attention/Wed, 04 Nov 2026 01:00:00 +0800https://zhangchitc.github.io/posts/flash-attention/FlashAttention does every multiplication standard attention does, yet runs several times faster. We build it up step by step from the GPU memory hierarchy: kernel fusion, safe softmax, online softmax, and tiling.