Prompt-Caching

Why serving LLMs fragments GPU memory, and how vLLM’s PagedAttention fixes it with OS-style paging: physical blocks, a block table, chained block hashes, and a walkthrough of how blocks are allocated, shared, freed and evicted.

A prompt cache doesn’t store your last answer. We walk through prefill, decode and attention to see what the KV cache keeps, and how a prompt cache reuses it across requests.