RadixAttention: How SGLang Reuses the KV Cache Across Requests
SGLang keeps the KV cache in a radix tree: edges hold runs of tokens, nodes hold the GPU slot indices where their KV lives. We walk through the slot pool, TreeNode, inserting and splitting nodes, LRU eviction, and four requests that fill the pool and force an eviction.