FlashAttention: Exact Attention Without Materialization
How FlashAttention computes exact attention without materializing quadratic score and probability matrices in GPU memory.
Systems
PagedAttention
How PagedAttention uses virtual-memory-style KV-cache paging to reduce fragmentation and increase LLM serving throughput.
The Evolution of FlashAttention: From Ampere to Blackwell
How successive FlashAttention implementations respond to the bottleneck exposed by each new GPU generation.
FlashInfer: A Customizable Attention Engine for LLM Serving
FlashInfer treats LLM attention as a serving systems problem spanning data layout, kernel generation, and runtime scheduling.