FlashAttention: Exact Attention Without Materialization
How FlashAttention computes exact attention without materializing quadratic score and probability matrices in GPU memory.
Flashattention
FlashAttention in Practice: Four Schedules and One Branch
How successive FlashAttention implementations respond to the bottleneck exposed by each new GPU generation, and what decoding needs instead.