LLM Inference

A series on LLM inference internals - the end-to-end inference flow, FlashAttention and PagedAttention, and agentic workflows with Mixture-of-Experts
  1. LLM Inference Introduction: the end-to-end flow and vector math
    LLM Basic
  2. PagedAttention
    How PagedAttention uses virtual-memory-style KV-cache paging to reduce fragmentation and increase LLM serving throughput.
  3. FlashAttention: Exact Attention Without Materialization
    How FlashAttention avoids materializing quadratic attention matrices by combining tiled execution, online softmax, and kernel fusion.
  4. The Evolution of FlashAttention: From Ampere to Blackwell
    How FlashAttention-2, -3, and -4 rescheduled the same exact-attention algorithm as GPU bottlenecks shifted from occupancy to asynchronous pipelines and specialized memory.
  5. Agentic Inference and Mixture-of-Experts: Two Levels of Conditional Execution
    A systems analysis of conditional execution across agentic control loops and sparse Mixture-of-Experts layers.
  6. FlashInfer: A Customizable Attention Engine for LLM Serving
    How FlashInfer unifies heterogeneous KV-cache layouts, specialized attention kernels, and dynamic serving schedules in one inference-oriented engine.