LLM Training: The End-to-End Flow
What each step of an LLM training iteration computes, what it costs in memory and arithmetic, and where the loop stalls on real hardware.
Llm
LLM Inference Introduction: the end-to-end flow and vector math
LLM Basic
The KV Cache: Sizing It, and the Attention Variants That Shrink It
What the KV cache costs per token, and how MQA, GQA and MLA rewrite attention to make that number smaller.
Tiling and Collective Operations
The two data-movement primitives behind training performance: tiling inside a device, collectives between devices.
PyTorch Distributed Training Internals
The mechanism behind step 5 of the training loop: bucketed gradient all-reduce triggered from autograd hooks, what NCCL does with it, and how to diagnose the result.
PagedAttention
How PagedAttention uses virtual-memory-style KV-cache paging to reduce fragmentation and increase LLM serving throughput.
Activations, Normalization and Optimizers, in Plain Terms
What each of the standard building blocks actually does, why it replaced the thing before it, and what it costs.
FlashAttention: Exact Attention Without Materialization
How FlashAttention computes exact attention without materializing quadratic score and probability matrices in GPU memory.
Neural Network Quantization
What quantization actually changes in a network, which parts of a transformer break first, and which techniques repair them.
The Evolution of FlashAttention: From Ampere to Blackwell
How successive FlashAttention implementations respond to the bottleneck exposed by each new GPU generation.
FlashInfer: A Customizable Attention Engine for LLM Serving
FlashInfer treats LLM attention as a serving systems problem spanning data layout, kernel generation, and runtime scheduling.
Agentic Inference and Mixture-of-Experts: Two Levels of Conditional Execution
How agentic orchestration and Mixture-of-Experts routing introduce conditional execution at different levels of the LLM inference stack.