LLM Inference Introduction: the end-to-end flow and vector math
Every vector an LLM computes during inference, from token ids to logits, with the shapes and the code.
Inference
The KV Cache: Sizing It, and the Attention Variants That Shrink It
What the KV cache costs per token, and how MQA, GQA and MLA rewrite attention to make that number smaller.
Neural Network Quantization
What quantization actually changes in a network, which parts of a transformer break first, and which techniques repair them.
FlashInfer: A Customizable Attention Engine for LLM Serving
FlashInfer treats LLM attention as a serving systems problem spanning data layout, kernel generation, and runtime scheduling.
Continuous Batching: Who Decides What Runs Next
The scheduler is the layer between paged memory and fast kernels, and it decides whether either of them matters.