The KV Cache: Sizing It, and the Attention Variants That Shrink It
What the KV cache costs per token, and how MQA, GQA and MLA rewrite attention to make that number smaller.
Inference
Neural Network Quantization
What quantization actually changes in a network, which parts of a transformer break first, and which techniques repair them.
FlashInfer: A Customizable Attention Engine for LLM Serving
FlashInfer treats LLM attention as a serving systems problem spanning data layout, kernel generation, and runtime scheduling.