Learning Series

Multi-part deep-dive series, in reading order

Active Series

LLM Inference

A series on LLM inference internals - the end-to-end inference flow, FlashAttention and PagedAttention, and agentic workflows with Mixture-of-Experts

  1. LLM Inference Introduction: the end-to-end flow and vector math
  2. The KV Cache: Sizing It, and the Attention Variants That Shrink It
  3. PagedAttention
  4. FlashAttention: Exact Attention Without Materialization
  5. The Evolution of FlashAttention: From Ampere to Blackwell
  6. FlashInfer: A Customizable Attention Engine for LLM Serving
  7. Agentic Inference and Mixture-of-Experts: Two Levels of Conditional Execution

CPU ISA for ML

A guide to CPU instruction-set features for ML workloads - numeric precision formats, the VNNI layout, and Intel AMX/TPP performance on Granite Rapids

  1. Everything on Precision: Numeric Formats in Modern ML and HPC
  2. VNNI Data Layout for Intel AMX Matrix Multiplication

CPU Microarchitecture

A deep dive into CPU microarchitecture on Intel Granite Rapids - cache coherence and MESI, hardware/software prefetching, and IOMMU-based device isolation

  1. Cache Coherence Demystified: The MESI Protocol on Intel Granite Rapids

CXL Series

A complete guide on CXL (Compute Express Link) technology and its applications

  1. CXL Introduction

Locking primitives

A complete guide on different locking primitives used in Linux Kernel, C, and C++

  1. Spin or Sleep: That is the question
  2. Read-Copy-Update (RCU): A Primer

LLM Training

A series on LLM training internals - the end-to-end training loop, PyTorch distributed training and profiling, and neural network quantization

  1. LLM Training: The End-to-End Flow
  2. Tiling and Collective Operations
  3. PyTorch Distributed Training Internals
  4. Activations, Normalization and Optimizers, in Plain Terms
  5. Neural Network Quantization

Past Series

Performance Profiling

A practical guide to Linux performance profiling - perf stat/record internals, adding custom /proc counters, and GPU memory profiling tools

  1. Understanding Linux perf: stat and record
  2. Adding a counter to the *proc* interface
  3. GPU Memory Profiling Tools (NVIDIA and Intel)