Learning Series

Multi-part deep-dive series, in reading order

Active Series

LLM Inference

A series on LLM inference internals - the end-to-end inference flow, FlashAttention and PagedAttention, and agentic workflows with Mixture-of-Experts

  1. LLM Inference Introduction: the end-to-end flow and vector math
  2. The KV Cache: Sizing It, and the Attention Variants That Shrink It
  3. PagedAttention
  4. FlashAttention: Exact Attention Without Materialization
  5. The Evolution of FlashAttention: From Ampere to Blackwell
  6. FlashInfer: A Customizable Attention Engine for LLM Serving
  7. Agentic Inference and Mixture-of-Experts: Two Levels of Conditional Execution

CPU ISA for ML

A guide to CPU instruction-set features for ML workloads - numeric precision formats, the VNNI layout, and Intel AMX/TPP performance on Granite Rapids

  1. Everything on Precision: Numeric Formats in Modern ML and HPC
  2. VNNI Data Layout for Intel AMX Matrix Multiplication

CPU Microarchitecture

A deep dive into CPU microarchitecture on Intel Granite Rapids - cache coherence and MESI, hardware/software prefetching, and IOMMU-based device isolation

  1. Cache Coherence Demystified: The MESI Protocol on Intel Granite Rapids

CXL Series

A complete guide on CXL (Compute Express Link) technology and its applications

  1. CXL Introduction

Locking primitives

A complete guide on different locking primitives used in Linux Kernel, C, and C++

  1. Spin or Sleep: That is the question
  2. Read-Copy-Update (RCU): A Primer

LLM Training

A series on LLM training internals - the end-to-end training loop, PyTorch distributed training and profiling, and neural network quantization

  1. LLM Training: The End-to-End Flow
  2. Tiling and Collective Operations
  3. PyTorch Distributed Training Internals
  4. Activations, Normalization and Optimizers, in Plain Terms
  5. Neural Network Quantization

GPU Performance Profiling

Practical tools and workflows for profiling GPU utilization, memory usage, and performance bottlenecks

  1. Reading torch.profiler Correctly: Schedules, Labels, and CPU vs GPU Time

Past Series

Performance Profiling

A practical guide to Linux performance profiling - perf stat/record internals and adding custom /proc counters

  1. Understanding Linux perf: stat and record
  2. Adding a counter to the *proc* interface