All posts

AI & ML

LLM Inference Introduction In Progress

4 articles

Concepts behind LLM inference - the end-to-end inference flow, the KV cache and attention variants, numeric precision, GPU kernels in Triton, and FlashAttention

  1. LLM Inference Introduction: the end-to-end flow and vector math
  2. The KV Cache: Sizing It, and the Attention Variants That Shrink It
  3. Everything on Precision: Numeric Formats in Modern ML and HPC
  4. FlashAttention: Exact Attention Without Materialization

LLM Inference Serving In Progress

5 articles

Challenges and solutions in serving LLMs - PagedAttention, FlashAttention in practice, FlashInfer, continuous batching, CUDA Graphs, and agentic/MoE workloads

  1. PagedAttention
  2. FlashAttention in Practice: Four Schedules and One Branch
  3. FlashInfer: A Customizable Attention Engine for LLM Serving
  4. Continuous Batching: Who Decides What Runs Next
  5. Agentic Inference and Mixture-of-Experts: Two Levels of Conditional Execution

A guide to CPU instruction-set features for ML workloads - the VNNI layout and Intel AMX/TPP performance on Granite Rapids

  1. VNNI Data Layout for Intel AMX Matrix Multiplication

Hardware & Architecture

A deep dive into CPU microarchitecture on Intel Granite Rapids - cache coherence and MESI, hardware/software prefetching, and IOMMU-based device isolation

  1. Why Innocent Shared Updates Stop Scaling: Cache Coherence on a Multi-Socket Server

CXL Series

1 article

A complete guide on CXL (Compute Express Link) technology and its applications

  1. CXL Introduction

Performance Engineering

GPU Performance Profiling In Progress

4 articles

Practical tools and workflows for profiling GPU utilization, memory usage, and performance bottlenecks

  1. Reading torch.profiler Correctly: Schedules, Labels, and CPU vs GPU Time
  2. GPU Profiling with Nsight Systems
  3. Introduction to GPU Profiling
  4. PyTorch Profiler vs Nsight Systems

Performance Profiling Completed

2 articles

A practical guide to Linux performance profiling - perf stat/record internals and adding custom /proc counters

  1. Understanding Linux perf: stat and record
  2. Adding a counter to the *proc* interface

Systems Software

A complete guide on different locking primitives used in Linux Kernel, C, and C++

  1. Spin or Sleep: That is the question
  2. Read-Copy-Update (RCU): A Primer