Posts

What each step of an LLM training iteration computes, what it costs in memory and arithmetic, and where the loop stalls on real hardware.

Deep dive into cache coherence and the MESI protocol on a 256-core Intel Granite Rapids system: microbenchmarks for ping-pong latency, false sharing, atomic contention, and lock primitives.

A walkthrough of numeric formats that matter today — FP32, TF32, FP16, BF16, FP8, INT8, INT4 — how they behave, when to use each, and where the pitfalls hide.

LLM Basic

What really happens in the kernel when you run perf stat or perf record.

In this post, we discuss the internals of a FAT based file system. FAT stands for File Allocation Table, which is also the name of the central data structure of this design. It is one of the oldest file system designs still in use – your pen drive is probably formatted with some variant of it (FAT32 or exFAT). The layout A FAT file system has a fairly simple on-disk layout: ...

In this post we are exploring how different types of binaries are generated, how function calls work when code is split across multiple files, and how the toolchain (compiler, linker, loader) helps. We’ll also cover how to build and install a custom libc for testing and debugging purposes. File Types and High-Level Flow Difference in types of files: .c / .cpp: source files .o: object files (compiled but not yet linked) .a: static library (archive of .o files) .so: shared library (dynamically linked at runtime) Typical build pipeline: ...

What the KV cache costs per token, and how MQA, GQA and MLA rewrite attention to make that number smaller.

The two data-movement primitives behind training performance: tiling inside a device, collectives between devices.

A technical tutorial on Intel AMX matrix multiplication, the VNNI layout requirement, its silent correctness failure mode, and LIBXSMM dispatch.

A practical guide to instrumenting the Linux kernel by adding custom counters to /proc/vmstat

In this post, we are going to discuss the internals of an inode based file system. There are different variants of inode: i-node, I-node, and, of course, inode. We are going to be using “inode” for this post. What is an inode? Inode is a data structure, used by modern file systems, such as ext4, to track the contents of a file across the disk. An inode is associated with every file and directory. Technically, from the point of view of a file system, there is no difference between the inode of a file or a directory. They are basically the same, with a flag in the inode stating whether it is pointing to a file or a directory. ...

Glibc is the standard library that is linked against all the linux applications. It provides the necessary functionality like printf. It also provides wrappers for most used system calls like open, read, write, close. Manually building and installing libc. Example from a StackOverflow answer export glibc_install="$(pwd)/glibc/build/install" git clone git://sourceware.org/git/glibc.git cd glibc git checkout glibc-2.28 mkdir build cd build ../configure --prefix "$glibc_install" make make install Followed by a test c code ...

The mechanism behind step 5 of the training loop: bucketed gradient all-reduce triggered from autograd hooks, what NCCL does with it, and how to diagnose the result.

How PagedAttention uses virtual-memory-style KV-cache paging to reduce fragmentation and increase LLM serving throughput.