IOMMU: The Gatekeeper Between Devices and Memory

8 minute read

Published: March 03, 2026 Last Updated: March 03, 2026

iommu virtualization dma hardware

IOMMU: The Gatekeeper Between Devices and Memory

What is IOMMU?

IOMMU stands for Input/Output Memory Management Unit. Just like the CPU has an MMU (Memory Management Unit) that translates virtual addresses to physical addresses for software, the IOMMU does the same thing — but for hardware devices performing DMA (Direct Memory Access).

When a network card, GPU, or NVMe drive wants to read/write system memory directly (DMA), the IOMMU intercepts those accesses and translates the device-virtual addresses (sometimes called IOVA — I/O Virtual Addresses) into physical addresses, enforcing access control along the way.

Where Does It Sit?

┌──────────────────────────────────────────────────────────────────┐
│                        CPU Package                               │
│                                                                  │
│   ┌─────────┐    ┌─────────┐                                    │
│   │  Core 0  │    │  Core N  │  ... (each with MMU/TLB)         │
│   └────┬─────┘    └────┬─────┘                                   │
│        │               │                                         │
│        └───────┬───────┘                                         │
│                │                                                 │
│         ┌──────▼──────┐                                          │
│         │   Last Level │                                         │
│         │    Cache     │                                         │
│         └──────┬──────┘                                          │
│                │                                                 │
│    ┌───────────▼────────────┐                                    │
│    │    Memory Controller   │                                    │
│    └───────────┬────────────┘                                    │
│                │                                                 │
│    ┌───────────▼────────────┐     ┌──────────────────────┐       │
│    │       ◆ IOMMU ◆        │◄────│  I/O Page Tables     │       │
│    │   (Intel VT-d /        │     │  (per-device or      │       │
│    │    AMD-Vi unit)         │     │   per-domain)        │       │
│    └───────────┬────────────┘     └──────────────────────┘       │
│                │                                                 │
│    ┌───────────▼────────────┐                                    │
│    │      PCIe Root         │                                    │
│    │      Complex           │                                    │
│    └───┬────────┬───────┬───┘                                    │
│        │        │       │                                        │
└────────┼────────┼───────┼────────────────────────────────────────┘
         │        │       │
    ┌────▼──┐ ┌───▼──┐ ┌──▼───┐
    │  NIC  │ │ GPU  │ │ NVMe │   ... PCIe devices doing DMA
    └───────┘ └──────┘ └──────┘

The IOMMU is integrated into the chipset or uncore region of the CPU package (on modern Intel/AMD processors, it’s part of the SoC). It sits on the path between PCIe devices and main memory. Every DMA transaction from a device passes through the IOMMU before reaching DRAM.

On our test system (Intel Xeon 6980P, Granite Rapids), there are multiple VT-d (DMAR) units — one per memory controller / PCIe root port cluster — totaling 155 IOMMU groups.

How Does It Work?

Address Translation (I/O Page Tables)

The IOMMU maintains I/O page tables — similar in structure to CPU page tables (4-level or 5-level):

Device DMA Request
   │
   ▼
┌──────────────────┐
│ Device sends      │
│ IOVA: 0xDEAD0000 │
└────────┬─────────┘
         │
         ▼
┌──────────────────────┐
│      IOMMU           │
│                      │
│  1. Check IOTLB      │ ◄── Cached translations (fast path)
│     (I/O TLB cache)  │
│                      │
│  2. If miss: walk     │ ◄── I/O page table walk (slow path)
│     I/O page tables   │
│                      │
│  3. Check permissions │ ◄── Read/Write/Execute bits
│                      │
│  4. Translate IOVA    │
│     → Physical Addr   │
└────────┬─────────────┘
         │
         ▼
┌──────────────────┐
│ Physical memory   │
│ addr: 0x1234_5000 │
└──────────────────┘

IOTLB — The Critical Cache

The IOTLB (I/O Translation Lookaside Buffer) caches recent address translations. It’s typically small (hundreds of entries), so:

4KB pages → each entry covers 4KB → many misses for large DMA buffers
2MB huge pages → each entry covers 2MB → dramatically fewer misses
1GB pages → even better for massive DMA buffers (GPUs, RDMA)

IOTLB misses are expensive — each miss triggers a multi-level page table walk over PCIe, which can cost hundreds of nanoseconds.

IOMMU Groups

An IOMMU group is the smallest set of devices that the IOMMU can isolate. Devices within the same group can potentially see each other’s DMA traffic. This matters for:

VFIO passthrough: You pass an entire IOMMU group to a VM, not individual devices
Security isolation: Devices in separate groups are fully isolated

Use Cases

1. Virtualization (Device Passthrough)

The primary use case. When you pass a physical PCIe device (GPU, NIC, NVMe) directly to a VM using VFIO:

The IOMMU ensures the VM’s device can only access memory assigned to that VM
Without IOMMU, a passed-through device could DMA into the host’s memory → security disaster

┌─────────────┐  ┌─────────────┐
│    VM 1      │  │    VM 2      │
│  (has GPU)   │  │  (has NIC)   │
│  IOVA space  │  │  IOVA space  │
└──────┬───────┘  └──────┬───────┘
       │                 │
       ▼                 ▼
┌──────────────────────────────┐
│          IOMMU               │
│  GPU → only VM1's memory     │
│  NIC → only VM2's memory     │
└──────────────────────────────┘

2. DMA Protection / Security

Even without VMs, the IOMMU protects against:

Rogue DMA attacks (e.g., via Thunderbolt/FireWire — “DMA attacks”)
Buggy device firmware that might write to wrong memory regions
Side-channel attacks from malicious PCIe devices

3. Device Address Space > Physical Memory

The IOMMU can present a contiguous virtual address range to a device even when physical memory is fragmented. This is crucial for devices that need large contiguous DMA buffers (older NICs, some accelerators).

4. Shared Virtual Memory (SVM)

Modern IOMMU features (like Intel Scalable Mode) allow devices to share the CPU’s page tables directly — the device sees the same virtual addresses as the process. This enables:

Simpler device programming (no explicit DMA mapping)
On-demand paging for devices (page faults handled by OS)

Pros and Cons

Pros

Benefit	Detail
Security	Prevents rogue DMA attacks; isolates devices from each other and from host memory
Virtualization	Enables safe device passthrough with near-native I/O performance
Memory flexibility	Scatter-gather DMA without requiring physically contiguous buffers
Fault isolation	A misbehaving device gets an IOMMU fault instead of corrupting random memory
Address space	32-bit devices can access memory above 4GB via IOMMU remapping (the “bounce buffer” problem goes away)

Cons

Drawback	Detail
Latency overhead	Every DMA transaction goes through IOMMU translation. IOTLB misses add ~100-500ns per access
IOTLB pressure	Small IOTLB + many 4KB mappings = thrashing. High-bandwidth devices (100GbE NICs, GPUs) can suffer
Map/unmap cost	`dma_map/dma_unmap` kernel API calls are more expensive with IOMMU (page table updates + IOTLB invalidations)
Complexity	IOMMU group granularity can be annoying (can’t pass individual functions of a multi-function device)
Throughput impact	For high-PPS networking workloads, IOMMU overhead can reduce throughput by 5-15%

Performance Impact in Numbers (Typical)

Metric	Without IOMMU	With IOMMU (4K pages)	With IOMMU (2M pages)
DMA map latency	N/A	~200-500ns	~100-200ns
IOTLB miss cost	N/A	~300-800ns	~200-400ns
Network PPS (small packets)	Baseline	-5% to -15%	-2% to -5%
NVMe IOPS	Baseline	-3% to -8%	-1% to -3%

Kernel Parameters

Key kernel command-line parameters affecting IOMMU:

# Enable/disable IOMMU (Intel)
intel_iommu=on       # Force enable
intel_iommu=off      # Disable

# Passthrough mode (devices not behind IOMMU get identity mapping)
iommu=pt             # Best performance for non-passthrough devices

# Strict vs lazy TLB invalidation
iommu.strict=0       # Lazy (batched) — better performance, slightly less secure
iommu.strict=1       # Strict — immediate invalidation after every unmap

# Force identity mapping for all devices
iommu.passthrough=1

How to Inspect IOMMU on Linux

# Check if IOMMU is enabled
dmesg | grep -i iommu

# List IOMMU groups and their devices
for g in /sys/kernel/iommu_groups/*/; do
    echo "IOMMU Group $(basename $g):"
    for d in $g/devices/*; do
        echo "  $(lspci -nns $(basename $d) 2>/dev/null)"
    done
done

# Check VT-d version
cat /sys/class/iommu/dmar*/intel-iommu/version

# IOMMU domain info
cat /sys/kernel/iommu_groups/*/type

On Our System

Our test server (Intel Xeon 6980P, Granite Rapids):

VT-d version: 7.0 (latest, with Scalable Mode support)
IOMMU groups: 155
Domain type: Translated (full IOMMU translation active)
TLB invalidation: Lazy mode (batched for performance)
NUMA nodes: 3 (including CXL-attached memory on node 2)

The IOMMU is particularly interesting here because with CXL memory (node 2, 512GB), DMA traffic can target either local DRAM or CXL-attached memory, and the IOMMU handles translation for both paths.

Summary

The IOMMU is the “firewall for DMA”. It sits between I/O devices and system memory, translating addresses and enforcing access control. It’s essential for secure virtualization and device passthrough, but comes with a small performance tax — especially for latency-sensitive, high-throughput I/O workloads. Using larger pages (2MB/1GB) for DMA buffers and lazy TLB invalidation can significantly reduce the overhead.

Written as part of the Granite Rapids Exploration project, March 2026.