Chapter 6 — Virtualization and the Shape of the Machine

“All problems in computer science can be solved by another level of indirection.” — David Wheeler

“These are a shadow of the things to come, but the substance belongs to Christ.” — Colossians 2:17 (ESV)

Why This Matters

For five weeks you have administered the machine you could point at — a host with a name, a NIC with a MAC, a disk with a serial number. This week the machine stops being a thing you can point at. A “server” becomes a configuration; a “GPU” becomes a slice of a slice; the host the operating system thinks it owns is an illusion the hypervisor maintains, frame by frame, so convincingly that the guest kernel never suspects. Virtualization is the discipline of running that illusion on purpose — and running it well enough that production trusts it.

You already know the payoff at the CPU layer. One physical box runs forty virtual machines; you snapshot one in a second, live-migrate another off a host you need to patch, and overcommit memory because not every guest is hungry at once. That world is mature. It is not where the difficulty has moved.

The difficulty has moved to the GPU. A modern AI accelerator — an NVIDIA H100, H200, or a Blackwell B200 — is a single device worth more than the rack it sits in, and it does not virtualize the way a CPU does. You cannot casually overcommit it. You cannot hand the same 80 GB of HBM to four tenants and hope. So a whole family of GPU-sharing techniques exists — passthrough, time-slicing, MPS, MIG, vGPU — each making a different, unequal promise about isolation, and each fitting a different workload. Choosing among them is the central administration decision of the AI era, and it is this chapter.

Both threads of this book run straight through here. AI as the tool you wield: placement and right-sizing — deciding which workload lands on which host with how much memory — is exactly the kind of bin-packing an AI capacity planner now drafts for you in seconds, and exactly the kind of confident draft you must check before it strands a 70B model on a card that cannot hold it. AI as the workload you govern: the GPU you are carving up is the AI workload. Every partition you cut is a decision about how much of a scarce, expensive, hard-to-share resource each model gets, and whether a tenant who faults the card can take its neighbors down.

The Christian question this week is the oldest one virtualization raises: what is shadow, and what is substance? Paul tells the Colossians that the old observances were “a shadow of the things to come, but the substance belongs to Christ” (ESV). A virtual machine is a shadow cast by real silicon — and a useful, load-bearing shadow, not a lie. But a shadow has limits the substance does not, and the administrator who forgets which is which ships an outage. We will hold that question through every layer: the vCPU that is real compute but not a real core, the time-slice that looks like a GPU but isolates nothing, the snapshot that captures a state but not the world that state assumed.

6.1 — The Hypervisor: The Layer That Casts the Shadow

A hypervisor (or virtual machine monitor) is the thin layer that lets one physical host present itself as many. It intercepts the privileged instructions a guest kernel believes it is running on bare metal, mediates access to real CPU, memory, and devices, and keeps each guest convinced it is alone. The whole edifice rests on hardware help: Intel VT-x / AMD-V for CPU virtualization and Intel VT-d / AMD-Vi for the IOMMU that makes safe device assignment possible. Remember those acronyms — VT-d/AMD-Vi return in §6.6 as the thing that makes GPU passthrough safe.

The classic split is Type 1 vs Type 2:

Every cloud you have ever touched is a Type 1 hypervisor with a billing system bolted on. KVM in particular is the one to know cold: it turns the Linux kernel itself into a Type 1 hypervisor, it is what most of the open infrastructure world runs, and it is the substrate under KubeVirt — the project that lets Kubernetes schedule full VMs, which is how the container world and the VM world meet on the same GPU node in §6.9.

Coach’s Note — “Type 1 vs Type 2” is a real distinction, but don’t fetishize it. The line that pays your rent is who is in the data path between the guest and the silicon, and how many privileged transitions that costs. A Type 1 hypervisor wins production because it short-circuits that path. When you debug a slow VM, you are almost always debugging an unexpectedly long path — an emulated device where you assumed a passed-through one, a host OS where you assumed bare metal. Find the path; the overhead is on it.

6.2 — vCPU, Memory, and the Honest Lie of Overcommit

The CPU and memory story is mature, and you must be fluent in it before the GPU story will make sense — because the GPU story is largely the CPU story with the safety nets removed.

A vCPU is not a core. It is a schedulable thread of execution that the hypervisor time-shares onto real physical cores. This is why you can give a 32-core host out as far more than 32 vCPUs: most guests are idle most of the time, so the hypervisor oversubscribes the physical cores and schedules whoever is actually runnable. The ratio you can get away with — vCPU:pCPU overcommit — is a real engineering number. For general server VMs, 3:1 or 4:1 is routine; for latency-sensitive or steadily-busy workloads it drops toward 1:1. Push it too far and you get CPU ready time — the guest is runnable but waiting for a physical core — which shows up as mysterious latency the guest’s own tools cannot explain, because from inside the guest the CPU looks idle. It is idle. It is idle because it is waiting in line.

Memory overcommit is the same bet with sharper teeth. The hypervisor hands out more guest RAM than the host physically has, betting not every guest touches its ceiling at once. It reclaims with ballooning (a guest driver gives pages back under pressure), transparent page sharing (dedup identical pages across guests), and, when those run out, swapping at the host level — which is where overcommit goes to die, because host swap is orders of magnitude slower than RAM and the guest has no idea why it stalled.

Coach’s Note — Overcommit is a probabilistic promise, and that is the whole lesson of this chapter compressed into one word. CPU and memory overcommit work because the resource degrades gracefully: oversubscribe a little too much and things get slow, not dead. Hold that thought hard, because GPU memory does not degrade gracefully. When you try to fit a model into VRAM that isn’t there, you don’t get slow — you get a CUDA out-of-memory error and a dead process. Half of why GPU virtualization is its own discipline is that the friendly, forgiving overcommit you lean on for CPU and RAM is off the table for VRAM. The shadow has a hard edge here.

6.3 — Snapshots and Live Migration: Capturing a State, Moving a Shadow

Two more capabilities define why virtualization won the data center, and both carry a quiet warning.

A snapshot captures a VM’s state — disk, and optionally memory and device state — at an instant, so you can roll back to it. It is the cheapest “undo” in infrastructure: snapshot before a risky upgrade, snapshot before you let an AI agent run a migration. But a snapshot captures the machine’s state, not the world’s. Roll a VM back twenty minutes and its clock, its TLS session tickets, its database replication position, and its neighbors’ expectations all snap back too — except the world moved on. Snapshots of database VMs are a classic way to resurrect data you had correctly deleted, or to replay a transaction the rest of the system thinks already happened. Use them for short-lived “undo,” not as backups (that is Chapter 11’s distinction, and it matters).

Live migration moves a running VM from one host to another with no perceptible downtime, by pre-copying its memory pages to the destination while it keeps running, then doing a final sub-second cutover. It is how you patch a hypervisor under a thousand running guests without a maintenance window. The catch returns in §6.6: a VM with a passed-through GPU usually cannot be live-migrated, because the hypervisor cannot copy the opaque state living inside a physical device it handed away. The moment you give a guest real silicon, you trade away the mobility that made virtualization magic. That trade is the spine of the GPU section.

6.4 — AI as the Tool: Capacity, Placement, and Right-Sizing

Here is the first AI thread, in the area where it is genuinely useful and genuinely dangerous: deciding what runs where.

Placement is bin-packing. You have a fleet of hosts with finite vCPU, RAM, and VRAM; you have a queue of workloads, each with a demand and a priority; you want a packing that maximizes utilization without starving anyone or stranding a workload it cannot serve. The cluster schedulers already do a version of this — VMware DRS has balanced VMs across hosts for years — and the 2026 layer on top is an AI capacity planner that reads your historical utilization, forecasts demand, and recommends a placement and a right-size in plain language: “VM app-07 has run under 15% CPU for 30 days; drop it from 8 vCPU to 2 and reclaim it; move the H100 job from node-3 to node-9 where a full card is free.”

That recommendation is worth real money — idle GPUs are the most expensive idle resource in the building — and it is exactly the kind of confident draft you must verify. The failure mode is specific: an AI planner reasons over the numbers it was given, and VRAM is the number it is most likely to get subtly wrong. It will happily recommend placing a “70B model” onto “a free 48 GB card” because 48 looks like enough next to the weights-only figure — forgetting the KV cache and framework overhead that push real deployed VRAM 15–40% above weights (the math you sized in Chapter 4). Follow that recommendation and you don’t get a warning; you get an out-of-memory crash at the worst possible moment. The planner does the arithmetic; you own whether the arithmetic was complete.

Coach’s Note — The right way to use an AI capacity planner is the right way to use any confident-fast partner: let it do the enumeration you’d be slow and bored at — “which of my 200 VMs are oversized, ranked by reclaimable cost” — and keep for yourself the judgment it can’t see — “but app-07 is oversized on purpose because it spikes 40× during Sunday services and you don’t have its event calendar.” The planner sees the metrics. You see the ministry. Right-sizing without that context is how an agent reclaims the headroom that exists for a reason.

6.5 — AI as the Workload: Why a GPU Doesn’t Virtualize Like a CPU

Now the second thread, which is the heart of the week. The accelerator you are trying to share is the workload you are trying to govern, and it resists sharing for concrete architectural reasons.

A CPU virtualizes cleanly because the hardware was designed to be time-shared and protected: privilege rings, an MMU that gives every process its own address space, and decades of hypervisor support (VT-x, VT-d). A data-center GPU was designed to be fed — to run one enormous, coherent, parallel workload as fast as possible — not to be carved among mutually distrustful tenants. Its memory (HBM) is fast and finite and, crucially, not overcommittable: there is no friendly swap, no ballooning, no graceful degradation. When the VRAM is gone, the next allocation dies.

So GPU sharing splits into a spectrum, ordered by how much isolation the hardware actually enforces. Memorize this ordering; it is the chapter:

The single most important line in that table is the isolation gap between time-slicing/MPS and MIG. Time-slicing and MPS look like sharing a GPU, and for a friendly room of cooperating processes they are fine. But they isolate nothing meaningful: under time-slicing all clients share the same VRAM with no caps, so one greedy notebook OOMs the card for everyone; under MPS a single fatal fault in one client can reset the GPU for all of them. MIG is categorically different — it partitions the silicon itself, so each instance has its own memory, its own compute, and its own fault domain. A neighbor crashing a MIG instance does not touch yours. That is the line between “shared dev box” and “multi-tenant production,” and you will draw it correctly or you will get paged.

Coach’s Note — The apartment-building analogy is the one to keep. Passthrough is leasing the whole building to one tenant — total control, total cost, no sharing. MIG is a building legally subdivided into condos with fireproof walls: your neighbor’s kitchen fire stays in your neighbor’s condo. MPS is roommates with a chore chart — they cooperate, but one of them can still burn the place down. Time-slicing is a hotel room rented by the hour with no lock on the door. When someone proposes “let’s just share the GPU,” your only correct response is which one of these four do you mean — because they are four completely different promises wearing one word.

6.6 — Passthrough: Giving Away a Whole GPU

GPU passthrough assigns one physical GPU directly to one VM, end to end, at near-bare-metal performance (the overhead is a community/vendor estimate of “low single-digit percent” — keep it qualitative). It is the simplest mode conceptually and the right answer whenever a single workload wants the whole card: a training run, or a latency-critical inference service that must not share.

It works through VFIO (Virtual Function I/O) and the IOMMU (VT-d / AMD-Vi from §6.1). You unbind the GPU from the host’s NVIDIA driver and bind it to the vfio-pci stub driver; the IOMMU then lets the guest DMA safely to the device without being able to reach the rest of host memory. On Linux the shape is:

# 1) Confirm IOMMU is on (kernel cmdline: intel_iommu=on  OR  amd_iommu=on)
dmesg | grep -e DMAR -e IOMMU

# 2) Find the GPU's PCI address and its IOMMU group (the whole group moves together)
lspci -nnk | grep -iA3 nvidia          # e.g. 17:00.0 and vendor:device 10de:2330

# 3) Bind it to vfio-pci instead of the nvidia driver (the GPU leaves the host)
echo "10de:2330" | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id

# 4) Attach to a guest, e.g. with libvirt/QEMU:
#    <hostdev mode='subsystem' type='pci'> ... <address domain='0' bus='0x17' slot='0x00'/> ...

Inside the guest, nvidia-smi now shows a full card — and on the host, the GPU has disappeared. That disappearance is the whole point and the whole cost. Two consequences you must teach yourself to expect:

1. No sharing. One GPU, one VM. You cannot passthrough the same card to two guests.
2. No live migration (§6.3). The hypervisor can’t copy the opaque state inside a device it gave away, so a passed-through VM is usually pinned to its host. You traded mobility for raw performance.

A subtlety that bites in practice: PCI devices pass through in IOMMU-group units. If the GPU shares a group with other devices, the whole group must move together — which is why server boards meant for virtualization put each GPU in its own group, and why this can be miserable on consumer hardware.

One more piece of vocabulary that returns in vGPU: SR-IOV (Single Root I/O Virtualization) is the PCIe mechanism that lets one physical device present multiple virtual functions (VFs) to the system, each assignable to a different guest. Passthrough hands over the whole physical function; SR-IOV is how a single card can be carved into several pass-through-like functions. On Ampere-and-later NVIDIA GPUs, SR-IOV must be enabled in BIOS before vGPU will work — hold that, because §6.8 leans on it.

6.7 — MIG: Carving One GPU Into Real, Walled-Off GPUs

Multi-Instance GPU (MIG) is the technique that makes a single H100/H200/B200 (and A100 before them) behave like up to seven smaller, truly isolated GPUs. It partitions the physical resources — streaming multiprocessors, memory controllers, cache, HBM — into instances with hardware-enforced fault and performance isolation. This is the mode that makes serious multi-tenant AI hosting possible: you can put two customers, or two teams, on one card and mean it when you say they can’t see each other.

The facts you must own and not get wrong:

• The ceiling is 7 instances — on the A100, H100, H200, and B200 alike. Seven compute slices, eight memory slices. This holds even on a 180 GB B200: more memory per slice, not more slices. (An earlier “up to 8 on B200” claim was an error — there are 8 distinct profile entries but a maximum of 7 simultaneous instances.)
• Profile syntax is [compute]g.[memory]gb. On a B200 (180 GB) the profiles run 1g.23gb (up to 7 of them), 2g.45gb, 3g.90gb, up to 7g.180gb (the whole card as one instance). On an H100 80 GB you’ll see 1g.10gb, 2g.20gb, 3g.40gb, 7g.80gb, and mixes thereof.
• Slices, not memory, are the budget. A 3g instance plus four 1g instances spends 3 + 1 + 1 + 1 + 1 = 7 — full. You can mix sizes, but the slices must sum to ≤ 7.

Enabling MIG resets the GPU, so you do it on an idle card, never on one with live work. The mechanics:

# Enable MIG mode on GPU 0 (resets the device — no live work on it!)
sudo nvidia-smi -i 0 -mig 1

# See which GPU Instance profiles this card supports (the IDs differ per card)
sudo nvidia-smi mig -i 0 -lgip

# Create a 3g.40gb + four 1g.10gb mix on an H100 (profile IDs from -lgip; -C also
# creates the Compute Instances). Total slices: 3 + 1 + 1 + 1 + 1 = 7.
sudo nvidia-smi mig -i 0 -cgi 9,19,19,19,19 -C

# List the instances; each is its own MIG-<UUID> device
sudo nvidia-smi mig -i 0 -lgi
nvidia-smi -L

The reference script code/gpu_partition.sh wraps exactly this with safety checks, and code/sample-nvidia-smi.txt is a captured layout you can practice reading offline — note how each MIG device gets its own memory column, and how instance 1 simply cannot see instance 13’s memory. That separate column is the isolation, made visible.

Sizing a slice to a model is the arithmetic you’ll do most, so here it is as a table for an H100 80 GB. Read it as: the slice must hold the model’s real deployed VRAM — weights plus KV cache plus 15–40% framework overhead (Chapter 4) — not just the weights.

The trap is the second column’s parentheticals. “An 8B model” sounds like it fits a 1g.10gb slice until you remember 8B at FP16 is ~16 GB of weights alone. Size the slice to the deployed number, every time.

To pin a workload to one instance, you address it like its own device:

# vLLM serving a small model on a single 1g.10gb slice — it sees only that slice
CUDA_VISIBLE_DEVICES=MIG-<uuid-of-1g.10gb> \
  vllm serve <model> --max-model-len 8192

Coach’s Note — MIG is where “I virtualized a GPU” earns the word production. Time-slicing a GPU for your dev team is an afternoon; MIG-partitioning a card so two paying tenants share it with hardware isolation is the real skill, and the failure mode is always the same overconfident mistake — sizing the model but not the slice. A 1g.10gb instance has ~10 GB. A 13B model at FP16 wants ~26 GB before the KV cache. It will not fit, MIG will not magically make it fit, and the only thing isolation buys you is that your tenant’s crash is contained instead of contagious. Size the slice to the workload. The seven-slice ceiling is a fence; don’t try to climb it.

On Blackwell there is a further layering worth knowing exists (confirm against the current vGPU user guide before relying on it): vGPU can be placed on top of MIG instances — e.g., a 1g.23gb MIG instance exposed as a B200X-1-23C vGPU type — combining hardware partition with VM-level sharing. And note the device-plugin constraint that trips people up on Kubernetes: time-slicing and MPS are mutually exclusive, MPS is unsupported on MIG-enabled devices, but MIG + time-slicing can combine (time-slice within a slice). A KubeVirt GPU node, meanwhile, runs only one mode at a time — containers, or passthrough VMs, or vGPU VMs — never a buffet.

6.8 — vGPU and Scheduling AI Against Everything Else

NVIDIA vGPU is the licensed mediated-sharing path: a host-side manager carves a physical GPU into virtual GPUs that full VMs consume, each VM getting a slice of memory and a scheduled share of compute. It is the natural fit for VDI (many virtual desktops needing a little acceleration) and for mixed estates where the consumers are VMs rather than containers. Two facts to carry:

• It is licensed software — the modern product line is NVIDIA vGPU, with vGPU 20 as the Production Branch (March 2026) and vGPU 19 as the long-term branch supported into July 2028; the old “NVIDIA GRID” brand is retired. On Ampere-and-later, vGPU requires SR-IOV enabled in BIOS.
• It is the one sharing mode built for VMs from the start — where MIG and time-slicing primarily serve containers/processes, vGPU’s whole reason to exist is letting many guest VMs share one card.

Underneath all of this sits the scheduling question that ties the chapter to everyday administration: how do AI workloads share a cluster with traditional ones? AI jobs are not like web servers. A training run is a long, greedy, all-or-nothing consumer that wants every cycle for hours; an inference service is latency-sensitive and bursty; a nightly batch embedding job is interruptible. Traditional VMs are steady and modest. Mixing them on shared infrastructure forces three levers you already half-know from OS scheduling, now at cluster scale:

• Priority — production inference outranks a researcher’s experiment when the card is contended.
• Preemption — a low-priority, checkpointable training job can be evicted so a high-priority job can run, then resumed. Make sure the job actually checkpoints, or preemption is just data loss with extra steps.
• Isolation — the reason MIG exists: so a runaway training job cannot starve the inference service sharing its hardware. Isolation is what makes co-scheduling safe instead of merely possible.

The decision tool code/gpu_sharing_picker.py encodes these rules so you can argue with them; the YAML in code/time-slicing-config.yaml shows what oversubscription (not partitioning) actually looks like in a device-plugin config — read its warning comment twice.

6.9 — Where the VM World Meets the Container World

You will spend most of the AI era running GPUs under containers, not full VMs — that is next week’s chapter. But containers and VMs are not rivals so much as two depths of the same idea, and where they meet on a GPU node is worth seeing now, because it constrains how you carve the card.

A VM virtualizes the whole machine — its own kernel, its own everything, isolated by the hypervisor. A container virtualizes the operating-system view — same host kernel, isolated by namespaces and cgroups, far lighter. For the GPU, the decision tree is the same one we’ve been drawing: a passed-through or vGPU GPU goes to a VM; a MIG instance or a time-slice is most often advertised to containers through the NVIDIA Kubernetes device plugin and GPU Operator.

The project that stitches the two worlds together is KubeVirt — it lets Kubernetes schedule full VMs alongside containers, on KVM underneath. And it carries a constraint you must design around: a KubeVirt GPU node runs only one mode at a time — containers, or passthrough VMs, or vGPU VMs. You don’t get a buffet on a single node. So “we’ll run our training VMs and our inference containers on the same GPU box” is usually a sentence about two pools of nodes, not one. Knowing that before you promise it is the difference between an architecture and a wish.

Coach’s Note — Don’t let the cleaner, lighter container model fool you into thinking it’s more isolated. It’s less — a shared kernel is a bigger blast radius than a hypervisor boundary. The whole reason MIG exists is to put hardware isolation underneath the lightweight container sharing on top, so you get container ergonomics without container-grade trust. That stack — MIG slice as the hard wall, container as the convenient wrapper — is the dominant production pattern for multi-tenant AI in 2026. Next week you’ll build the container half; this week you built the wall it stands on.

6.x — Interactive Lab: GPU Partition Visualizer

Below this chapter on the site is the GPU Partition Visualizer — an interactive panel that gives you one physical GPU and lets you carve it. Use it now; the table in §6.5 will not stick until you have felt the tradeoffs move.

Here is the drill. Start with the GPU whole and toggle through the four modes — passthrough, time-slicing, MPS, MIG, and vGPU. Watch three things change with each: how many VMs or clients can share the card, how much memory each one is actually guaranteed, and what the panel reports for isolation (none → soft → hardware). In passthrough you’ll see one tenant own everything. Flip to time-slicing and watch the client count climb while the isolation indicator stays red and the memory stays shared — that is the hotel-room-with-no-lock made visual. Then switch to MIG and carve a mix: try a 3g plus four 1g instances and watch the slice budget hit 7 and refuse the eighth. That refusal — the fence at seven, even when memory remains — is the single most important thing this widget teaches.

Then do the part that turns the toy into judgment: for each mode, the panel suggests the right workload. Put a 13B inference service, a multi-tenant SaaS with two distrustful customers, a researcher’s bursty notebook, and a full training run into the modes you think fit, and check yourself against §6.5 and the picker in code/gpu_sharing_picker.py. When you can predict — before you toggle — which mode the visualizer will recommend for a workload and why, you have the chapter. That prediction is exactly what Project 6 asks you to defend in prose.

6.10 — The Steward and the Servant: Shadow and Substance

Paul’s word to the Colossians is about religious observances, but it names a distinction the virtualization administrator lives inside every day: “These are a shadow of the things to come, but the substance belongs to Christ” (Colossians 2:17, ESV). A shadow is real — it is cast by something, it has a true shape, you can navigate by it — but it is not the substance, and treating it as the substance is the root of a particular kind of error.

A virtual machine is an honest shadow. It is cast by real silicon; its compute is real compute; it does true work and you can trust it in production. But it is not the bare metal, and the places where it differs from the substance are exactly where outages live. The vCPU is real execution but not a real core — forget that and you misread ready-time latency as an application bug. The snapshot captures a real state but not the world that state assumed — forget that and you resurrect deleted data. The time-slice presents a real GPU but isolates nothing — forget that and you sell a tenant a promise the hardware never made. In each case the sin is the same: mistaking the shadow for the substance, trusting the indirection past the point where it was load-bearing.

This is why MIG matters morally as well as technically. Time-slicing lets you say “each tenant has a GPU” while the substance — the hardware — enforces nothing. MIG makes the shadow honest: the isolation you promise is the isolation the silicon delivers. When you tell a paying ministry that their data is walled off from the church across the diocese sharing the same card, MIG is what lets that sentence be true rather than merely marketed. Stewardship, in the Lutheran sense, is being found faithful with what is not yours to make (1 Corinthians 4:2) — and you did not make this GPU, this hypervisor, this tenant’s trust. Faithfulness here is refusing to let a convenient shadow stand in for a substance you owe people.

And there is a humility in it. Every layer of indirection Wheeler praised is a layer between you and the truth of the machine — useful, even necessary, but never to be mistaken for the thing itself. The administrator who remembers which is shadow and which is substance keeps the system trustworthy. The one who forgets ships the shadow and calls it the substance, and the people downstream pay for the confusion. Hold the line where the judgment lives. The AI planner can pack the bins; only you can tell which promises are real.

6.11 — Common Pitfalls

Pitfall: Confusing time-slicing with partitioning — selling “shared GPU” as if it were isolated. Example: You set the device plugin’s time-slicing replicas: 4 and tell two teams they each “have a GPU.” Team A’s notebook allocates 70 GB, OOMs the card, and Team B’s inference service dies with it. Fix: If you need isolation, use MIG (hardware-enforced). Time-slicing is oversubscription with no memory or fault isolation — only for cooperative, same-trust, bursty dev work. Name the mode precisely.

Pitfall: Sizing the model but not the MIG slice (or the time-slice). Example: You carve 1g.10gb instances and try to serve a 13B model that needs ~26 GB at FP16. It will not load; isolation doesn’t conjure memory. Fix: Size the slice to the workload’s real VRAM (weights + KV cache + 15–40% overhead, per Chapter 4). Pick the smallest profile that actually holds the model at your target context length.

Pitfall: Expecting to live-migrate a VM that has a passed-through GPU. Example: You schedule hypervisor maintenance assuming you can evacuate every guest, then discover the GPU VMs are pinned and must take downtime. Fix: Plan passthrough/vGPU VMs as non-migratable by default. Build maintenance windows or checkpoint-and-reschedule for them; don’t assume the live-migration magic that works for plain VMs.

Pitfall: Enabling or disabling MIG on a GPU that has live work. Example: You run nvidia-smi -i 0 -mig 1 on a card mid-inference; the toggle resets the device and kills every running process on it. Fix: MIG mode changes reset the GPU. Drain the card first, toggle on an idle device, then schedule work onto the new instances.

Pitfall: Trusting an AI capacity planner’s placement without checking the VRAM math. Example: The planner moves a 70B-class job onto “a free 48 GB card” using weights-only sizing; in production the KV cache pushes it over and it OOMs under load. Fix: Treat the planner’s placement as a draft. Re-run the real VRAM equation (Ch. 4) including KV cache and framework overhead before you let any placement ship — automated or not.

Pitfall: Reaching for MPS when you need fault isolation. Example: You put two tenants’ inference processes under MPS for better concurrency; one hits a fatal CUDA fault and resets the GPU for both. Fix: MPS gives soft caps and concurrency, not fault isolation — a fatal fault can reset the device for all clients. For distrustful or production-critical tenants, use MIG.

Pitfall: Pushing CPU/memory overcommit ratios from the VM world onto VRAM. Example: You assume you can oversubscribe GPU memory “a little, like RAM” and degrade gracefully under pressure. Fix: There is no graceful GPU-memory degradation — no ballooning, no host swap. VRAM either fits or the allocation dies. Plan GPU memory as a hard ceiling, not an overcommittable pool.

6.12 — Reps

Open the exercises for the full set. This week’s reps build the muscles the project demands: enabling and reading a MIG layout, choosing a sharing mode for a workload and defending it, sizing a slice to a model, and stress-testing an AI-drafted placement against the real VRAM math.

AI policy for the reps (Phase 1): do it by hand first, then bring AI in to critique or to draft — and you own the verdict. Where a rep says “ask an AI to recommend a partition,” you must check its answer against the §6.5 table and the slice math before you accept it. A short Check Your Reps quiz sits at the bottom of this page; take it after the reps, not before.

A preview:

• Rep 1 — Classify five workloads onto the five sharing modes and justify each in one sentence.
• Rep 3 — Hand-design a MIG partition for an H100 80 GB serving three models; prove the slices sum to ≤ 7.
• Rep 6 — Read code/sample-nvidia-smi.txt and answer what each instance can and cannot see.
• Rep 8 — Take an AI-drafted placement, find where the VRAM math fails, and correct it.
• Rep 10 — Decide passthrough vs MIG for a training cluster and write the migration tradeoff you accepted.

Do every one. The reps are conditioning; the project is the game.

6.13 — This Week’s Project

You’re ready for Project 6 — Carve the GPU, in Project 6.

You will take one (real or, if you have no data-center GPU, faithfully simulated) accelerator and carve it for a real tenancy — a small AI hosting estate for a Christian university running several models for several departments. The Normal tier has you enable MIG, create a documented partition whose slices sum to ≤ 7, pin a workload to an instance, and prove the isolation. The Medium tier adds a second sharing mode for a different workload class and a right-sizing pass that maps each model’s real VRAM to the smallest slice that holds it. The Hard tier is the architect’s deliverable: a memo that recommends a sharing strategy for the whole estate — passthrough vs MIG vs vGPU vs time-slicing, workload by workload — with the isolation, migration, and cost tradeoffs argued and an AI-drafted placement critiqued, not merely accepted. The code carves the card; the memo is where you prove you knew which shadow was load-bearing.

See Appendix A for the lab environment, Appendix B for running models locally and in the cloud, and Appendix C for the agentic-AI rules that govern how you may use AI on the project.

6.14 — Coach’s Final Word for Week 6

This week the machine stopped being a thing you could point at. You learned that a hypervisor casts a useful shadow — Type 1 in every cloud, KVM under the open world — and that vCPU and memory overcommit are probabilistic promises that degrade gracefully, while VRAM is a hard ceiling that doesn’t degrade at all. You learned that snapshots capture a state but not the world, that live migration moves a shadow but not a passed-through device, and that the moment you hand a guest real silicon you trade mobility for performance.

And you learned the decision that defines AI administration: the five GPU-sharing modes, ordered by the isolation the hardware actually enforces. Passthrough gives one tenant the whole card. Time-slicing and MPS look like sharing but isolate nothing meaningful — the hotel room and the chore chart. MIG carves the silicon into seven hardware-walled instances and makes “each tenant has a GPU” a true sentence. vGPU shares a card across full VMs under license. Choosing among them, workload by workload, is the central placement decision of the era — and an AI planner can draft it but cannot own it, because the planner sees the metrics and you see the ministry.

The Colossian question runs under all of it: what is shadow, and what is substance? A virtual machine is an honest shadow, real and load-bearing, and the administrator’s whole craft is knowing exactly where the shadow stops being the substance — the vCPU that isn’t a core, the snapshot that isn’t a backup, the time-slice that isn’t isolation. Wheeler said every problem yields to another level of indirection. He was right, and he left out the warning Paul supplies: indirection is a shadow, and a shadow trusted past its limit ships an outage. Keep the line. Carve the card honestly. Let the hardware enforce the promises you make with your mouth.

See you next week.

Up next: Read the exercises and complete every rep. Then open Project 6 and carve your GPU for a real tenancy. Lean on Appendix A, Appendix B, and Appendix C as you go. After that, Chapter 7 — containers and the sending out, where the isolation you carved into the hardware this week meets the lighter, kernel-sharing isolation of namespaces and cgroups, and you put a model in a box and ship it whole into the world.