Chapter 06 · Reps

Virtualization and the Shape of the Machine — Reps

← Back to Chapter 6

Chapter 6 — Reps

Conditioning for the week the machine stopped being a thing you can point at. These reps drill the one skill the project grades: choosing and carving a GPU-sharing mode for a real workload, and knowing which promises the hardware actually keeps.

Ground rules

  • Type it yourself. No copy-paste from the chapter. Your fingers learn nvidia-smi mig flags by typing them wrong a few times.
  • Run everything you can. If you have access to a MIG-capable GPU (A100/H100/H200/B200/RTX PRO 6000 Blackwell) in the lab (Appendix A), run the real commands on an idle card — MIG toggles reset the device. If you don’t, run the paper versions: design the partition, do the slice arithmetic by hand, and read code/sample-nvidia-smi.txt as your “live” GPU.
  • AI policy (Phase 1): do it by hand first, then bring AI in to critique. Several reps explicitly ask you to draft something yourself, then ask an AI for its version and judge the difference. The human owns the verdict. End each AI-touching rep with one honest line: what the AI got right, what it got wrong, and what you changed.
  • Predict before you measure. Where a rep produces a number (a slice budget, a VRAM total), write your predicted answer down before you run it. The gap between your guess and the truth is the rep.

Reps 1–3: The Sharing-Mode Map

Rep 1 — Five workloads, five modes

For each workload below, name the GPU-sharing mode you’d use and justify it in one sentence tied to isolation:

  1. A bursty research notebook one PhD student runs a few hours a week.
  2. A multi-tenant SaaS where two competing customers’ models run on the same physical card.
  3. A 24/7 production inference service that must not share its latency with anyone.
  4. A pool of 30 virtual desktops needing light GPU acceleration.
  5. Three cooperative inference processes owned by one team, latency-sensitive, same trust boundary.

Then check yourself against code/gpu_sharing_picker.py — run it with matching flags and see where you and the script disagree. Write: any disagreement, and who’s right.

Rep 2 — The isolation ladder, from memory

Without looking, order these five modes from least to most isolation, and for each write what it does not protect against: time-slicing, passthrough, MIG, MPS, vGPU. Write: the one sentence that distinguishes MPS from MIG (the fault-isolation line).

Rep 3 — Design a MIG partition

An H100 80 GB must serve three things: an 8B chat model (~18 GB real VRAM), an embedding model (~3 GB), and a small 1B classifier (~3 GB). Design a MIG partition using [compute]g.[memory]gb profiles. Prove your slices sum to ≤ 7, and prove each model fits its slice’s memory. Write: your profile list and the arithmetic. (Hint: you do not have to use all 7 slices.)

Reps 4–6: Reading and Carving the Card

Rep 4 — Enable and inspect (or simulate)

On an idle MIG-capable GPU:

sudo nvidia-smi -i 0 -mig 1          # resets the device — idle only!
sudo nvidia-smi mig -i 0 -lgip       # list supported GPU Instance profiles

Write: the largest and smallest profile your card offers, and how many of the smallest you could create at once. If you have no GPU, answer from a B200’s documented profiles (1g.23gb … 7g.180gb) and state the 7-instance ceiling explicitly.

Rep 5 — Carve, pin, prove isolation

Create a 3g + four 1g mix (or your Rep 3 design), then pin a workload to one instance and confirm it sees only that slice:

sudo nvidia-smi mig -i 0 -cgi 9,19,19,19,19 -C
sudo nvidia-smi mig -i 0 -lgi
CUDA_VISIBLE_DEVICES=MIG-<uuid> nvidia-smi   # inside the pinned context

Write: what nvidia-smi -L shows now, and one sentence on why a process pinned to instance A cannot allocate instance B’s memory. Tear down cleanly when done (-dci, then -dgi, then -mig 0).

Rep 6 — Read a layout cold

Open code/sample-nvidia-smi.txt. Without re-reading the chapter, answer: How many instances exist? Which is the largest and what’s it sized for? What is the total slice budget consumed, and is the card full? Can GI 1 read GI 13’s memory? Write: your four answers, then check them against the file’s footnotes.

Reps 7–9: The CPU/Memory Substrate and Its Limits

Rep 7 — Overcommit arithmetic

A host has 32 physical cores and 256 GB RAM. You run general server VMs at a 4:1 vCPU overcommit and provision each VM 16 GB. Write: how many vCPUs you can advertise, roughly how many such VMs that supports, and the one symptom (and where it appears) when you push CPU overcommit too far. Name why this same overcommit logic is off the table for VRAM.

Rep 8 — Break an AI-drafted placement

Ask an AI assistant: “I have a free 48 GB GPU. Place a 70B model on it for inference.” Take its answer at face value, then find the flaw using the Chapter 4 VRAM math (weights + KV cache + 15–40% overhead). Write: the AI’s recommendation, the specific number it got wrong or omitted, the OOM it would cause, and your correction. (This is the chapter’s central AI-failure mode — feel it.)

Rep 9 — Snapshot the world, not just the VM

Describe a concrete scenario where rolling a VM back to a 20-minute-old snapshot causes a correctness bug outside the VM (clocks, replication position, deleted data, in-flight transactions). Write: the scenario in three sentences and the rule it teaches about snapshots vs backups.

Reps 10–11: Scheduling and the Migration Trade

Rep 10 — Passthrough vs MIG for training

A four-GPU training cluster will run long, checkpointable jobs and occasional latency-critical inference. Decide between passthrough and MIG for the training nodes. Write: your choice, the live-migration consequence you accepted, and how preemption + checkpointing change the math.

Rep 11 — Co-schedule AI against traditional load

You must run a greedy nightly embedding job and a 24/7 inference service on shared GPUs. Using priority, preemption, and isolation, sketch a scheduling policy that keeps the inference service’s latency safe. Write: which lever does the protecting, and what breaks if the embedding job does not checkpoint before preemption.

Done? One Last Thing.

A Christian university wants to host four small models on one H100 80 GB for four departments, two of which must be hard-isolated from each other (one handles student records). In under one page: choose the sharing mode(s), design the partition (profiles + slice arithmetic ≤ 7), map each model’s real VRAM to its slice, and state — in one sentence each — the isolation promise you’re making to each department and how the hardware keeps it. Then ask an AI to produce the same plan and critique its answer: where did it conflate time-slicing with isolation, mis-size a slice, or exceed the 7-slice ceiling? End with the line you’d put in the runbook so the next admin doesn’t trust a shadow past its limit.

This is Project 6 in miniature. If you can do it on one page by hand and catch the AI’s mistakes, you’re ready for the full carve.

Up next: Project 6 — Project 6: Carve the GPU.