Project 6 — Carve the GPU

“These are a shadow of the things to come, but the substance belongs to Christ.” — Colossians 2:17 (ESV)

Chapter: 6 — Virtualization and the Shape of the Machine Due: End of Week 6 Submit: A link to a public GitHub repository containing your partition scripts/configs, your pinned-workload evidence (nvidia-smi captures or simulator output), and REPORT.docx. See Appendix A for the lab environment and Appendix B for running models locally/in the cloud. Allowed tools: nvidia-smi and the MIG CLI; KVM/QEMU/libvirt or KubeVirt for the VM path; Ollama or vLLM (Appendix B) to actually load a model onto a slice. If you have no data-center GPU, the GPU Partition Visualizer on the chapter page and code/sample-nvidia-smi.txt are an acceptable simulated substrate — say so in your report. AI policy (Phase 1): AI is OFF for the building — you carve the card and write the report yourself. AI is ON for one thing only: the Hard-tier placement critique, where you ask an AI for a placement and then tear its answer apart. Log that exchange in your report. The human owns every verdict. See Appendix C.

The Setup

Concordia Lakes University, a small LCMS school, just took delivery of a single NVIDIA H100 80 GB — the only data-center GPU the IT department owns, and it has to serve four hungry constituencies at once:

• The CS department runs an 8B chat assistant for coursework (~18 GB real VRAM at FP16, modest context).
• The library runs an embedding model for its catalog search (~3 GB).
• The registrar runs a small 1B classifier over student records — FERPA-protected data that must not share a fault or memory domain with anything else on the card.
• A research student wants a bursty notebook for a few hours a week and does not care about isolation.

One card. Four tenants. Unequal isolation requirements. This is the real shape of GPU administration on a budget, and it is exactly the decision the chapter armed you to make. Your job is to carve the card honestly — to make the promises you give each department true at the hardware level — and to write down why.

Setup (the starter)

This chapter’s code/ folder is your reference kit:

• code/gpu_partition.sh — a safety-checked MIG carve script; adapt its profile IDs to your card’s -lgip output.
• code/time-slicing-config.yaml — a device-plugin time-slicing config (read its warning before you use it for anything you call “isolated”).
• code/gpu_sharing_picker.py — encodes the §6.5 decision rules; use it to check your reasoning, not replace it.
• code/sample-nvidia-smi.txt — a real MIG layout to read, and your simulated GPU if you have no hardware.

Learning Targets

You will demonstrate that you can:

• Enable MIG on an idle GPU and create a documented partition whose slices sum to ≤ 7.
• Map each workload’s real VRAM (weights + KV cache + overhead, per Chapter 4) to the smallest slice that holds it.
• Pin a workload to a specific instance and prove memory/fault isolation.
• Choose the right sharing mode per workload class and defend the choice against alternatives.
• Critique an AI-drafted placement and catch where it conflates a shadow (time-slicing) for substance (hardware isolation) or busts the VRAM math.

Normal Tier

Goal: Carve the H100 with MIG, place the registrar’s isolated workload, and prove the isolation.

Required features

1. Enable MIG mode on an idle GPU (or simulate via the visualizer / sample file) and capture -lgip.
2. Create a partition using [compute]g.[memory]gb profiles that gives the registrar’s classifier its own MIG instance, plus instances for the CS chat model and the library embedding model. Your slices must sum to ≤ 7, and you must show the arithmetic.
3. Pin at least one real or simulated workload (e.g., a model loaded via Ollama/vLLM, or a stated CUDA context) to a specific instance with CUDA_VISIBLE_DEVICES=MIG-<uuid>.
4. Prove isolation: capture evidence that the pinned workload sees only its slice’s memory and cannot reach a neighbor’s.
5. REPORT.docx documents the partition, the per-model VRAM-to-slice mapping, and the isolation promise made to the registrar.

Normal-tier rubric (out of 100)

Medium Tier (+up to 25% extra credit)

Goal: Add a second sharing mode for the non-isolated tenant and a rigorous right-sizing pass.

1. Serve the research student’s bursty notebook with the appropriate lighter-weight mode (time-slicing within a slice, or a dedicated small instance) — and justify why you did not give them a full MIG instance of their own.
2. Right-size every model: show the real VRAM each needs (weights + KV cache at your stated context + 15–40% overhead) and prove each lands in the smallest profile that holds it. Flag any model that would not fit the slice you’d naively assign.
3. Show what breaks if you get it wrong: deliberately under-size one slice (on paper or in the sim), demonstrate the OOM, and explain why MIG’s isolation does not conjure the missing memory.
4. Run code/gpu_sharing_picker.py for each tenant and reconcile its recommendation with yours in the report.

Hard Tier (+up to 25% additional extra credit)

Goal: The architect’s deliverable — a sharing-strategy memo for the whole estate, with an AI placement critiqued.

Write STRATEGY.txt (1–2 pages) that an IT director could act on:

1. The estate decision. Recommend a sharing strategy for all four tenants — naming passthrough, MIG, vGPU, MPS, and time-slicing where each fits — and argue the tradeoffs: isolation, live-migration consequences (what becomes non-migratable), licensing (vGPU), and cost of idle capacity.
2. The growth question. Concordia Lakes might add a second H100, or a researcher might need a full-card training run. State what your partition does under each future, and when you’d switch from MIG back to passthrough — and what you’d lose.
3. The AI critique (the graded judgment). Ask an AI assistant to design the placement for these four tenants. Paste its answer, then dismantle it: where did it conflate time-slicing with isolation, mis-size a slice, exceed the 7-slice ceiling, or miss that the registrar’s data needs hardware isolation? Document at least two real flaws (or, if it got it right, prove you could have caught the flaws it avoided). End with the one-line runbook note that stops the next admin from trusting a shadow past its limit.

The Hard tier is graded on the judgment an agent cannot make for you — the estate-level recommendation and the critique. The card-carving is craft; the memo is the architecture.

Submission

Push a public GitHub repo containing your partition scripts/configs, your isolation evidence (captures or labeled simulator output), REPORT.docx, and (for Hard) STRATEGY.txt. Put the repo link in the LMS. If you used the visualizer/sample file instead of hardware, say so plainly — a well-reasoned simulation beats a hand-waved “I ran it.”

Hints (Read Before You Begin)

• Toggle MIG only on an idle card. It resets the device. Drain first.
• Slices, not memory, are the budget. 3g + 1g + 1g + 1g = 6 slices; you don’t have to fill all 7.
• Size the slice to the real VRAM, not weights-only. The KV cache is what blows the budget — re-use the Chapter 4 math.
• “Shared GPU” is four different promises. Name the mode every single time you say it.
• The registrar is the whole point. If the FERPA workload isn’t on a hardware-isolated instance, the Normal tier hasn’t been met no matter how pretty the partition.

What Mastery Looks Like (Beyond the Rubric)

A mastered submission reads like an IT director could hand it to an auditor. The partition is documented to the slice; every isolation promise is tied to the hardware that keeps it; the right-sizing shows real VRAM math, not vibes; and the AI critique demonstrates you can use the fast partner without being fooled by it. The tell of mastery is a sentence like: “The registrar’s classifier is on a dedicated 1g.10gb MIG instance because FERPA data requires hardware fault isolation, which time-slicing and MPS do not provide.” That sentence is the whole course.

Coach’s Note — The temptation under deadline is to time-slice everything, call it “shared,” and move on. Resist it precisely where it’s easiest — the registrar. The difference between a passing engineer and a trustworthy one is whether the promise you make with your mouth is the promise the silicon keeps. Carve the hard wall where the protected data lives, and let MIG make your sentence true.

When You’re Done

• MIG enabled and partition documented with slice arithmetic ≤ 7
• Registrar workload on a hardware-isolated instance
• A workload pinned to an instance with isolation proven
• Each model’s real VRAM mapped to the smallest slice that holds it (Medium)
• STRATEGY.txt recommends an estate strategy and critiques an AI placement (Hard)
• REPORT.docx is reproducible and honest about hardware vs simulation

A theological footnote. Paul tells the Colossians that the old observances were “a shadow of the things to come, but the substance belongs to Christ” (Colossians 2:17, ESV). A virtual machine, a vGPU, a time-slice — each is a shadow cast by real silicon, useful and honest until you mistake it for the substance. This week you make promises about isolation, and your whole task is to ensure the shadow you offer is backed by a substance that keeps it: MIG’s hardware wall under the registrar’s protected data, not a time-slice that merely looks like separation. Stewardship is being found faithful with what is not yours to make (1 Corinthians 4:2) — and you did not make this card, nor the trust the registrar places in your partition. Carve it so the wall is real.

See you next week.