Chapter 4 — Reps

Conditioning, not grading. This week the gym is your storage stack: real commands, real napkin math, real governance calls. You will size things before you measure them, every time.

Ground rules:

1. Type every line yourself. No copy-paste. The sizing formulas — params × bytes-per-weight, vectors × dims × bytes — only become reflex by your own fingers writing them until you stop reaching for a calculator.
2. Predict, then measure. Before you run estate_sizer.py, run df, or check a pgvector table’s on-disk size, write down your prediction. The gap between your number and the real number is the entire lesson. An administrator who can predict the footprint can plan the budget.
3. Run everything read-only and off-peak. code/storage_audit.sh only reads. The du and find scans are I/O heavy — don’t run them against production at noon.
4. AI policy (Phase 1): do it by hand first, then grade the AI. Where a rep says “now ask an AI,” you must have your own answer on paper before you read the model’s. You own the verdict. You cannot direct an agent to right-size a node or write a retention rule in Phase 2 if you have never done it yourself.
5. Numbers, not vibes. Every claim in a written reflection gets a number behind it. “Big” is not a size. “140 GB at FP16” is.

These reps assume Linux (or WSL2/macOS with coreutils) and Docker available (Appendix A); the local-AI and pgvector setup is in Appendix B. The chapter’s code/ folder has every file referenced below.

Reps 1–3: Read the Weight of Your Data

Rep 1 — Audit a real tree

Run the audit against a directory you actually have (your home dir, a project, /var):

bash code/storage_audit.sh ~

Read all five sections. Write down: (a) which filesystem is closest to the ~85% alarm line; (b) which of your block devices are rotational (ROTA=1) vs SSD (ROTA=0); (c) the single largest consumer du found; (d) one file in the “not read in 180+ days” list that you would keep anyway and why. That last one is the steward’s call (§4.9) — the atime says “cold,” but you decide “keep” or “cast away.”

Rep 2 — IOPS vs throughput, named correctly

For each of the following workloads, write one word — IOPS or throughput — for the number that bottlenecks it, and one sentence of why (§4.2):

(a) Loading a 140 GB FP16 70B model's weights off disk into VRAM at startup.
(b) A vector database answering 2,000 scattered nearest-neighbor lookups/sec.
(c) Streaming a 4 TB training dataset sequentially into a preprocessing job.
(d) A Postgres database doing many small random row reads under OLTP load.

Then state, in one sentence, why you would never serve a model’s weights off a spinning HDD. (Hint: cold-start latency is a throughput story, and 150 IOPS is an IOPS story — name which kills you here.)

Rep 3 — Choose the RAID level

For each scenario, pick a RAID level (0, 1, 5, 6, or 10) and defend it in one sentence (§4.2):

(a) Reproducible scratch space for embeddings you can rebuild anytime.
(b) A hot Postgres + pgvector node that needs both performance and survivability.
(c) A large, slow, cheap archive array where rebuilds take 14+ hours.
(d) The OS boot volume on a two-disk appliance.

Then answer in one sentence: a node on RAID 10 is hit by ransomware — how much of your data does the redundancy save? (This is the §4.10 pitfall in person.)

Reps 4–6: The Napkin Math

Rep 4 — Size the weights by hand, then check the machine

On paper, without the script, compute the weights-only disk footprint for each model at each precision, using ~2 GB/B (FP16), ~1 GB/B (INT8), ~0.5 GB/B (INT4):

7B, 13B, 70B, 405B   ×   FP16, INT8, INT4

Write down your twelve numbers. Predict which is the smallest GPU that can serve a 70B at INT4. Then run the script and compare:

python3 code/estate_sizer.py

Any number you missed by more than ~10%, write the formula three times. Then state the two facts §4.4 says you should own cold: the floor card for a 70B INT4, and the node that holds a 405B at FP8.

Rep 5 — Don’t forget the KV cache

The weight tables are weights-only. Using the chapter’s rule of thumb (deployed VRAM ≈ weights × 1.15–1.4, §4.5), compute the realistic deployed VRAM for:

(a) An 8B model at INT4, short context.
(b) A 70B model at INT4, long context (use the high end of the headroom).

Then, in two sentences, explain to a teammate why serving many users at 128K context can make the KV cache exceed the weights — name the two terms in the KV formula that blow up. Confirm your (a)/(b) against estate_sizer.py’s deployed_vram_gb().

Rep 6 — Size the vector estate, then quantize it

Hand-compute raw vector storage (N × dims × bytes, §4.6) for 10,000,000 vectors at 1536 dims in: float32 (4B), halfvec (2B), int8 (1B), and binary (0.125B). Write the four numbers. Predict the float32→int8 savings ratio before you compute it.

Then run the script’s vector block and compare:

python3 code/estate_sizer.py   # read the "Vector DB" section

Finally: by what factor does truncating 1536 → 256 dims via Matryoshka shrink the index, and what’s the typical quality cost (§4.6)? One sentence on when you’d take that trade.

Reps 7–9: Build It and Govern It

Rep 7 — Stand up pgvector and measure the truth

Bring up Postgres + pgvector in Docker, then load the schema:

docker run -d --name pgv -e POSTGRES_PASSWORD=devpass -p 5432:5432 pgvector/pgvector:pg17
# wait for it to be ready, then:
docker exec -i pgv psql -U postgres < code/pgvector_setup.sql

Insert ~50,000 random halfvec(1536) rows (a quick INSERT ... SELECT with random() generation is fine for sizing — quality doesn’t matter here). Predict the on-disk size from the napkin math first, then run the pg_total_relation_size query at the bottom of code/pgvector_setup.sql. How close was your prediction? Where did the index overhead land relative to the raw vectors?

Rep 8 — Reproduce and fix the overfiltering bug

This is a real pgvector failure mode (§4.6). Run a nearest-neighbor query with a WHERE clause that prunes the vast majority of rows (e.g., WHERE sensitivity = 'restricted' when almost nothing is restricted) and a LIMIT 5. Note how a plain HNSW scan can return fewer than 5 rows. Now enable the fix and re-run:

SET hnsw.iterative_scan = 'relaxed_order';

Write two sentences: what the bug is (the graph scan stops before it has enough surviving rows) and why a storage/AI administrator must know index-level failure modes, not just capacity.

Rep 9 — Classify one dataset, by hand and then with AI

Take one real-ish dataset (invent a plausible one: “members’ pastoral-care notes,” “anonymized sermon transcripts,” “a fine-tuned model’s training set”). Using code/data_classification.yaml as the template, write its entry by hand: sensitivity, tier, irreplaceable true/false, retention, and whether lineage/legal-basis are required (§4.7).

Then ask an AI to classify the same dataset. Compare. Where did it agree? Where did it over- or under-classify? Write the one sentence that is the whole point: name a label the AI proposed that, if applied unreviewed, would have crossed a control boundary (encryption, access, or deletion) — and state why the human ratification gate (§4.9) exists.

Reps 10–11: Steward’s Judgment

Rep 10 — Reproducible, irreplaceable, or liability?

Sort each asset into exactly one of the three §4.9 buckets and give the one-line consequence of casting it away:

(a) A fine-tuned set of 70B weights.        (e) PII you no longer have legal basis to hold.
(b) An HNSW vector index.                   (f) A curated, hand-labeled training dataset.
(c) Scratch embeddings from last week.      (g) Your prompt library.
(d) The source documents behind your RAG.   (h) A nightly cache of derived features.

Then write two sentences: which one was hardest to place and why, and what additional fact about your organization would settle it. (The hard cases are where stewardship lives.)

Rep 11 — Write a retention rule you can defend

Pick one of the assets from Rep 10 and write a retention policy line for it, in the YAML shape of code/data_classification.yaml: the tier, the retention window (or indefinite), encryption/WORM if any, and lineage requirement. Then write the two-sentence defense: why this window and not double or half it, tying it to either an irreplaceability argument, a cost argument, or a compliance argument (EU AI Act Art. 10/12, §4.7). If you cannot defend the number, you do not have a policy — you have a guess.

Done? One Last Thing.

A miniature of the project. For a single planned deployment — one local 13B model at INT4 serving a RAG system over a 2-million-vector store at 1536 dims — produce, on one page:

1. The sizing. Weights-on-disk, deployed VRAM (with KV headroom), raw vector storage at int8, plus ~10% HNSW overhead, and a total estate size. Show the arithmetic; check it against estate_sizer.py.
2. The tiers. Which asset goes on hot/warm/cold (§4.3), and one sentence of why each.
3. The keep-or-cast-away call. For each asset, irreplaceable / reproducible / liability (§4.9), with its retention window.
4. One AI line. State where you would let an AI classifier help, and the one decision you would not let it make unreviewed (§4.9).

If you can fit a defensible estate plan on one page with the numbers behind every claim, you are ready for Project 4. If you reached for a calculator on the weight math, do Rep 4 again first.

Up next: Project 4 — Project 4: Size the AI Estate.