Project 4 — Size the AI Estate

“a time to keep, and a time to cast away.” — Ecclesiastes 3:6 (ESV)

Chapter: 4 — Storage Administration and the Weight of Data Due: End of Week 4 Submit: A link to a public GitHub repository containing SIZING.txt (the estate plan), data_classification.yaml (your governance policy), your pgvector proof (the SQL you ran + a screenshot or pasted output of the on-disk size), and REPORT.docx (the narrative + Hard-tier policy memo). See Appendix A for the environment — including the one-command pgvector standup in §A.11 (The Course Toolkit) — and Appendix B for the local-AI setup. Allowed tools: A Linux/WSL2/macOS shell with coreutils, Docker, Postgres + pgvector, Python 3.10+, a real editor, the textbook, and code/ from this chapter. AI policy — Phase 1 (Weeks 1–8): AI is OFF for the building. You size the estate, write the math, and make the keep-or-cast-away calls yourself. You may, in one clearly-labeled section of REPORT.docx, run an AI classifier or sizing assistant over your plan and critique it — but the plan, the numbers, and the verdicts are yours, by hand, first. Every submission includes an honest AI usage line. You cannot direct an agent to right-size a GPU node or set a retention policy in Phase 2 if you have never sized an estate or defended a retention window yourself. Phase 2 (Weeks 9–16) turns agentic AI on and requires an agent-log.txt. Not yet.

The Setup

Grace & Mercy Lutheran Ministries runs a small school, a relief arm, and a congregation across three sites. The IT team — you — has been told to “add AI” without anyone first asking what it weighs. Three workloads are coming, and each lands on your storage:

1. A local LLM for drafting communications and answering staff questions about internal policy, served on-premises (data residency: the pastoral and donor data cannot leave the building). The team is choosing between a 13B model for the school’s single 48 GB card and a 70B if they buy more GPU.
2. A RAG system over the ministry’s documents — sermons, policies, the school handbook, anonymized pastoral-care summaries — backed by a vector database of roughly 5,000,000 chunks at 1536 dimensions.
3. Datasets: the corpus behind the RAG, plus a small fine-tuning dataset of approved internal Q&A used to specialize the model. Some of this is restricted (contains PII); the fine-tuned weights, once produced, are irreplaceable.

Leadership wants one thing from you before they spend a dollar: a defensible sizing-and-storage plan. How much storage and VRAM does each piece need? What tier does each asset belong on? How long do you keep each one, and which must be deleted on a schedule? And — the question they don’t know to ask but you do — what here is irreplaceable, what is reproducible, and what is liability?

You are going to answer all of it, with the napkin math of Chapter 4 and a real pgvector build to prove the vector numbers, and you are going to put a recommendation in writing that a budget committee and an auditor could both read.

Setup (the starter)

Work from this chapter’s code/:

• code/estate_sizer.py — the sizing math (weights, deployed VRAM, vectors). Use it to check your hand calculations; do not let it replace them.
• code/pgvector_setup.sql — the real pgvector schema, HNSW index, and the on-disk-size query. The basis for your vector proof.
• code/data_classification.yaml — the governance/tiering policy template you will adapt into your deliverable.
• code/storage_audit.sh — read-only audit; use it to ground your tiering discussion in real df/lsblk output.

Learning Targets

By completing this project, you will demonstrate that you can:

• Size model weights and deployed VRAM from params × bytes-per-weight plus KV-cache/framework headroom — by hand, at a chosen quantization.
• Size a vector database from num_vectors × dimensions × bytes plus HNSW overhead, and reason about quantization and Matryoshka truncation.
• Lay out an AI estate across storage tiers (hot/warm/cold/archive) with IOPS/throughput justification.
• Stand up pgvector, build an HNSW index, and measure real on-disk size against a prediction.
• Write a data-governance policy — classification, retention, lineage — and defend a keep-or-cast-away decision for each asset class.
• (Hard) Make the steward’s recommendation an agent cannot make: which assets are irreplaceable vs reproducible vs liability, and what abides.

Normal Tier

Goal: Produce SIZING.txt, a complete, correct sizing-and-tiering plan for the ministry’s three workloads, with the arithmetic shown.

Required features

1. Model sizing. For both the 13B and the 70B candidate, at a chosen quantization (state it), compute weights-on-disk (params × bytes-per-weight). State the smallest GPU that can serve each and whether the school’s 48 GB card suffices. Show the formula and the numbers (§4.4).
2. Vector sizing. For the 5M-chunk, 1536-dim store, compute raw vector storage at a chosen precision (float32 / halfvec / int8 — state it and why) plus ~10% HNSW index overhead (§4.6). Show the math.
3. Dataset sizing. Give a reasoned estimate for the RAG corpus and the fine-tuning dataset (state your assumptions — number of documents, average size). Approximation is fine; assumptions must be explicit.
4. A total estate size — the sum, on one line, with a one-sentence “this is what we are buying.”
5. A tiering table. Every asset (weights, vector index, RAG corpus, fine-tuning data, logs, scratch) assigned to hot / warm / cold / archive, with a one-clause IOPS-or-throughput justification each (§4.2–§4.3). Use real df/lsblk output from code/storage_audit.sh to ground at least one choice.
6. A data_classification.yaml adapted from the starter, covering every asset with a sensitivity label and a retention window.

Normal-tier rubric (out of 100)

Medium Tier (+up to 25% extra credit)

Goal: Add the realism that separates a guess from a plan.

1. KV-cache headroom. Don’t quote weights-only. Compute deployed VRAM for your chosen model using the ≈1.15–1.4 rule, and state the context-length/batch assumption behind your headroom factor (§4.5). Re-check whether the 48 GB card still holds the 70B INT4 once the cache is in.
2. A quantization trade study. A short table: the 70B at FP16 / INT8 / INT4 — disk, deployed VRAM, and the quality cost of each step (cite the §4.4 ranges, hedged “as of 2026”). End with a one-sentence recommendation and the reason.
3. The pgvector proof. Actually stand up Postgres + pgvector (the docker run … pgvector/pgvector:pg17 recipe is in Appendix A §A.11, The Course Toolkit), load the schema from code/pgvector_setup.sql, insert a representative number of halfvec(1536) rows, and run pg_total_relation_size. Paste the measured size next to your predicted size and explain any gap (index overhead, page alignment, TOAST). This is “predict, then measure” graded.
4. A Matryoshka note. Compute what truncating 1536 → 256 dims would save on the vector estate and state the typical quality cost — then recommend for or against it for this ministry.

Hard Tier (+up to 25% additional extra credit)

Goal: The steward’s recommendation. This is the part an agent cannot write, because it is a judgment about this ministry’s values and obligations, not a fact in any model’s weights. Produce a policy memo (in REPORT.docx) addressed to ministry leadership.

1. Classify every asset as irreplaceable / reproducible / liability (§4.9), with the one-line consequence of casting each away. The fine-tuned weights, the curated fine-tuning set, the prompt library — which of these comes back if lost, and which does not?
2. Set and defend a retention window for each, tying each to one of: an irreplaceability argument, a cost argument, or a compliance argument. For the restricted PII training data, name the obligation (EU AI Act Article 10 provenance/quality, Article 12 logging — hedge the high-risk timeline as provisional, §4.7) and the deletion schedule, because here “cast away” is the faithful act, not thrift.
3. The hard call. Identify the one asset where keep and cast-away both feel like loss, and make the decision anyway — in writing, with the reason. State explicitly which part of this memo you would not delegate to an AI classifier and why (§4.8).
4. A recommendation leadership can act on: 13B or 70B, which card(s), which vector precision, and the total cost — one clear paragraph, the tradeoffs named, the line committed to.

Submission

Push a public repo containing:

• SIZING.txt — the estate plan (Normal; KV/quant/Matryoshka if Medium).
• data_classification.yaml — your governance policy.
• The pgvector proof — the SQL you ran and the measured on-disk size (Medium).
• REPORT.docx — the narrative, the predict-vs-measure discussion, the AI usage line, and the Hard-tier policy memo.

Submit the repo URL.

Hints (Read Before You Begin)

• Do the math by hand first, then check estate_sizer.py. If your hand number and the script disagree by more than ~10%, find out why before you trust either. The script is the answer key, not the worker.
• Weights-only is a trap. Every VRAM number in SIZING.txt should be deployed VRAM (with headroom), or explicitly labeled “weights-only.” A committee that buys to a weights-only number buys a card that OOMs (§4.10).
• Raw vectors dominate; size them first. 5,000,000 × 1536 × bytes is the big number; the HNSW graph is a ~10% rider. Quantize before you buy disk.
• Tier by access pattern, not by sentiment. “Important” is not a tier. Served live is hot; kept for audit, rarely read is cold/archive (§4.3).
• Ground at least one tiering claim in real output. Run code/storage_audit.sh; cite the lsblk rotational/SSD line. Real numbers beat assertions.
• Hedge the regulation. The EU AI Act high-risk timeline is in flux as of mid-2026 (Digital Omnibus deferral to Dec 2, 2027, not yet formally adopted). Say “as of 2026… re-confirm,” and you’ll be right where a precise date would make you wrong.

What Mastery Looks Like (Beyond the Rubric)

A great Project 4 reads like a plan a CFO could sign and an auditor could defend. The sizing isn’t just correct — it’s shown, so a reviewer can follow 70 × 2 = 140 GB FP16 → ~38 GB INT4 → +30% KV/framework → fits a 48 GB card with room without trusting you. The pgvector proof closes the loop the whole book is built on: you predicted a number, you measured it, and you explained the gap instead of hiding it.

But the thing that makes it master-level is the Hard-tier memo. A weak memo says “keep everything, to be safe.” A great one discriminates: it names the fine-tuned weights as irreplaceable and guards them, names the vector index as reproducible and lets it go cheaply, names the stale PII as liability and deletes it on a defended schedule — and when two options both feel like loss, it decides and owns the decision. That discrimination is the storage administrator’s real work, and it is exactly the work that does not transfer to the tool.

Coach’s Note — Students are tempted to make this a calculator exercise and skip the memo. Don’t. The math proves you can size an estate; an agent will size one flawlessly in five seconds in Phase 2. The memo proves you can decide what is worth keeping — and that decision is a fact about this ministry’s mission, its donors, its obligations, and its budget, not a fact in any model’s training. The memo is where you stop being a prompt and become the steward.

When You’re Done

1. Re-read SIZING.txt as if you were the CFO: is there a number you cannot defend? Fix it.
2. Confirm every VRAM figure is deployed (with headroom) or labeled weights-only.
3. Run the pg_total_relation_size query one more time; confirm the measured size in your repo matches what’s running.
4. Read the Hard memo aloud. Does it discriminate — irreplaceable vs reproducible vs liability — or does it hide behind “keep everything”? Make it discriminate.
5. Write the honest AI usage line. If you used an AI to critique your plan, say what it caught and what it got wrong.
6. Commit, push, submit the repo URL.
7. Read Chapter 5 — the KV cache you just sized goes over the wire next, and the interconnect decides whether your inference scales.

A theological footnote. “A time to keep, and a time to cast away” (Ecclesiastes 3:6, ESV) is the whole project compressed into one line. You did not create this ministry’s data — the donor records, the pastoral notes, the model someone fine-tuned with care — it was entrusted to you, and “it is required of stewards that they be found faithful” (1 Corinthians 4:2, ESV). Faithfulness here is not hoarding; the administrator who keeps every byte in fear costs the ministry money, widens its exposure, and buries the treasure under the dross. Nor is it carelessness; the one who deletes the irreplaceable to save a few dollars has thrown away what cannot be rebuilt. Faithfulness is discernment — guarding the irreplaceable, releasing the reproducible, and being rid of the liability on a schedule you can defend before God and the auditor alike. When you wrote that memo and decided what abides and what is cast away, you did a steward’s work. The Preacher named the seasons; you set the policy. Be found faithful with what you did not make.

See you next week.