Project 7 — Ship a Model in a Box

“Go therefore and make disciples of all nations…” — Matthew 28:19 (ESV)

Chapter: 7 — Containers and the Sending Out Due: End of Week 7 Submit: A link to a public GitHub repository containing your Dockerfile, compose.yaml, the app source, smoke_test.sh, a REPORT.docx, and (Hard tier) a DECISION.docx and REVIEW.docx. Run it with a real local toolchain + git — see Appendix A. Local-and-cloud AI (Ollama, model pulls) setup is in Appendix B. Allowed tools: Docker or Podman, Ollama / vLLM / Open WebUI, a real editor, curl, git, the textbook, and AI as a reviewed assistant. AI policy: Phase 1 (Weeks 1–8): AI is studied and used, but you build the traditional way first and own the verdict. You will hand-author your Dockerfile and compose file before you ask an agent to generate one, and the Hard tier requires a written, line-by-line review of agent-generated config. Include an honest “AI usage” section in REPORT.docx: what you asked AI, what it got wrong, and what you changed. The agentic rules of the road are in Appendix C.

The Setup

Mercy Lutheran — a small church-and-school you have been administering all term — wants an in-house “Verse Helper”: a chat tool, on their own hardware, that explains Bible passages in plain language and quotes only the ESV. Two hard constraints make this a containers problem and not a cloud API problem. First, the school keeps student questions private and will not send them to a third-party API. Second, the volunteer who maintains the server changes every year, so whatever you build has to come up the same way on a fresh machine with one command — no fifteen-page install document that rots.

So you are going to put a model in a box. A local LLM, served behind the universal OpenAI-compatible API, fronted by something a non-technical volunteer can use, packaged so that git clone and one up command stands the whole thing up identically anywhere. “It works on my machine” is not a deliverable here; the whole point is that it works on the next person’s machine, whole, unchanged.

And then — because this is the AI era and the school’s principal will ask — you will be ready to defend which serving tool you chose, and to prove that the image you ship is one you would actually trust on a network with minors on it.

Setup (the starter)

This chapter’s code/ folder is your reference, not your answer. Read it, understand every line, then build your version:

• code/Dockerfile — a multi-stage, non-root, healthchecked reference image.
• code/compose.yaml — Ollama + Open WebUI, with a commented GPU block.
• code/app.py — the tiny FastAPI front that proxies the backend over /v1 and tags answers "verified": false.
• code/requirements.txt — pinned dependencies.
• code/smoke_test.sh — proves the box serves, not just starts.

Learning Targets

By completing this project, you will demonstrate that you can:

• Write a correct, reviewed, hardened Dockerfile (multi-stage, non-root, pinned, healthchecked).
• Persist model weights and service state in volumes, never in the container layer.
• Stand up a multi-service stack with Compose so it comes up with one command.
• Serve a local LLM behind the OpenAI-compatible API and reach it from a front-end.
• Smoke-test a deployed service for real behavior, not just liveness.
• Make and defend a serving-tool decision (Ollama vs vLLM vs NIM) under stated constraints.
• Review AI-generated container config line by line and keep the security verdict in your own hands.

Normal Tier

Goal: Stand up a containerized local LLM behind the OpenAI-compatible API, in a pinned, non-root image with a persistent model volume, that comes up with one command and passes a smoke test.

Required features

1. A model in a box. A containerized Ollama (or llama.cpp) serving a small open-weight model (e.g., llama3.2:3b) on its OpenAI-compatible /v1 endpoint. Reaching the model through /v1/chat/completions must work.
2. Persistent weights. The model cache is on a named volume; prove it survives docker rm of the container and a re-up (no re-download).
3. A hardened front-end image. Your own Dockerfile (not the starter copied) that builds your app image: it pins the base (no :latest), runs as a non-root USER, copies dependencies and installs them before the source, and declares a HEALTHCHECK.
4. One-command bring-up. A compose.yaml that starts the whole stack with docker compose up -d. No manual docker run choreography.
5. A smoke test. A smoke_test.sh (adapt the starter) that checks app liveness, backend reachability, and one end-to-end inference call, and exits non-zero on any failure.
6. REPORT.docx with: the exact commands to run your stack; a screenshot or transcript of a successful inference; your image size and how you kept it small; and an honest AI usage section.

Example interaction

$ git clone <your-repo> && cd <your-repo>
$ docker compose up -d
$ docker compose exec ollama ollama pull llama3.2:3b
$ ./smoke_test.sh
== 1. App liveness (/health) == ok (200)
== 2. Backend reachable == ok
== 3. End-to-end inference == ok — model answered
ALL CHECKS PASSED.

Normal-tier rubric (out of 100)

Medium Tier (+up to 25% extra credit)

Goal: Put a real face on it and clean the front-end/back-end seam.

1. Open WebUI added as a second service in compose, talking to the backend over the internal network by service name (not localhost), with WEBUI_AUTH=true so it requires a login (RBAC).
2. A .dockerignore that keeps .env, .git, caches, and secrets out of the build context — and a note in REPORT.docx proving no secret is baked into any layer (docker history evidence).
3. Runtime config, not baked config. The backend URL, model name, and any key are injected at runtime (env / compose), never ENV-baked into the image.
4. A second backend, same client. Point your app.py (or a curl) at a different OpenAI-compatible backend by changing only the base URL — Ollama → llama.cpp, or Ollama → a vLLM container if you have a GPU — and document in REPORT.docx that nothing else changed. Prove the “universal API” claim with your own hands.

Hard Tier (+up to 25% additional extra credit)

The Hard tier is graded on judgment an agent cannot supply. Two written deliverables.

1. DECISION.docx — the serving-tool memo. Mercy Lutheran’s principal asks: “Why this tool and not another?” Write a one-page decision memo recommending Ollama vs vLLM vs NIM for a stated, realistic future: the Verse Helper graduates from 1 volunteer testing it to ~40 classrooms hitting it during the same first-period Bible class, on a single donated GPU server, with no IT staff and a tiny budget. Address concurrency and throughput (Ollama’s ~1–3 users vs vLLM’s continuous batching), operational burden, support/SLA and air-gap needs (where NIM fits), and cost. Commit to one recommendation and name the constraint that decides it. State explicitly what would change your answer.

2. REVIEW.docx — the agent Dockerfile review. Ask a coding agent to generate a Dockerfile for the app, paste its raw output into REVIEW.docx unedited, and then annotate it line by line against the §7.4 review table: base pinning, non-root USER, install order, dependency pinning, secrets handling, healthcheck. For every issue, write the exact corrected line. Close with one paragraph: what would have shipped to a network with minors on it if you had trusted the agent’s first draft?

The Normal and Medium tiers prove you can build the box. The Hard tier proves you can decide what box to build and refuse to ship an unsafe one — which is the part of the job that is still yours.

Submission

Push everything to a public GitHub repo: Dockerfile, compose.yaml, app source, smoke_test.sh, REPORT.docx, and (Hard) DECISION.docx + REVIEW.docx. The repo must come up with the documented commands on a clean machine. Submit the repo link. See Appendix A for git and toolchain setup.

Hints (Read Before You Begin)

• Build by hand before you generate. Phase 1 rule. Author your own Dockerfile first; only then ask an agent and compare. You cannot review what you have never written.
• Pin everything. Base image tag, dependency versions. The first time you type :latest, stop and replace it.
• Prove persistence, do not assume it. Actually docker rm the backend and bring it back up. If the model re-downloads, your volume is wrong.
• host.docker.internal reaches the host from inside a container on Docker Desktop only; inside a compose network, use the service name (http://ollama:11434). On a Linux host — the cloud/no-admin path (Appendix A, Path A) — that hostname does not resolve: run with --network=host and http://localhost:11434, or use the service name.
• A model that starts is not a model that serves. Make smoke_test.sh do a real inference call. “200 on /health” is necessary, not sufficient.
• Check the image for secrets with docker history --no-trunc <image> before you push. A baked key in a deleted layer is still in the image.

What Mastery Looks Like (Beyond the Rubric)

A master’s-level submission is one a stranger could git clone and stand up in five minutes with zero questions, whose image you would defend in a security review, and whose DECISION.docx reads like a real architect wrote it — committing to a recommendation, naming the deciding constraint, and saying plainly what would change the call. The give-away of mastery is the REVIEW.docx: it shows you used the agent and out-judged it.

Coach’s Note — The easiest 25% you will ever earn is the Hard tier here, because most students will build a working box and stop. The box is the conditioning. The memo and the review are the game. The administrator who can stand up a stack is useful; the one who can stand it up and tell the principal why this tool and why this image is safe is the one who gets trusted with the network.

When You’re Done

• docker compose up -d brings the whole stack up on a clean machine.
• The model serves on /v1; an inference call returns an answer.
• Weights persist across docker rm + re-up (you tested it).
• The image pins its base, runs non-root, installs deps before source, has a healthcheck.
• smoke_test.sh checks liveness + backend + inference and fails loud.
• No secret is baked into any layer (docker history clean).
• (Hard) DECISION.docx commits to one serving tool with a named deciding constraint.
• (Hard) REVIEW.docx annotates an agent Dockerfile line by line with corrected lines.
• REPORT.docx has an honest AI-usage section.

A theological footnote. The Great Commission is a sending with a specification attached — carry this, do these things, teach all that I commanded (Matthew 28:19–20, ESV). A faithful sending arrives whole; an unfaithful one arrives corrupted, “It works on my machine.” The image you build this week is, in miniature, that same discipline of fidelity in transmission: pin it, harden it, prove it stands up the same on the next person’s machine. But note the limit the project builds in on purpose — app.py tags every answer "verified": false. The box can carry the model’s words faithfully and the words can still be false. A faithful courier can deliver a false message perfectly. The container guarantees the delivery; only you, in the loop, guard the truth of what is delivered. Steward both: send it whole, and verify before anyone trusts it.

See you next week.