Containers and the Sending Out
What does it take to send something out, whole, into the world?
Chapter 7 — Containers and the Sending Out
“Build, ship, run.” — Docker’s three-word manifesto
“Go therefore and make disciples of all nations…” — Matthew 28:19 (ESV)
Why This Matters
Last week you carved a GPU. You learned that one physical card can be sliced — passthrough, time-slicing, MIG, vGPU — and you learned which slice fits which workload. That was about the shape of the machine: how to divide one expensive piece of hardware among many tenants without letting them step on each other.
This week is the opposite motion. Not dividing — sending out. You have something that works on your machine: a script, a service, a model. The question every administrator eventually faces is how to send that thing out — whole, intact, reproducible — so that it runs the same on a teammate’s laptop, a staging server, a GPU node in a datacenter, and a cloud you have never logged into. “It works on my machine” is not a deployment. It is a confession of failure.
The container is the answer the industry settled on, and it is not the same answer as the virtual machine you studied in Chapter 6. A VM virtualizes the hardware and ships a whole guest operating system. A container virtualizes the operating system and ships only your application and its dependencies, sharing the host’s kernel. That single architectural difference — share the kernel instead of duplicating the OS — is why a container boots in under a second where a VM boots in tens of seconds, and why you can pack dozens of containers onto a host that would groan under a handful of VMs. You will spend the first half of this chapter making that difference precise, because the rest of your career as an administrator will involve choosing between them.
The dual-AI thread runs straight through this week, hot on both rails. AI as the tool you wield: the agent will happily write your Dockerfile and your compose file — and it will just as happily bake in a :latest tag, run your process as root, and COPY . . your secrets into an image layer that lives forever. You will learn to review what it generates, line by line, because a Dockerfile is a security boundary and an agent does not know your threat model. AI as the workload you run and govern: the most common containerized service of 2026 is an LLM. You will put a model in a box — Ollama for the easy path, vLLM for the production path, Open WebUI for the face — and learn that the OpenAI-compatible API has become the SSH of model serving: one interface, every backend.
And here is the week’s question, which is older than any of this. What does it take to send something out, whole, into the world? Our Lord’s last command in Matthew’s Gospel is a sending: “Go therefore and make disciples of all nations…” (ESV). The Great Commission is the original deployment problem — take what was true in one place, package it faithfully, carry it everywhere, and have it arrive uncorrupted. The container exists because we kept failing at the technical version of that: the thing that was true on the developer’s machine arrived broken on the server. Hold that question; we will return to it with the engineering in hand.
7.1 — Container vs VM: What Is Actually Being Shared
You already know what a virtual machine is. A hypervisor presents virtual hardware; a full guest OS boots on top of it; your app runs inside the guest. The isolation is excellent — each VM thinks it owns a whole computer — and the cost is excellent too: every VM carries its own kernel, its own init system, its own gigabytes of operating system you did not write and mostly do not use.
A container keeps the isolation but drops the duplicate OS. All containers on a host share the host’s single kernel. What makes them feel separate is not a second kernel — it is two Linux kernel features doing the work:
- Namespaces give a process its own view of the system. A PID namespace makes the container’s first process believe it is PID 1 and hides every other process on the host. A mount namespace gives it its own filesystem tree. Network, user, UTS (hostname), and IPC namespaces complete the illusion. The process is not in a different computer; it is wearing blinders that hide everyone else.
- Cgroups (control groups) limit and account for resources — this container gets at most 2 CPUs and 4 GB of RAM and no more. Namespaces decide what you can see; cgroups decide what you can use.
That is the whole trick. A container is a normal Linux process that has been put inside namespaces (so it sees only itself) and bound by cgroups (so it cannot starve its neighbors). There is no guest kernel. There is no virtual hardware. There is your process, isolated and capped.
| Dimension | Virtual Machine | Container |
|---|---|---|
| Virtualizes | hardware | the operating system |
| Kernel | its own guest kernel | shares the host kernel |
| Boot time | tens of seconds (full OS boot) | sub-second (just start a process) |
| Size on disk | gigabytes (whole OS) | tens to hundreds of MB (app + deps) |
| Density per host | handful to dozens | dozens to hundreds |
| Isolation strength | strong (hardware-enforced) | weaker (kernel-enforced, shared kernel) |
| Right when | strong isolation, different OS/kernel, untrusted tenants | dense packing, fast scaling, same-kernel workloads |
Read the last two rows together, because they are the tradeoff. The VM’s strong isolation is exactly the thing the container gives up to win its density and speed. A kernel bug is a shared kernel bug — a container escape is a host compromise, where a VM escape has one more wall to climb. So the honest rule is: containers for density and velocity among workloads you mostly trust; VMs (or the two combined) when the isolation boundary has to be hard. This is not “containers won.” In practice you run containers inside VMs constantly — the VM draws the hard security line, the container draws the fast deployment line.
Coach’s Note — Students reach for the word “lightweight VM” to describe a container, and it will get you in trouble. A container is not a small VM; it is a fenced process. The day you internalize that — there is no second kernel in there, it’s your host’s kernel wearing namespaces — is the day container networking, container security, and “why can’t my container
modprobe” all start making sense. The interactive lab below exists to drill exactly this.
7.2 — Images, Layers, and the OCI Standard
A running container starts from an image — a frozen, read-only template of a filesystem plus the metadata to run it (the default command, the environment, the exposed ports). The relationship is the one you already know from object-oriented programming: the image is the class, the container is the instance. One image, many containers.
Images are built in layers, and the layering is not a detail — it is the economics. Each instruction in a Dockerfile that changes the filesystem creates a new layer stacked on the ones before it. Layers are content-addressed and cached: if two images both start FROM python:3.12-slim, they share that base layer on disk and over the wire — it is pulled and stored once. This is why image order matters and why you put the lines that change rarely (installing dependencies) before the lines that change every commit (copying your source). Get the order wrong and every code change busts the dependency cache and rebuilds the world.
FROM python:3.12-slim # base layer — shared, cached, pulled once
WORKDIR /app
COPY requirements.txt . # changes rarely
RUN pip install -r requirements.txt # expensive — cache this above the source copy
COPY . . # changes every commit — keep it last
CMD ["uvicorn", "app:app", "--host", "0.0.0.0"]
The format is not Docker’s private property. The OCI (Open Container Initiative) standardized the image format, the runtime behavior, and the distribution protocol, which is why an image built by Docker runs under Podman, ships through any registry, and is pulled by Kubernetes. You are building to a standard, not to a vendor — the same lesson the OpenAI-compatible API will teach again later in the chapter. Bet on the standard.
A registry is where images live between build and run — Docker Hub, GitHub Container Registry (ghcr.io), Amazon ECR, Harbor for self-hosted. You push after you build and pull before you run. An image is named registry/repository:tag; the tag is a label like 0.1 or llama3.2:3b. Which brings us to the most common deployment sin in the field.
Coach’s Note — Never deploy
:latest.:latestis not a version — it is “whatever happened to be newest when this machine pulled,” which means two machines pulling on different days run different code while both swear they run “latest.” Pin a real tag, or better, pin the immutable digest (@sha256:...). Every Dockerfile in this chapter’scode/pins versions on purpose. When the agent hands you a Dockerfile with:latest, that is your first edit, every time.
7.3 — Docker, Podman, and the Build/Run Loop
The daily verbs are few, and you should be able to type them without looking:
docker build -t verse-api:0.1 . # build an image from the Dockerfile here, tag it
docker run --rm -p 8000:8000 verse-api:0.1 # run it; --rm cleans up on exit; map host:container ports
docker ps # what is running
docker logs <container> # what did it say
docker exec -it <container> bash # get a shell inside a running container
docker stop <container> # stop it
docker image ls # what images do I have
docker system prune # reclaim disk from stopped containers / dangling layers
Podman is the drop-in alternative worth knowing, because it differs in exactly the way that matters for administrators: it is daemonless and rootless by default. Docker (classically) runs a root daemon that every docker command talks to — a privileged, always-on process and a real attack surface. Podman runs containers as child processes of your user, no central daemon, no root. The command surface is so close that alias docker=podman works for most of what you do. On a hardened server, “no root daemon” is not a nicety; it is a smaller blast radius. Know both; reach for rootless when the host’s security posture demands it.
| Dimension | Docker | Podman |
|---|---|---|
| Architecture | client → root daemon (dockerd) | daemonless; fork/exec child of your user |
| Default privilege | daemon runs as root | rootless by default |
| CLI | docker ... | podman ... (near-identical; alias works) |
| Compose | docker compose (built in) | podman compose / podman-compose |
| Best when | broad tooling/ecosystem familiarity | hardened hosts, no privileged daemon allowed |
Both build and run OCI images, so an image is portable between them — the standard, again, is doing the work.
docker compose (one file, many services, one up) graduates you from single containers to a stack. You will use it heavily this week — it is how the model and its UI come up together. We get there in §7.5.
Coach’s Note —
-p 8000:8000reads host-port:container-port, and the order trips everyone once. The container always serves on the port the app binds inside the container;-pchooses what door on the host maps to it.-p 9000:8000means “the app listens on 8000 inside, reach it at 9000 outside.” Forget which side is which and you will stare at a “connection refused” for ten minutes. Predict the mapping before you run, then check.
7.4 — AI as the Tool: Generating (and Reviewing) the Dockerfile
Ask any 2026 coding assistant — Claude Code, GitHub Copilot, Cursor — to “write a Dockerfile for this FastAPI app” and you will get a competent file in seconds. This is genuinely useful: the agent remembers the multi-stage build pattern, the WORKDIR, the EXPOSE, the syntax you half-remember. Wielded well, it removes the boilerplate and lets you spend your attention on the parts that need judgment.
But a Dockerfile is a security and supply-chain boundary, and the agent does not know your threat model. Here is the review checklist I want burned into your hands, because the agent gets these wrong constantly:
| What the agent often writes | Why it is wrong | The fix you make |
|---|---|---|
FROM python:latest | unpinned base; non-reproducible builds | pin a real tag/digest (python:3.12-slim) |
(no USER line) | container runs as root → host root via shared kernel | add a non-root USER |
COPY . . early, then pip install | busts the dependency cache every commit; can copy secrets | copy requirements.txt and install before COPY . .; add a .dockerignore |
RUN pip install -r requirements.txt with no pins | floating deps; image is different every build | pin versions in requirements.txt |
secrets via ENV API_KEY=... | the secret is baked into a layer forever, visible to anyone who pulls | inject at runtime (-e, secrets mount), never ENV |
no HEALTHCHECK | orchestrator thinks “process alive” = “app healthy” | add a HEALTHCHECK that hits a real endpoint |
Look at this chapter’s code/Dockerfile: it is multi-stage (build deps stay in the builder, runtime stays lean), it creates and switches to appuser (uid 10001), it copies requirements.txt and installs before the source, and it ends with a real HEALTHCHECK. None of that is exotic. All of it is what a careful reviewer adds to the agent’s first draft.
The same review discipline applies to the compose file the agent writes — and the sins there are different ones. Agents routinely emit restart: always where you wanted unless-stopped, bind-mount the host’s whole working directory in (- .:/app) and ship that to production, expose a database’s port to the host that should only be reachable inside the network, and — the AI-workload special — forget the named volume on the model service so every redeploy re-pulls gigabytes. Read the compose file with the same eyes: what does each ports, volumes, and environment line grant, and would you grant it on the real network?
Coach’s Note — The spine rule of this whole book lands hard on Dockerfiles. The agent accelerates the typing; you own the verdict on whether the image is safe to ship. A confident, fast, sometimes-wrong partner that bakes a
:latestroot container with a hardcoded key into an image you push to a public registry is exactly the “powerful and dangerous in equal measure” partner the thesis warns about. Read every line. The agent proposes; you gate.
7.5 — AI as the Workload: Putting a Model in a Box
Now the other rail. The single most common containerized service you will deploy in 2026 is an LLM, and the ecosystem has sorted itself into a clean split. Learn the split; it is the whole decision.
The stack divides by concurrency and audience:
| Tier | Tool | Use it for | Scale |
|---|---|---|---|
| Personal / dev | Ollama, LM Studio, llama.cpp | one to a few users, laptops, quick local serving | ~1–3 concurrent |
| Production GPU serving | vLLM, SGLang | multi-user, multi-GPU, high throughput | many concurrent |
| Enterprise / supported | NVIDIA NIM | air-gapped, HIPAA/SOC2, vendor SLA | scaled, supported |
Ollama is the easy path. It serves GGUF-quantized models behind an OpenAI-compatible API on :11434, and getting a model running is two commands:
docker run -d -p 11434:11434 -v ollama-models:/root/.ollama --name ollama ollama/ollama:0.30.10
docker exec ollama ollama pull llama3.2:3b
# now talk to it on the universal API:
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Explain Matthew 28:19."}]}'
Notice the named volume -v ollama-models:/root/.ollama. Without it, the multi-gigabyte model you pulled vanishes when the container is removed and you re-download it every time. Model weights are state; state belongs in a volume, never in the container’s writable layer. That is a storage lesson (Chapter 4) wearing container clothes.
vLLM is the production path. Where Ollama is one barista, vLLM is a kitchen line: PagedAttention for efficient KV-cache memory and continuous batching so the GPU stays busy across many simultaneous requests. The metric flips from “does it run” to “tokens per second under load.” It also serves the same /v1 API:
docker run --gpus all -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:v0.23.0 \
--model meta-llama/Llama-3.2-3B-Instruct \
--tensor-parallel-size 1
(Version note: vLLM moves weekly — v0.23.0 is a mid-2026 snapshot and was reportedly yanked for a tensor-parallel bug; pin and re-verify the current line before a real deployment.)
A word on which model goes in the box, because it is a sizing decision you carry from Chapter 4. Open-weight models you can self-host span a wide range as of 2026 — from small, laptop-friendly options (llama3.2:3b, gpt-oss-20b runs on ~16 GB) up to single-GPU-class models (gpt-oss-120b fits on one 80 GB GPU) and large mixture-of-experts releases that need a node. Pick the smallest model that passes your task’s quality bar; a 3B model that answers correctly in a container on a CPU beats a 70B model that needs a GPU you do not have. The box is only as deployable as the model you choose to put in it.
Open WebUI puts a face on either backend — a self-hosted, ChatGPT-style UI with RAG, document upload, and RBAC. It is a front-end, not an inference engine; it talks to Ollama or vLLM over that same OpenAI-compatible API. (A licensing note worth knowing as an administrator: Open WebUI is the dominant self-hosted front-end, but it adopted a branding-protection license in 2025 that adds conditions above a user threshold — read the license before you deploy it organization-wide, the same way you would for any dependency.) This separation is the architecture: the engine serves tokens, the UI serves humans, and they meet at /v1.
The lingua franca is the payoff worth stating plainly. The OpenAI-compatible /v1/chat/completions endpoint is the SSH of model serving. Point your code’s base_url at Ollama (:11434/v1), llama.cpp (:8080/v1), LM Studio (:1234/v1), vLLM, SGLang, or NIM, add a dummy key, and the same client code runs unchanged. That is exactly why this chapter’s code/app.py reads its backend from one environment variable and nothing else moves. Bet on the standard, not the vendor — twice in one chapter.
Coach’s Note — “Which serving tool?” is a right-tool-for-the-job decision, and the matrix is short: laptop or demo → Ollama. Embedded/CPU/edge → llama.cpp. Startup self-hosting at real concurrency → vLLM or SGLang. Enterprise that needs a support contract, air-gap, or HIPAA → NIM. Memorize that ladder. The widget below builds the exact
docker run/compose for whichever rung you pick.
7.6 — GPUs in Containers: the NVIDIA Container Toolkit
A container shares the host kernel, but a GPU is not automatically visible inside it — you have to grant it. The mechanism is the NVIDIA Container Toolkit, which installs a runtime hook that injects the GPU devices and the right driver libraries into the container at start. Install the toolkit on the host (the host carries the GPU driver), then add one flag:
docker run --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi # all GPUs
docker run --gpus '"device=0,1"' ... # specific GPUs
If nvidia-smi runs inside the container and lists the card, the path works end to end. In Compose, the same grant is the deploy.resources.reservations.devices block shown (commented) in this chapter’s code/compose.yaml. This connects directly to last week: the partition you carved in Chapter 6 — a whole card via passthrough, or a MIG slice like 1g.10gb — is what you then expose to the container. Virtualization decides how the card is divided; the Container Toolkit decides how a container gets its slice. One mode at a time on a node, as you learned: a node serving containers is not simultaneously serving passthrough VMs.
Coach’s Note — The number-one GPU-in-container failure is a host/driver/CUDA mismatch — the toolkit installed but
--gpus allyields “could not select device driver.” The container carries the CUDA runtime; the host carries the driver; they must be compatible. When it breaks, runnvidia-smion the host first (driver OK?), then in a stocknvidia/cudacontainer (toolkit OK?), then in your image (your CUDA OK?). Bisect the stack; do not guess.
7.7 — Volumes and Networks: Where State and Connections Live
Two facts about containers trip up administrators coming from VMs, because the defaults are opposite to what you expect.
A container’s filesystem is ephemeral by default. Everything a container writes lands in a thin writable layer that is destroyed when the container is removed. That is a feature, not a bug — it is what makes containers reproducible and disposable. But it means anything you need to keep must live somewhere else. That somewhere is a volume:
- Named volumes (
-v ollama-models:/root/.ollama) — Docker manages the storage; survivesrm; the right home for model weights, databases, and any AI asset you do not want to re-download. This is the one you reach for. - Bind mounts (
-v /host/path:/container/path) — map a specific host directory in; great for injecting config or watching source in development; couples you to the host’s layout. - tmpfs — in-memory, vanishes on stop; for secrets or scratch you explicitly do not want persisted.
The rule from §7.5 restated as a law: weights are state, and state belongs in a volume. A model in the writable layer is a model you re-download.
A container’s network is its own namespace. Inside, the app binds a port; that port is invisible to the host until you publish it (-p) or join a shared network. On a user-defined Docker network — which Compose creates for you automatically — containers reach each other by service name: in code/compose.yaml, Open WebUI reaches the backend at http://ollama:11434, not localhost. That is built-in DNS for service discovery, and it is why a compose stack just works once the services are named. localhost inside a container means that container, not the host — the single most common “why can’t they talk” bug. To reach the host itself from a container, use host.docker.internal (Docker Desktop) rather than localhost. That hostname only resolves on Docker Desktop, though — on a Linux host, which is the cloud/no-admin path from Appendix A (Path A), run the container with --network=host and point it at http://localhost:11434, or just use the compose service name http://ollama:11434.
Coach’s Note — Two opposite defaults to memorize: a container forgets its disk and hides its ports, both by design. From a VM background you expect the reverse — VMs persist their disk and you firewall ports shut. Flip your intuition for containers: assume the data is gone unless you put it in a volume, and assume nothing can reach the port unless you published it or joined a network. Internalize those two and most “it lost my data” / “it can’t connect” tickets answer themselves.
7.8 — From One Box to Many: Orchestration on the Horizon
You can run a stack by hand with docker compose up. But what restarts a container when it crashes at 3 a.m.? What spreads ten replicas across five hosts? What rolls out a new image without dropping requests, and rolls back when the new one is bad? Compose does none of that across machines. Orchestration does, and the industry standard is Kubernetes.
You are not learning Kubernetes this week — it earns its own treatment through the operations half of the course. But you should leave Week 7 holding the vocabulary, because it is built from exactly what you just learned:
| Term | What it is | Built on |
|---|---|---|
| Pod | one or more containers scheduled together, sharing a network namespace | the container of §7.1 |
| Deployment | ”keep N replicas of this pod running”; reconciles reality to that wish | a control loop over pods |
| Service | a stable address in front of pods that come and go | the service-discovery DNS of §7.7 |
| Ingress | routes outside traffic to services | the published-port idea, at cluster scale |
Kubernetes is not magic — it is a control loop wrapped around the container primitives you spent this chapter learning. A node serving AI inference at scale in 2026 typically runs Kubernetes scheduling vLLM pods, with the NVIDIA GPU Operator handling the --gpus grant you did by hand in §7.6, and a serving layer like KServe in front. Every one of those pieces is a container concept promoted to a cluster. Master the box and the orchestrator is the next, smaller step rather than a cliff. The same step holds for AI workloads specifically: an Ollama you ran by hand becomes a deployment of replicas, the model volume becomes a persistent claim, and the /v1 endpoint becomes a service — same nouns, more of them.
7.x — Interactive Lab: Container vs VM Explorer
Below this chapter on the site is the Container vs VM Explorer — use it now, before the pitfalls. It has two halves, and each drills a different half of this chapter.
The left half is the architecture toggle. Flip between VM and Container and watch the diagram redraw: in VM mode you see the hypervisor and a full guest kernel stacked under each app; in container mode the guest kernels collapse into the one shared host kernel, with namespaces and cgroups drawing the fences. The panel updates boot time, image size, density-per-host, and isolation strength as you toggle. Your job is to predict each number before you flip — say out loud “the container boots faster and isolates weaker because it shares the kernel,” then confirm it. That one sentence is the whole of §7.1; the widget makes you feel it instead of memorize it.
The right half is the run-command builder. Choose a serving tool (Ollama / vLLM), a model, and whether you want a GPU, and it assembles the exact docker run or compose service for that choice — the right -p, the right -v for model persistence, the --gpus flag only when you asked for it, a pinned version instead of :latest. Build the command for “Ollama, llama3.2:3b, no GPU,” then rebuild for “vLLM, a 3B instruct model, one GPU,” and read the diff. You are watching the §7.5 decision ladder turn into a command. Copy what it builds; it is a correct starting point you will then review the way §7.4 taught you.
What it teaches, in one line: a container is a fenced process sharing the host kernel, and serving a model is choosing the right rung of the tool ladder and grabbing the matching command.
7.9 — The Steward and the Sending Out
Now the question we left open. What does it take to send something out, whole, into the world?
The Great Commission — “Go therefore and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit, teaching them to observe all that I have commanded you” (Matthew 28:19–20, ESV) — is a sending with a specification attached. It is not “go improvise something religious.” It is “carry this, do these things, teach all that I commanded — and lo, I am with you always.” Faithfulness in a sending is faithfulness to the thing being sent: it must arrive whole, not corrupted in transit, not quietly altered by the carrier to suit local taste.
That is, with the spiritual stakes removed, the exact problem a container solves. “It works on my machine” is a sending that arrived corrupted — the thing that was true at the source was not true at the destination, because the carrier (the runtime, the OS, the missing library) altered it on the way. An image is a discipline against corruption: pin the base, pin the dependencies, declare the environment, and what you tested is bit-for-bit what runs in the field. The whole art of the Dockerfile is fidelity in transmission.
The administrator is a steward here in two directions at once. You steward what you send — you do not ship the :latest, root-running, secret-baked image, because an unfaithful package corrupts the thing it carries and endangers whoever receives it. And you steward what you run on behalf of others — the model in the box is not yours; it serves people who trust that what comes out is true. This is why code/app.py tags every model answer "verified": false. The box can transmit the model’s output faithfully and the output can still be wrong. Fidelity in transmission is not the same as truth in content. A faithful courier can carry a false message perfectly. The container guarantees the first; only a human in the loop can guard the second.
So the LCMS frame is not decoration this week. Vocation is being faithful in what is entrusted to you — and the administrator who packages and sends out the organization’s services is doing, in miniature and in silicon, the oldest job there is: carry the thing entrusted to you, whole, to the people who need it, and do not let the carrying corrupt it. Build it faithfully. Ship it faithfully. And never confuse a clean delivery with a true message.
7.10 — Common Pitfalls
Pitfall: Deploying :latest.
Example: Two nodes pull ollama/ollama:latest a week apart and silently run different versions; a bug reproduces on one and not the other.
Fix: Pin a real tag (ollama/ollama:0.30.10) or an immutable digest (@sha256:...). Treat :latest as “unknown version.”
Pitfall: Running the container as root.
Example: A Dockerfile with no USER line; a process escape lands the attacker as uid 0 on the host through the shared kernel.
Fix: Create and switch to a non-root user (USER appuser). Drop capabilities you do not need. Reach for Podman’s rootless mode on hardened hosts.
Pitfall: Storing model weights (or any state) in the writable container layer.
Example: ollama pull downloads 4 GB into the container; docker rm deletes it; the next run re-downloads everything.
Fix: Mount a named volume for the model cache (-v ollama-models:/root/.ollama). State lives in volumes, not in the container.
Pitfall: Baking secrets into an image with ENV or COPY.
Example: ENV OPENAI_API_KEY=sk-... — the key is now in a layer, visible to anyone who pulls the image, even after you delete the line in a later layer.
Fix: Inject secrets at runtime (-e, a secrets mount, the orchestrator’s secret store) and add a .dockerignore so .env never enters the build context.
Pitfall: Confusing the -p host:container port order.
Example: App binds :8000 inside, you run -p 8000:9000, then “connection refused” because nothing inside listens on 9000.
Fix: Remember -p HOST:CONTAINER. The right side is the port the app actually binds inside the container.
Pitfall: Trusting an agent-generated Dockerfile or compose file without review.
Example: The agent emits FROM python:latest, no USER, COPY . . before install, and a hardcoded key — all four sins in one file.
Fix: Run the §7.4 review table over every generated file. The agent proposes; you read every line and gate the ship.
Pitfall: Assuming a GPU is visible inside the container by default.
Example: docker run vllm/... with no --gpus, then “no CUDA devices found” inside.
Fix: Install the NVIDIA Container Toolkit on the host and pass --gpus all (or '"device=0"'). Verify with nvidia-smi inside the container.
7.11 — Reps
Open the exercises for the full set. This is Phase 1, so the AI policy holds: do it by hand first, then bring AI in to critique or accelerate — and you own the verdict. This week that means you write a Dockerfile by hand before you ask an agent for one, and then you review the agent’s version against yours and write down what you would change. A short Check Your Reps quiz sits on this page; answer it before you move on.
A preview of the reps:
- Rep 1 — Build and run your first image; map the port; read the logs.
- Rep 3 — Watch the layer cache work (and break it) by reordering Dockerfile lines.
- Rep 5 — Pull a model into Ollama in a container with a persistent volume; lose it once on purpose by omitting the volume.
- Rep 6 — Split front-end from back-end: stand up Ollama + Open WebUI with
docker compose; talk to the model through the UI. - Rep 9 — Generate a Dockerfile with an agent and run the §7.4 review table over it in writing.
Type every command. Predict every port mapping and image size before you run. Read every error — a container error is almost always the kernel telling you exactly which fence you hit.
7.12 — This Week’s Project
You’re ready for Project 7 — Ship a Model in a Box, in Project 7.
You will containerize a small AI service end to end: a local LLM served behind the OpenAI-compatible API, fronted by a tiny app, packaged in a reviewed, hardened image and a reproducible compose stack — and you will prove it serves with a smoke test, not just “it started.” The Normal tier gets the stack up with a pinned, non-root image and a persistent model volume. The Medium tier adds Open WebUI and a clean front-end/back-end split. The Hard tier is the architect’s deliverable: a serving-tool decision memo (Ollama vs vLLM vs NIM) for a named scenario, plus a line-by-line security review of an agent-generated Dockerfile — judgment an agent cannot supply for you. Toolchain and submission live in Appendix A; local-and-cloud AI setup in Appendix B.
7.13 — Coach’s Final Word
Last week you divided one machine. This week you sent something out. Those are the two motions of administration — carve what is scarce, ship what is finished — and you now have both in your hands at the level where the AI era actually lives: a GPU sliced for tenants, a model boxed for the world.
Hold onto the one idea under all of it: a container is a fenced process sharing the host kernel, and an image is a discipline against corruption in transit. Everything this week was a corollary. The OpenAI-compatible API is the same “bet on the standard” lesson the OCI taught — one interface, every backend. The review table is the same “the agent proposes, you gate” lesson that runs through the whole book. And the box that ships your model faithfully still cannot make its message true — that verdict is yours, and stays yours.
The Great Commission is a sending with a specification attached: carry this, whole, to everyone, and do not let the carrying corrupt it. That is the administrator’s quiet vocation too. Build it faithfully. Ship it faithfully. Verify before you trust. Then send it out.
See you next week.
Up next: Complete every rep in the exercises. Then build Project 7 — Ship a Model in a Box. Set up your toolchain with Appendix A, your local-and-cloud AI with Appendix B, and review the agentic-AI rules in Appendix C. Then Chapter 8 — Infrastructure as Code, and the Limits of the Servant, where the agent stops writing one Dockerfile and starts writing your whole estate — and the midterm arrives.
Check Your Reps — Containers and the Sending Out
docker run -d -p 11434:11434 -v ollama-models:/root/.ollama ollama/ollama:0.30.10