Milestone 13 — Documentation Set & the Clean-Machine Test

“…what you have heard from me in the presence of many witnesses entrust to faithful men, who will be able to teach others also.” — 2 Timothy 2:2 (ESV)

Chapter: Chapter 13 — Documentation Is a Deliverable Week: 13 of 16 Due: End of Week 13. The clean-machine test must be run before the deadline, not described as a plan. Submit: Your repository link plus the commit SHA of the documentation commit, and the completed docs/clean-machine-test.md, to the Week 13 dropbox. Where it lands in the final package: README.md, docs/architecture.md, docs/runbook.md, docs/onboarding.md, docs/adr/README.md, the successor-facing section of docs/ai-usage.md, and docs/clean-machine-test.md. These feed the documentation lines of the Week-16 rubric — the largest single block inside the 50% implementation-and-documentation component. The clean-machine record is also the evidence for the Week-16 acceptance test. See Appendix C. AI policy: Open, with one hard boundary. An assistant may draft, restructure, and tighten any document here. It may not be the source of a single command, path, version, port, or expected output that you have not executed yourself. Every AI-drafted section gets a row in docs/ai-usage.md naming what it drafted and what you found wrong when you ran it. The clean-machine test itself is closed to AI — a model cannot be your tester, and a model’s confident “this should work” is exactly the thing the test exists to disprove.

The Setup

It is Week 13. You have a release candidate, a green test suite, and a change log. You also have a project that exactly one person on earth can run, and you are that person.

Here is the scenario, and it is not hypothetical — some version of it will happen to you within two years of graduating. A team lead you have never met is deciding whether to adopt your project, hire you, or hand it to a junior engineer. They have your repository open and roughly fifteen minutes of goodwill. They will not email you. They will not open an issue. They will clone it, try it, and form a conclusion you will never get to correct.

There is a second reader, and this one is certain rather than hypothetical: the grader who opens your repository in Week 16. That reader has the same fifteen minutes and a rubric. Every documentation line on that rubric is a line you can earn this week, while there is still time to fix what you find, or lose in Week 16, when there is not.

So this week you produce the documents that speak for you in both rooms, and then you run the only test that tells you whether they work: you hand your repository to a real person on a clean machine and sit silently while they use it. The documents are the deliverable. The test is the proof.

The milestone is not “write documentation.” It is prove that a stranger can run your project without you — and record honestly what happened when you tried.

What each artifact is worth in Week 16

The milestones are graded twice — once on their own and again inside the final — and this is the week where that fact bites hardest. Skipping Milestone 13 does not save you points; it forfeits the milestone and relocates the rest to a week that has none of the hours they need. The full accounting is in Appendix C.

Learning Targets

You will demonstrate that you can:

• Write a README to a professional standard — eleven sections, versioned prerequisites, every command paired with its real expected output, and honest non-goals.
• Explain a system on one page — a component diagram and five paragraphs that let a competent engineer hold your architecture in their head, including the part that would surprise them.
• Write operational documentation from evidence — a runbook whose failure catalogue comes from your own defect log, not your imagination, and which names the irreversible commands.
• Prepare a successor — an onboarding guide with a real first task, and a decision record index that preserves reasoning, including the decisions you would want reversed well.
• Document the parts that can change without your code changing — model versions, prompts, cost, and what must not be assumed stable.
• Run an empirical acceptance test on your own work — recruit a tester, stay silent, log friction, fix, and re-run.
• Use an assistant as a drafter and yourself as the verifier, and report the correction rate without rounding it down.

The weekly status block — paste this into your submission comment

Every milestone submission carries these five lines. It takes two minutes, it is the same five lines every week, and it is the difference between an instructor who can help you in Week 6 and one who finds out in Week 14. Copy it into the Canvas submission comment when you submit — not into a document.

Shipped:     what actually works this week, in one line
Cut:         what I dropped or deferred, and where it went (backlog / change request / gone)
Hours:       planned N / actual N        (from docs/hours-log.csv)
Blocked on:  the one thing in my way, or "nothing"
Ship confidence: green | amber | red — if amber or red, the one thing that would move it

Say red when it is red. A red in Week 6 is a conversation and a scope cut. A red discovered in Week 15 is a grade. Nobody has ever been penalised in this course for an honest amber; the only expensive answer is the green that was not true.

Normal Tier

Required deliverables

1. README.md covering all eleven sections: what it is and who it is for (with non-goals), demo, prerequisites with versions and check commands, install, configure, run, test, project layout, troubleshooting, license, contact and further reading. Every command paired with its expected output. Start from code/readme-template.md.
2. docs/architecture.md — a one-page overview added to the front of the file you started in Week 6, as a new §0 — Overview section: a component diagram (four to eight boxes, labeled arrows) and five paragraphs, the last of which names the one thing that would surprise a new reader. Do not condense the Week-6 sections away to make room for it. The module responsibilities, interface contracts, data model, and critical-path flows stay in the file underneath §0 and get updated as built this week. That is a 10-point Week-16 rubric line and a grader compares those sections to your code; a file that has been reduced to a diagram and five paragraphs loses most of it.
3. docs/runbook.md — from code/runbook-template.md: where it runs, start/stop/restart, the health check with healthy and unhealthy output, routine operations, normal readings, at least five failure entries traced to real defects, escalation, and a last-verified date and name. At least one entry names a destructive command the operator must not run.
4. docs/onboarding.md — the ten-minute orientation with your project’s vocabulary, a five-row feature-to-code map, a real unclaimed first issue written up for a stranger, how work moves, and where decisions live.
5. docs/adr/README.md — an index of every architecture decision record with number, decision, current status, and the consequence a reader would notice. Includes at least one record written this week for a decision that had never been recorded.
6. The successor-facing section of docs/ai-usage.md — for an AI feature: behavior and failure behavior, exact model identifier with the date pinned, the file the prompt lives in, parameters, cost measured by you on a stated date with a note that pricing changes, and what must not be assumed stable. If your project has no AI feature, do the same six fields for your most volatile third-party dependency.
7. docs/clean-machine-test.md — the completed record from code/clean-machine-protocol.md: run header, stopwatch table, the acceptance action defined before the run, the friction log with the tester’s own words, the fixes applied with commits, and a second run after the fixes.
8. The commands verified. Every command in every document above has been executed by you, in order, from a clean state, and its real output pasted in.

What makes the clean-machine run valid

A run that breaks any of these is void and must be repeated. They are not formalities; each one exists because it is the rule students break.

1. A genuinely clean environment — fresh container, fresh cloud development environment, borrowed laptop, or a new user account. Nothing your project installed is already there.
2. A real tester who is not you, competent and uncoached.
3. Silence. You may watch and take notes. You may not speak, point, type, or fix.
4. The repository URL is the only input. No verbal preamble, no messages, no “oh, you also need to…”.
5. The clock runs from opening the repository to completing the acceptance action, which you defined before the run. Stop at sixty minutes and record the failure.
6. Every stumble is logged, including the ones that look like the tester’s fault.

Normal-tier rubric (out of 100)

Read the fourteen points for the clean-machine test carefully. They are awarded for a real run, honestly recorded — including one that went badly. A run where the tester got stuck four times, you logged all four, fixed them, and re-ran scores higher than a suspiciously frictionless run with no friction log. A test you did not actually run scores zero, and it is easier to spot than students expect: real friction logs have the tester’s own words in them.

Medium Tier (+ up to 25% extra credit)

Pick what genuinely fits your project. Two done well beat five done thinly.

• A demo that loads before the reader gives up. A 60–90 second screen recording or an animated GIF at the top of the README, committed to the repository (not hotlinked from somewhere that may disappear). For a CLI, a real terminal session with real output.
• One-command bootstrap. A make setup target, a script/bootstrap, or a dev container definition that collapses your documented steps into one command — and which your clean-machine tester actually used. Write it after the first run, so it automates the steps that proved to matter.
• Documentation tested in CI. A job that, on a clean runner, executes the README’s install/configure/run/test sequence and fails when a documented command breaks. This is the only way documentation stops rotting. Wiring code/doc_lint.py into the same job counts toward this.
• A second clean-machine run on a different platform — different OS, different architecture, or a browser-based environment versus a local one — with both records kept and the platform-specific differences documented.
• A troubleshooting section built entirely from evidence, every row traceable to either a real defect or a real friction entry, quoting the symptom in the words the person actually used.
• A documentation accessibility pass. Alt text on every image, tables with real headers, no instruction that depends on color alone, headings in a sensible order. The W3C Web Accessibility Initiative’s guidance at w3.org/WAI is the reference. Note what you changed and why.

Two rules govern this tier. First, anything you claim here must be demonstrated in the repository — a bootstrap script that exists but was never run by your tester earns nothing. Second, say in your submission which medium-tier items you attempted; a grader who has to go hunting will find fewer of them than you built.

Hard Tier (+ up to 25% additional extra credit)

Required: a memo no assistant can write for you. Two to three pages, in the repository, addressed to the engineer who inherits this project.

The Undocumented Memo

An honest account of the limits of your own documentation set. It must do four things:

1. Name what a successor still cannot learn from this repository. Not what you forgot — what you chose not to document. The judgment call you make by feel. The reason you distrust one module. The workaround whose real cause you never found. Be specific enough that a reader could go look at the code and see what you mean.
2. Price each gap. For every item, estimate what documenting it properly would cost in hours, and what leaving it undocumented will cost the successor in hours — and then say which side of that trade you chose, and why. “I chose not to spend six hours documenting the import pipeline’s edge cases because it will be rewritten before it is extended” is a defensible engineering decision. “I ran out of time” is a report, not a decision.
3. State your bus factor honestly, with evidence. After this week’s work, how many people could keep this project alive? What specifically would have to be true for the answer to be two? Point at the artifact that would have to exist.
4. Name the one document you would delete, and defend it. Every documentation set contains something that costs more to maintain than it returns — a duplicated setup section that will drift, a diagram that will be wrong within a month, a file nobody will open. Identify yours and argue for its removal. Then either delete it or explain why you kept it anyway.

This tier is graded on judgment, honesty, and self-knowledge. An assistant can list documentation gaps; it cannot tell you which ones are worth the hours of the person who inherits your specific project, and it cannot be accountable for the call. That is the whole point of the tier.

A memo that says “documentation could always be improved” earns nothing. A memo that says “I did not document why the sync job retries with a five-second backoff, because I never determined whether the upstream rate limit is per-key or per-IP; a successor will lose about a day finding that out, and the way to find it is X” earns full marks. Specific, priced, honest, actionable — in that order.

Submission

Commit the documentation set, then submit the repository link and the commit SHA.

your-capstone/
├── README.md                     # the eleven sections, every command verified
├── LICENSE                       # chosen in Week 5; verify it matches the README
├── CHANGELOG.md                  # from Week 12
├── docs/
│   ├── architecture.md           # EXTENDED — §0 overview above the Week-6 spec
│   ├── runbook.md                # NEW — operations + five real failure entries
│   ├── onboarding.md             # NEW — orientation, code map, first issue
│   ├── clean-machine-test.md     # NEW — the recorded run, friction log, re-run
│   ├── ai-usage.md               # EXTENDED — the successor-facing section
│   ├── undocumented-memo.md      # Hard tier only
│   ├── requirements.md           # from Weeks 3–4
│   ├── test-plan.md              # from Week 11
│   ├── defect-log.md             # from Week 11 — the runbook's source material
│   └── adr/
│       ├── README.md             # NEW — the decision record index
│       ├── 0001-choose-the-stack.md
│       └── ...
└── src/  tests/  script/

Suggested commit message: docs: documentation set + clean-machine test (milestone 13).

Also update your hours log for the week. Week 16 asks for the full record, and reconstructing thirteen weeks of hours in the last week is a miserable, inaccurate job.

Hints

• Do the reps in order. The exercises are sequenced so each one feeds the next: the audit tells you what is missing, the writing reps fill it, the verify reps prove it.
• Write the non-goals first. The “what it does not do” paragraph is the hardest ten minutes in the README and the paragraph that makes an experienced reader trust everything else you wrote.
• Recruit your tester on Monday, not Thursday. The single most common way this milestone goes wrong is discovering on Thursday night that nobody is available. Ask three people; one will say yes.
• Define the acceptance action before the run. Not “the server started.” Something a user would recognize as the system working.
• Sit on your hands. Print rule 3 and put it in front of you: you may watch, you may not help. The sentence you want to say out loud is the sentence missing from your README — write it down instead.
• Mine the defect log for the runbook. You already did the hard part in Week 11. Five real defects become five real failure entries in about an hour.
• Use the assistant for gaps, not for facts. “You have never seen this project — list every question you still cannot answer” is the prompt that earns its keep. Then verify everything it hands you.
• Run code/doc_lint.py last, not first. It catches the mechanical failures a grader sees immediately, but a clean report proves nothing about whether a person can follow your instructions.
• Budget the week the way the chapter does. Roughly 3.5 hours on the README, 2 on the architecture page, 2.5 on the runbook, 2 on onboarding plus the ADR index plus the AI section, 2 on the test, 1.5 on the fixes and the re-run. If the README takes forty-five minutes, you tidied it rather than rewrote it.
• Do not wait for the code to be finished. It will not be. Document what exists today and mark the rest honestly; a README describing a project’s ambitions is worth less than one describing its actual current state.

If something goes wrong

• No tester available. A fresh container or cloud development environment plus a strict solo run is the fallback — but say so plainly in docs/clean-machine-test.md, and expect to lose part of the fourteen points. Self-testing cannot find what you cannot un-know. Then recruit someone for a second run before Week 16.
• The tester failed to reach the acceptance action. Record the failure at the checkpoint it stopped, log the friction, fix, and run again. A recorded failure followed by fixes and a successful second run is a full-credit outcome. Two failed runs with an honest log still scores far above a fabricated success.
• You broke rule 3 and helped. Note it in the log, void the run, and start over. Then treat every sentence you said out loud as a top-priority documentation fix.

What Mastery Looks Like

A mastered submission is one where the clean-machine record is boring.

Not because nothing went wrong — something always goes wrong on the first run — but because the author expected it, wrote it down in the tester’s own words, fixed it, and ran it again, and the second run took eleven minutes. The README states what the project does not do before it states what it does. The runbook names the command that destroys data and tells the operator not to run it. The architecture page has a sentence that begins “the thing that would surprise a new reader is,” and it is true. The ADR index contains a record written thirteen weeks late, dated honestly, for a decision the author had never admitted making. And docs/ai-usage.md says, in plain numbers, how many of the assistant’s commands were wrong.

None of that is impressive writing. All of it is trustworthy writing, and at this level trustworthy is the higher bar. A grader reading a submission like that stops checking, because the author has already checked.

Coach’s Note — You are going to be tempted to run the clean-machine test with your most sympathetic friend, on a machine that already has half your stack installed, while narrating helpfully. That version of the test will make you feel good on Thursday and cost you real points in Week 16, when a grader runs the honest version and it fails. Run the hard version now, while there are three weeks left to fix what it finds. The whole reason this milestone sits in Week 13 instead of Week 16 is that in Week 13 you still have time.

When You’re Done

• README.md scores 20+ out of 22 on the eleven-section audit, and the README Grader agrees
• Every prerequisite pinned to a real version, with a check command and a named failure mode
• Every command in every document executed by you, with real output pasted in
• docs/architecture.md: §0 overview — diagram plus five paragraphs, surprise named, ADRs linked — with every Week-6 section still present and updated as built
• docs/runbook.md: health check, normal readings, five real failure entries, destructive commands flagged, last-verified date
• docs/onboarding.md: vocabulary, feature-to-code map, a real first issue left undone
• docs/adr/README.md: complete index, statuses current, the previously unrecorded decision written
• docs/ai-usage.md: successor-facing section, model or vendor pinned with a date, instability named
• Clean-machine test run with a real tester on a clean environment, in silence
• Friction log written in the tester’s words, fixes committed, second run recorded
• python3 doc_lint.py README.md reports zero errors
• Hours log updated; everything committed and pushed

A theological footnote. Habakkuk is told, “Write the vision; make it plain on tablets, so he may run who reads it” (Habakkuk 2:2, ESV) — and notice the purpose clause. Not so the vision is preserved. Not so the writer is credited. So that the one who reads it can run. Writing plainly is an act performed for somebody else’s speed, and it costs the writer time in order to buy a stranger time. That is the whole economics of documentation, and it is also, in miniature, what Paul asks of Timothy: hand on what you received, prepared for hands that are not yours, to people who can hand it on again (2 Timothy 2:2). This week you are not being graded on eloquence. You are being graded on whether someone else can run — and the clean-machine test is simply the moment you stop assuming they can and go find out. Do it honestly. An honest failure this week is a gift; a comfortable illusion is a debt you will pay in Week 16.