Milestone 6 — Tested Release Candidate, Defect Log & Change Log

“Who can discern his errors? Declare me innocent from hidden faults.” — Psalm 19:12 (ESV)

Chapter: Chapter 6 — Test, Integrate, and Cut Week: 6 of 8 · roughly 20 hours Due: End of Week 6, committed and pushed Submit: A push to your repository, plus the release tag name and the commit SHA it points at. Commit message: milestone 6: tested release candidate, defect log, change log Where it lands in the final package: docs/test-plan.md, docs/traceability.csv, docs/test-results.md, docs/defect-log.md, new files in docs/change-requests/, the change table and release sections of CHANGELOG.md, the tests/ directory, the annotated tag itself, plus updates to docs/requirements.md, docs/architecture.md, docs/ai-usage.md, and docs/hours-log.csv. These feed the working software, verification evidence, defect log, change control, and release engineering lines of the Week-8 rubric — the 50% component. See Appendix D for the exact point values. AI policy: Open, with the spine rule. An assistant may propose test cases, edge cases, and partitions — one of its strongest uses, and you should use it. It may help you debug across a seam and run a first-pass security review. Only you may approve an assertion, because an assertion is a claim about what your software is supposed to do and you signed the requirements. It may not be the source of a security finding: every advisory identifier is verified at https://github.com/advisories or https://osv.dev/ before it goes in a document. It may not decide a change request; the impact analysis and the disposition are yours, in your words, with your name and date on them. Log every substantive use in docs/ai-usage.md.

The Setup

Your core works. You demoed it last week and it did what you said it would do.

Now imagine somebody else in the room. Not a friendly grader — a QA engineer on their third coffee, who has never seen your project, who has your requirements specification open on one monitor and your running application on the other, and whose entire job this afternoon is to find the places where those two things disagree. They will not use your software the way you use it. They will type an apostrophe into a name field. They will set a date to today, then to yesterday, then leave it blank. They will submit the form twice. They will unplug the network mid-request.

That person is you, this week, and the hardest part is that you already know where the bodies are buried.

Then, on Thursday, you become somebody else again: the person who counts. You are six weeks into eight. About 120 of your 160 hours are gone, and the 40 that remain belong to Week 7 (documentation, deployment, handoff) and Week 8 (final package, presentation, delivery). Your discretionary build time for the rest of this project is what is left of this week plus a reserve you protect — call it ten hours. Everything in your fix queue has to fit inside that, and everything that does not fit gets written down as a decision rather than left as an accident.

That is what this milestone is: the unease converted into artifacts, and the arithmetic converted into decisions. A plan that says what “done” means in numbers. A matrix that maps every promise to the test that verifies it. A suite that runs from one command. A log that records, in your own name, every fault you found. A folder of change requests that shows what you were asked for and what you did about it. And a tagged candidate that clones clean on a machine that has never seen your project.

Milestone 6 is graded twice — and by now you know exactly what that means. It scores in the 25% milestone bucket, and these artifacts then go straight into the Week-8 package that carries 50% of your grade. Producing them this week costs twenty hours. Producing them in Week 8, alongside deployment, documentation, a handoff package, and a thirty-minute talk, costs the grade.

Learning Targets

You will demonstrate that you can:

• Write a test plan with a scope, an honest out-of-scope section, named levels, a reproducible environment, and exit criteria a stranger could check.
• Trace every requirement to the test cases that verify it, and report verified-requirement coverage rather than line coverage.
• Derive test cases from requirements, not from code — including partitions, boundary values, and negative cases.
• Write a defect report a stranger could act on, with all seven required fields.
• Triage with severity, priority, and hours as three separate inputs, and produce a fix queue that fits a real remaining budget.
• Practice regression discipline: failing test first, then the fix, then proof the test was ever red.
• Integrate independently built components and prove the Tier 1 seams hold with tests, not hope.
• Measure against your own non-functional targets, with a stated method and a tail statistic, and run a structured security pass with every finding dispositioned.
• Perform change control: a three-line impact analysis and a defensible ACCEPT / DEFER / REJECT for every request against the Week-4 baseline.
• Cut scope honestly, in writing, and produce a release candidate that a stranger can clone and run.

The weekly status block — paste this into your submission comment

Every milestone submission carries these five lines. It takes two minutes, it is the same five lines every week, and it is the difference between an instructor who can help you in Week 3 and one who finds out in Week 7. Copy it into the Canvas submission comment when you submit — not into a document.

Shipped:     what actually works this week, in one line
Cut:         what I dropped or deferred, and where it went (backlog / change request / gone)
Hours:       planned N / actual N        (from docs/hours-log.csv)
Blocked on:  the one thing in my way, or "nothing"
Ship confidence: green | amber | red — if amber or red, the one thing that would move it

Say red when it is red. A red in Week 3 is a conversation and a scope cut. A red discovered in Week 7 is a grade. Nobody has ever been penalised in this course for an honest amber; the only expensive answer is the green that was not true.

Normal Tier

Required deliverables

1. docs/test-plan.md — from code/test-plan-lite.md. Five sections filled: scope (named commit), out of scope with reasons, levels, environment, and exit criteria as checkable numbers, written before you evaluated against them, with a pass/fail result recorded for each.

2. docs/traceability.csv — from code/traceability-matrix.csv. One row per acceptance criterion: requirement ID, MoSCoW priority, criterion ID, test case ID, level, status, defect ID. Every Must requirement has at least one test case. Requirements with none appear as gaps rather than being quietly omitted.

3. An automated suite in tests/, run by ./script/test, green in the CI pipeline you built in Week 3. At minimum: unit tests over your core logic with boundary and negative cases for every requirement containing a number, a date, a length, or a count; at least two integration tests crossing Tier 1 seams from your Week-3 design; and acceptance tests derived one-to-one from acceptance criteria, at least one per Must requirement, each naming the criterion it verifies. Test data checked in, generated from a fixed seed, seeded by one command, containing no real personal data.

4. docs/test-results.md — the verification record, in three sections. The run: commit SHA, date, command, pass/fail counts, duration, and the four coverage numbers from code/coverage_report.py. Performance: every numeric NFR measured with realistic volume — target, method, sample, p50, p95, verdict, action — with at least one target re-measured after a fix and both numbers shown. Security: the four areas worked, evidence in every row, every finding fixed or accepted in writing, dependency findings carrying a verified advisory ID and a reachability judgment.

5. docs/defect-log.md — from code/cut-kit.md. At least six defects, each with all seven fields, severity and priority assigned separately with a stated reason, an hour estimate on everything you intend to fix, and — for every defect marked fixed — the named regression test proved red against the pre-fix code.

6. docs/change-requests/ — one file per change request since the Week-4 baseline, named CR-0NN-<short-slug>.md, each carrying the three-line impact analysis (cost, blast radius, risk in both directions), a disposition, a decider, and a date. At least one REJECT and one DEFER with a named destination and a size. Every one also gets a row in the change table at the top of CHANGELOG.md. Anything cut is marked Deferred in docs/requirements.md with a date and CR reference — nothing deleted.

7. CHANGELOG.md — Keep a Changelog format, covering this candidate, entries traced to requirement and defect IDs, telling the reader what they must do differently, with a Known issues section listing what ships broken, with severity and a workaround.

8. An annotated release-candidate tag (for example v0.9.0-rc.1) pushed to your repository, pointing at a commit that clones clean into a fresh directory, sets up with ./script/setup, and passes ./script/test using only the documented steps.

Normal-tier rubric (out of 100)

If Your Project Does Not Look Like a Web Application

This course is stack-agnostic and so is this milestone. The three levels are required; what they contain depends on what you built. Find your row and write the mapping into your test plan’s Levels section.

Two things do not change. Every Must requirement gets an acceptance test — and if your acceptance level is manual, the “test case” is a written script with numbered steps and expected results that someone else could execute, not a memory of you clicking around. Every defect gets logged, whether a machine found it or your own hands did.

Medium Tier (+up to 25% extra credit)

Depth that raises the floor rather than adding features. Two done well beat five done thinly — and remember what the budget says: these come out of the same ten hours.

• A coverage gate in CI. Wire code/coverage_report.py into script/test so an unverified Must requirement fails the build. Report line coverage underneath verified-requirement coverage, never instead of it.
• Property-based tests. Two or more invariants that must hold across generated inputs (Hypothesis, fast-check, jqwik, or your ecosystem’s QuickCheck descendant), each stated in prose in the test plan before it is stated in code.
• A chaos rep in the suite. An automated test that runs your system with a dependency stubbed to fail and asserts the degraded behavior, so your fallback cannot silently rot.
• A kill switch. A configuration flag that disables an integration without a redeploy, exercised in a test and documented for the runbook you write in Week 7.
• An accessibility pass as tests. Run an automated checker over your main flows, log every finding as a defect with a severity, and add one manual check — keyboard-only navigation through your primary task. Automated tools catch only a fraction of real barriers; see https://www.w3.org/WAI/ for what the criteria mean.
• A contract test for a third-party dependency. Capture a real response, commit it as a fixture, and test that your parser survives that shape plus a 429, a timeout, and malformed content.
• Cost and rate telemetry for a metered dependency: calls, tokens or units, and estimated cost per request, logged and summarized. Report what a heavy day would actually cost.

Hard Tier (+up to 25% additional extra credit)

A two-to-three page Cut Memo, in docs/cut-memo.md, addressed to the engineer who will inherit this project.

An assistant can generate test cases all day and can draft an impact analysis in seconds. It cannot do this, because this requires owning the consequences of decisions made with incomplete information and a deadline that will not move. Five sections, each answering a question you cannot look up:

1. Defend your exit criteria, and your misses. Why these numbers? Why is zero-open-S2 the right bar for your project and not zero-open-S3? Name the user impact behind each threshold. Then state plainly which you met and which you did not.

2. Defend what ships broken. For every open defect in this candidate: severity, priority, workaround, and — this is the part — the user who is affected and why you decided their inconvenience is acceptable this term. Write it as though they will read it, because your handoff guide in Week 7 means someone will.

3. Defend the cut. Take the requirement you dropped and argue the case against your own decision as strongly as you can — the strongest version, not a strawman. Then answer it. If you cannot state the opposing case well, you have not decided; you have rationalized.

4. Name the inherited requirement. Every adapted brief leaves one. Which requirement in your specification was still describing somebody else’s project in Week 6? How did it survive Week 2’s review? What would have caught it earlier — and what does that tell you about how you read inherited material?

5. Predict Week 8. Given your measured velocity from docs/hours-log.csv, name the one thing most likely to be unfinished at delivery, and the trigger that would tell you — early enough to matter — that it is happening.

Graded on judgment, specificity, and candor. A memo that says everything went well earns nothing. A memo that names a real weakness and defends a real tradeoff earns full marks even when the tradeoff is uncomfortable.

Submission

Everything lives in your repository. What a grader expects to find by the end of this week:

your-capstone/
├── README.md
├── CHANGELOG.md                  <- change table on top, Known issues at the bottom
├── LICENSE
├── .env.example                  <- and .env is NOT tracked
├── docs/
│   ├── requirements.md           <- deferrals marked, not deleted
│   ├── architecture.md           <- seam map; third-party failure modes
│   ├── test-plan.md              <- NEW this week
│   ├── traceability.csv          <- NEW this week
│   ├── test-results.md           <- NEW: run + performance + security
│   ├── defect-log.md             <- NEW this week
│   ├── change-requests/          <- one file per CR
│   ├── cut-memo.md               <- Hard tier only
│   ├── ai-usage.md               <- updated
│   ├── hours-log.csv             <- updated
│   └── adr/
├── script/                       <- setup, test, smoke
├── src/
├── tests/                        <- unit / integration / acceptance / data
└── .github/workflows/            <- or your CI equivalent; the suite runs here

Verify before you submit:

git status --porcelain            # empty
git tag -n                        # the annotated tag and its message
git rev-parse v0.9.0-rc.1         # the SHA you are submitting
rm -rf /tmp/rc-check && git clone . /tmp/rc-check
(cd /tmp/rc-check && ./script/setup && ./script/test)

Suggested Pacing

Twenty hours, and the order matters more than the total.

If Wednesday’s acceptance pass finds nothing, Wednesday went wrong — not the software. Go back to the hostile data and the boundaries.

Hints

• Write the exit criteria on Monday. Criteria written on Friday are always, mysteriously, criteria the current build happens to meet.
• Start with the requirement that scares you, not the one that is easiest to test. That is where the defects are, and Monday is much cheaper than Thursday.
• Expect your first coverage number to be low. Thirty to sixty percent is normal. It is not a judgment of you; it is a measurement you have never taken before.
• Log before you fix. Always. A defect log of three entries reads as no testing at all.
• Write the hours number before you write the queue. Priority set without a budget is just enthusiasm with a label on it.
• Try the authorization hole yourself. Log in as user B and request user A’s object by ID, directly, not through the interface. This is the most common serious defect in student capstones.
• When you defer, give it a destination and a size. “Later” is not a disposition. “Backlog B-07, ~14 h, needs acceptance criteria first” is.
• Run the clean-clone gate early in the week, not the night you tag. It is designed to fail the first time. That failure is the useful part, and it is Chapter 7’s to-do list.
• Keep the assistant away from your implementation when generating tests. Give it the requirement text only, or it will write tests that lovingly preserve your bugs.
• Templates: code/test-plan-lite.md, code/cut-kit.md, code/traceability-matrix.csv, code/coverage_report.py. Blank forms live in the Document Kit, Appendix C; your brief’s what-to-cut-first list is in Appendix B; the vocabulary is in Appendix E.

What Mastery Looks Like

A master’s Week-6 package has a particular smell, and a reviewer picks it up in about ninety seconds.

The exit criteria are numbers, and at least one of them was not met — and that fact sits beside the number rather than having been erased. The traceability matrix contains gaps, clearly marked, because the author would rather show you an unverified requirement than pretend it does not exist. The defect log has eleven entries, not four, and three are marked “will not fix” with a reason a user could read without being insulted. Severity and priority disagree on at least two lines, which is how you know the author understood the difference. Every fixed defect names a regression test, and those tests have real names like test_expiry_window_includes_today rather than test_bug_fix_2.

One file in docs/change-requests/ is a refusal — written politely, with a number in it, addressed to somebody whose opinion the author values. It takes six minutes to write and it is the most professionally impressive document a student produces all term. Another is the honest withdrawal of a requirement inherited from a Fast-Start brief that never became this project. And CHANGELOG.md has a Known issues section that is not empty. That section is the tell: anyone can list what they finished, but a candidate who tells you unprompted exactly what is still broken and how to work around it is a candidate who can be trusted about the things they say did work.

Somewhere in docs/test-results.md there is a sentence like: “Verified-requirement coverage is 84%. FR-16 was withdrawn through CR-021; NFR-07 remains unverified and is listed in Known issues with the test a successor should write first.”

That sentence is worth more than a green badge, because a green badge is a claim and that sentence is a confession with a plan attached.

Coach’s Note — Somewhere around Thursday you will be tempted to soften an exit criterion so the build passes it. It will feel like a small edit. It is not. That edit is the exact moment a professional becomes a person performing professionalism, and it is the habit that follows you into a job where the number protects a real user. Miss the criterion. Write down that you missed it. Ship the honest list. It costs you three rubric points and buys you a career.

When You’re Done

• docs/test-plan.md complete: scope, out of scope with reasons, levels, environment, numeric exit criteria, signature
• Exit criteria written first, then evaluated; every miss annotated, none edited away
• docs/traceability.csv covers every requirement; every Must has a test case; gaps are visible
• Suite runs from ./script/test and is green in CI on a clean checkout
• Boundary and negative cases exist for every numeric, date, or length requirement
• At least two integration tests cross Tier 1 seams; the seven seam failure modes probed on purpose
• One acceptance test per Must requirement, each naming its acceptance criterion
• Test data reproducible from a committed seed; no real personal data
• docs/defect-log.md has six or more defects with all seven fields, plus estimates
• Severity, priority, and hours assigned separately, each with a reason; the fix queue fits the real budget
• Every fixed defect names its regression test, proved red against the pre-fix code
• docs/test-results.md records the run, the performance measurements with method and p50/p95, and the security pass with every finding dispositioned
• Every advisory ID verified in a real advisory database; reachability judged; any leaked credential rotated at the provider first
• One file per change request in docs/change-requests/, each with a three-line analysis and a disposition; at least one REJECT and one DEFER
• Every CR has a row in the change table at the top of CHANGELOG.md; cuts marked Deferred in the requirements, not deleted
• CHANGELOG.md traced to IDs, with a Known issues section that is not empty
• Clean-clone gate passes; annotated tag pushed
• docs/ai-usage.md and docs/hours-log.csv current through Week 6
• Weekly quiz taken; everything committed and pushed

A theological footnote. “Who can discern his errors? Declare me innocent from hidden faults” (Psalm 19:12, ESV). The psalmist’s problem is not that he refuses to face his faults. It is that the worst of them are hidden — invisible from the inside, precisely because they are his. That is exactly the condition of a developer testing his own code, and it is why sincerity is not a method: you cannot find, by trying harder, the fault you cannot see. You need something outside yourself — a written requirement you cannot renegotiate at midnight, a matrix that does not care how the week went, a machine that runs your claims and reports without flattery. And then, this week, a second discipline that Scripture is just as blunt about. Jesus asks whether a builder “does not first sit down and count the cost, whether he has enough to complete it” (Luke 14:28, ESV), and the unfinished tower in that passage is not a monument to low ambition but to uncounted cost. Counting is the ten-hour number. It is also why “Let what you say be simply ‘Yes’ or ‘No’” (Matthew 5:37, ESV) belongs in a chapter on change control — the offense named there is not refusal but the comfortable ambiguity that spares you a hard conversation at somebody else’s expense. Your defect log is not a monument to failure and your change-request folder is not a record of what you failed to build. They are the map of what to fix next and the evidence that you counted before you promised. Search the work. Count the cost. Say the true word. Then finish.