Milestone 5 — Technology Evaluation & Architecture Decision Records

“The simple believes everything, but the prudent gives thought to his steps.” — Proverbs 14:15 (ESV)

Chapter: Chapter 5 — Choosing a Stack That Holds Together Week: 5 of 16 — the front edge of the design phase. The hat this week is architect. Due: End of Week 5, committed and pushed. The Week 8 design review will open these documents in front of you. Submit: Committed to your capstone repository. Post the repository link and the commit hash in the Canvas dropbox. Where it lands in the final package: docs/tech-evaluation.md (with docs/tech-evaluation.csv), the numbered records in docs/adr/, the spike write-ups in docs/spikes/, the LICENSE file at the repository root, and appended rows in docs/hours-log.csv and docs/ai-usage.md. These feed the architecture-decision-record line and the licensing line of the Week-16 rubric, and they are the raw material for the technical specification you write next week. See Appendix C for the full grading contract. AI policy: Open, with one hard condition. An assistant may generate options, draft tradeoff tables, name criteria you forgot, and argue against your choice. It may not be the source of any factual claim that reaches your documents. Every version, price, limit, license, and compatibility claim carries a vendor source and a date you checked it. Record substantive AI use in docs/ai-usage.md, including the claims it got wrong.

The Setup

You have a specification. Four weeks in, you can say what your system does, who it does it for, and how anyone would verify it. What you do not yet have is a shape.

This week a real client would be asking a question you cannot dodge: what are you building it in, and why that? Not “what’s popular.” Not “what did you use last semester.” Why that, against these requirements, with your hours, on hardware and hosting you actually have access to — and what does it cost you when you are wrong?

The honest answer is not a preference. It is a document. Three or more real decisions, evaluated against criteria that trace back to numbered requirements, with weights you committed before you saw the scores. A written record of the seams between your choices and a small experiment run against the ones you have never crossed. A count of how many new things you are asking yourself to learn at once, and a decision about which single one earns your learning budget. A cost sheet you could show a client. A license inventory that tells you what you are legally allowed to ship.

And on top of all of it, the artifact this milestone is really named for: architecture decision records — short, immutable, numbered documents that say why, in a form that survives your memory. In Week 14 you hand this repository to somebody who was not in the room this week. The ADRs are the only thing you will leave them that explains any of it.

Learning Targets

You will demonstrate that you can:

• Derive architectural drivers from a requirements specification, and distinguish them from preferences.
• Build a weighted evaluation matrix whose criteria trace to requirement identifiers and whose every score carries evidence.
• Identify integration risk at the seams between components, not just within them.
• De-risk an unknown with a time-boxed spike that has a failure criterion written in advance.
• Assess your own capacity honestly — count novelty load, and demote something when the count is too high.
• Evaluate a vendor, including an AI provider, on cost, limits, data handling, deprecation, and failure behaviour.
• Determine what you may legally ship, using SPDX identifiers and the projects’ own license files.
• Record a decision so that a stranger six months from now can reconstruct your reasoning and argue with it.
• Verify time-varying facts rather than repeating them — from documentation, from a classmate, or from an assistant.

The weekly status block — paste this into your submission comment

Every milestone submission carries these five lines. It takes two minutes, it is the same five lines every week, and it is the difference between an instructor who can help you in Week 6 and one who finds out in Week 14. Copy it into the Canvas submission comment when you submit — not into a document.

Shipped:     what actually works this week, in one line
Cut:         what I dropped or deferred, and where it went (backlog / change request / gone)
Hours:       planned N / actual N        (from docs/hours-log.csv)
Blocked on:  the one thing in my way, or "nothing"
Ship confidence: green | amber | red — if amber or red, the one thing that would move it

Say red when it is red. A red in Week 6 is a conversation and a scope cut. A red discovered in Week 15 is a grade. Nobody has ever been penalised in this course for an honest amber; the only expensive answer is the green that was not true.

Normal Tier

The bar. This is what a competent senior produces in about fifteen hours.

Required deliverables

1. docs/tech-evaluation.md containing, in this order:
   • your architectural drivers table (four to eight rows, each tied to a requirement identifier or a stated constraint);
   • a weighted evaluation of at least three distinct decisions (for example: data store, hosting, and one of framework / auth / the AI provider), each with at least two real options you genuinely considered;
   • the seam inventory — six to ten rows — with a risk rating and a spike id for every High seam;
   • your novelty load count and what you did about it;
   • the cost sheet at student scale and the free-tier watch list;
   • the license inventory with SPDX identifiers and a ship/no-ship call;
   • the verification log: every time-varying claim, its vendor source, and the date you checked it.
2. docs/tech-evaluation.csv — the machine-checkable matrix, in the format of code/tech-evaluation-matrix.csv, passing code/score-stack.py with exit status 0. Its history must show a commit with weights and no scores before a commit with scores.
3. At least four architecture decision records in docs/adr/, numbered from 0001, in the format of code/adr-template.md. One decision per record. Every record dated, statused, attributed, and tied to requirement identifiers. Every record has negative consequences and a revisit trigger with a number in it.
4. At least two spike plans in docs/spikes/, from code/spike-plan-template.md, with at least one actually run inside its time box and its Result and Decision sections filled in.
5. A LICENSE file at the repository root, chosen deliberately, with the choice justified in one of your ADRs.
6. Updated logs — docs/hours-log.csv for this week’s hours, and docs/ai-usage.md including your Rep 12 verification hit rate.

What “at least two real options” means

A real option is one you could have chosen. Compare:

A matrix with one serious candidate and two decoys is not an evaluation. It is a justification, and a design reviewer will name it in about forty seconds.

The bad version of this document, and the good one

Most submissions that lose points do not look empty. They look full — and say nothing. Here is a real shape of the failure, taken from the middle of a technology evaluation:

## Data store

We evaluated several options and chose PostgreSQL because it is a robust,
industry-standard relational database with excellent community support and
proven scalability. Alternatives such as MongoDB and MySQL were considered
but PostgreSQL was determined to be the best fit for our needs.

Four sentences, zero information. No requirement is named, so nobody can check the fit. “Proven scalability” cannot discriminate between the three options at student scale, so it is decoration. “Alternatives were considered” is a claim with no evidence behind it. And the passive voice — was determined — hides the only thing a reader wants: who decided, on what basis, and what it cost them.

The same decision, written so a stranger can act on it:

## Data store

Drivers: FR-011 (three-way join on the main screen), NFR-P-02 (500 ms at
1,000 records), NFR-M-01 (restore documented in under 15 minutes).

Scored: SQLite 4.70, PostgreSQL 3.90, MongoDB 2.10 (docs/tech-evaluation.csv,
weights committed 2026-02-09 before scoring).

Chose PostgreSQL despite the lower score. SQLite loses on a seam the matrix
could not see: the host in ADR 0003 runs two web workers, and spike SP-01
measured write contention under exactly that configuration in 40 minutes.
Cost of the override: a running service in local development (2 h, Week 6)
and a database in CI (2 h, Week 9). Recorded in ADR 0002.

Same decision. One of them a grader can argue with, act on, and inherit. Write that one.

Normal-tier rubric (out of 100)

Medium Tier (+ up to 25% extra credit)

Depth where it will actually pay you back. Pick what fits your project:

• A fifth and sixth decision evaluated — but only if they are genuinely consequential. Evaluating your linter is padding; evaluating your background-job strategy is not.
• A second spike run, on a different seam, with its result changing something in the matrix or an ADR. Show the change.
• A measured score. Replace one guessed score with a number you obtained — a benchmark on the actual query in FR-*, a cold-start time on the candidate host, a p95 latency against the real API.
• A superseding ADR. If a spike overturned an earlier choice, write ADR 000N marking the earlier one Superseded by, rather than editing the original. Demonstrating the immutability discipline is worth more than getting it right the first time.
• A reversibility rating on every decision — how many hours it would cost to undo in Week 12 — and a sentence about how that rating broke a tie.
• A dependency-refresh plan: how you will know, before Week 14, that a version or a free tier moved under you.

Hard Tier (+ up to 25% additional extra credit)

Required: a two-to-three page memo no assistant can write for you. Call it docs/decision-memo.md, addressed to the person who will inherit this project.

Cover four things, in your own voice, with your own evidence:

1. The decision you are most likely to regret. Not a hedge — name one, and argue the case against yourself. What is the strongest version of the argument that you chose wrong? Why are you proceeding anyway? What would you have to see, and by when, to reverse it?
2. Where you overrode the matrix, or why you did not. If the highest-scoring option lost, defend the override with the evidence that the matrix could not see — usually a seam, a constraint, or a capacity limit. If you took the top score every time, say why you trust the weights that much, and name the weight you are least sure of.
3. Your résumé motive, in the open. From Rep 2. What did you want to use, what did it actually cost or save you, and how did you keep the want from bending the weights? A student who can write this paragraph honestly is doing something most working engineers avoid for a whole career.
4. The integrity call on your tools. Where did you use an assistant this week, what did it get wrong, and how do you know? Give your verification hit rate as a fraction and name the category of claim you now distrust most.

This memo is graded on judgment, self-awareness, and honesty — three things a model cannot supply, because all three require owning a decision, and ownership is the one thing a tool cannot do.

Submission

Commit and push. The tree should look roughly like this:

your-capstone/
├── LICENSE
├── README.md
└── docs/
    ├── requirements.md          # from Weeks 3-4
    ├── tech-evaluation.md       # NEW - drivers, matrix, seams, cost, licenses, verification log
    ├── tech-evaluation.csv      # NEW - passes score-stack.py
    ├── hours-log.csv            # updated
    ├── ai-usage.md              # updated, including your verification hit rate
    ├── decision-memo.md         # Hard tier only
    ├── adr/
    │   ├── 0001-choose-the-primary-language-and-framework.md
    │   ├── 0002-choose-the-data-store.md
    │   ├── 0003-choose-the-hosting-target.md
    │   └── 0004-buy-or-build-the-<capability>.md
    └── spikes/
        ├── SP-01-<slug>.md      # run, with Result and Decision filled in
        └── SP-02-<slug>.md

Then post the repository link and this week’s commit hash to the Canvas dropbox. Also take the weekly quiz — it is one of the sixteen that together carry 15% of your grade, and it exists to tell you now whether you are on pace.

Hints

• Do Rep 4 in a separate sitting from Rep 5. Commit the weights with no scores. That commit is your evidence, in Week 8, that you did not tune the weights to fit your favourite.
• Run the checker early and often. code/score-stack.py will not let you leave an evidence cell empty. Fill it as you go instead of at midnight.
• Write the ADR while the alternatives are still in your head. The Context section is impossible to reconstruct a week later, and a thin Context is the single most common reason an ADR is worthless six months on.
• Write the failure criterion before you run the spike. Then honour the timer. “I was almost there” is exactly the state in which a spike becomes the project.
• Verify licenses from the project’s own LICENSE file, not from a search summary and not from memory. Several well-known products have changed terms in recent years; a stale memory here is expensive.
• If your novelty load is 3 or more, fix it this week. Not “carefully.” A named substitution, or a cut requirement. The rubric awards the demotion, not the bravery.
• One interface per external dependency. You will thank yourself in Week 12, when one of them has to be swapped.
• Do not put a secret in the repository. Not in a config file, not in an ADR, not “temporarily.” Use a placeholder and document where the real value comes from. Appendix A covers the mechanics.

What Mastery Looks Like

A master’s technology evaluation reads like it was written by someone who expected to be questioned.

The criteria have requirement identifiers next to them, so nobody has to take the author’s word about what matters. The weights were committed before the scores, and the repository history proves it. At least one score is a measurement rather than an opinion. The seam inventory contains a row the author clearly did not enjoy writing. And somewhere in the ADRs, there is a paragraph that says, in effect: the matrix said X, I chose Y, and here is the evidence the matrix could not see.

The consequences sections have teeth. They name what got harder, in hours, with a mitigation and a date. Nothing in the whole package is a slogan. Every time-varying claim has a URL and a date beside it, including the ones an assistant supplied — especially those.

And the whole thing could be read by a stranger who then correctly predicts what the system looks like in Week 12. That is the test. Not “is it thorough.” Could somebody else act on it.

Coach’s Note — The temptation this week is to make the matrix agree with the choice you already made. It is a quiet temptation, it takes about four keystrokes, and nobody in the world will ever catch you. Which is exactly why it is worth resisting. The habit you are actually building this week is not spreadsheet technique — it is the willingness to let evidence beat you in private. Every engineer you would want to work for has that habit, and every one of them built it on a decision that did not matter much. This is one of those.

When You’re Done

• Four to eight architectural drivers, each tied to a requirement identifier
• Three or more decisions, each with two or more real options
• Weights committed in a separate, earlier commit than the scores
• Every score has evidence; score-stack.py exits 0
• Sensitivity pass run; result recorded
• Seam inventory written; every High seam has a spike plan
• At least one spike actually run, inside its time box, with Result and Decision
• Novelty load counted; demotion made or token defended
• Four ADRs, numbered and dated, each with negative consequences and a numeric revisit trigger
• Cost sheet and free-tier watch list, every row with a fallback
• License inventory with SPDX identifiers; LICENSE file chosen and committed
• Verification log complete; no undated claim anywhere in the package
• docs/hours-log.csv and docs/ai-usage.md updated
• Repository link and commit hash submitted; weekly quiz taken

A theological footnote. “but test everything; hold fast what is good” (1 Thessalonians 5:21, ESV). Notice the order, and notice that both halves are commands. Testing without holding fast is just restlessness — the engineer who evaluates forever and commits to nothing has not been careful, only afraid. Holding fast without testing is credulity wearing the costume of loyalty. The verse asks for both: do the work of sorting, then commit firmly to what survived it. That is what this milestone is. You test the vendor’s claim, the assistant’s claim, and — hardest of all — your own preference, and then you write down what you decided and why, with your name on it. There is a kind of confidence available only on the far side of examination, and it is the only kind worth having. It is also a gift you are giving away: the stranger who inherits this repository in Week 14 gets to trust your choices because you tested them, and gets to argue with them because you wrote down why. Test everything. Hold fast to what is good. Then hand it on.