Project 8 — The Capstone SoC Investigation

“A false balance is an abomination to the LORD, but a just weight is his delight.” — Proverbs 11:1 (ESV)

Chapter: 8 — Measure, Compare, Decide Weight: 20% of the course grade — the final. (The recorded briefing in the exam document is a separate 8%.) Due: End of Week 8 Submit: One submission — a public repository link or a single archive — containing report.docx, measurements.xlsx, soc-architecture-review.docx, placement-decision.docx, and ai-usage.txt. Nothing else is required and no other document is graded. Workbench setup is in Appendix A; the report and measurement-log templates are in Appendix C; the grading contract is Appendix D. Allowed tools: Everything in Appendix A — a C++17 compiler, Python 3, this chapter’s code/ harness and analysis tools, perf / simpleperf / Instruments where you have them, and any vendor documentation you can cite. An Android device (Workbench D) is optional everywhere and required nowhere. Every Normal-tier requirement below is completable on Workbench B. AI policy — the strictest week: Use it to explain, never to source. You may use a language model to explain a mechanism, a compiler flag, a statistic, or a paragraph of vendor documentation you are struggling with. You may not use one to supply a hardware figure, a benchmark number, a citation, or a date. Every external figure in this submission carries a primary-source citation. An uncited figure scores zero on its rubric line. A fabricated figure — a number, a source, or a quotation that does not exist — fails the integrity line and the project, regardless of the rest. ai-usage.txt is mandatory and is graded.

The Setup

You are the architect on a small hardware-evaluation team. In four weeks your organization has to commit — to a part, to a placement, or to a design — and the decision will be expensive to reverse. Two documents have already crossed your desk. One is a vendor deck with a bar chart and a single number on it. The other is a competitor’s rebuttal with a different bar chart and a different single number. Both are, as far as anyone can tell, technically true.

Your job is the third document: the one somebody can actually decide from. It will contain fewer numbers than either of the others and every one of them will carry its conditions. It will say what you measured, how, under what state, with what dispersion. It will say plainly what you could not measure and what it would take to find out. And it will end in a recommendation you are willing to defend to a room of people who would rather it were simpler.

That is the whole assignment, and it is the actual job. The eight weeks of this course were the preparation: block diagrams (Chapter 1), core classes (Chapter 2), the memory hierarchy that decides most results (Chapter 3), the accelerator fleet and the placement question (Chapter 4), the energy and thermal budget that governs everything (Chapter 5), the integration and radios that break systems at the joints (Chapter 6), the security mechanisms that cost performance (Chapter 7), and this week’s discipline for saying any of it out loud honestly.

Setup (the starter)

Everything you need ships with this chapter:

• code/bench_harness.cpp — the measurement instrument. Warm-up, N repetitions, median and interquartile range, an optimizer barrier, CSV output. Extend it with your own kernels; the table at the bottom of main() is where you add them.
• code/counters.py — IPC, MPKI, branch miss rate, and a documented first-pass diagnosis, from perf stat / simpleperf stat output.
• code/sample-perf-stat.txt — synthetic sample counter output, for anyone without counter access.
• code/soc-compare.csv — synthetic comparison data over four fictional parts, SoC-A..SoC-D.
• code/compare.py — derived bandwidth, sustained/peak ratio, performance-per-watt, and the five rankings.

On the synthetic data, and this is graded. soc-compare.csv and sample-perf-stat.txt are teaching data, declared as such on their first lines, and the parts in the comparison file are fictional. You may use them freely — to demonstrate a method, to practise an analysis, to make a point about metric selection. You may not present anything derived from them as evidence about real silicon, and if you use them at all your report must say, in the sentence where you use them, that they are synthetic and what that limits. Distinguishing measured data from modeled data is one of the things this project exists to grade.

Learning Targets

By completing this project you will demonstrate that you can:

• Read a real SoC’s block diagram and specification table and explain why an architect made those choices, not merely what the parts are.
• Take a defensible measurement — warmed up, repeated, reported as a median with dispersion under fully stated conditions — and know what it does and does not support.
• Diagnose a bottleneck from evidence rather than from utilization, and say which of The Four Questions each piece of evidence bears on.
• Build and defend a placement table: workload → processor → why → what it costs.
• Compare or design under stated constraints without letting a chosen metric make the argument for you.
• State the boundary of your own evidence — what you could not measure, and precisely what instrumentation or access would be required to.
• Cite every external figure to a primary source, and disclose your AI use honestly.

Choose Your Track

Pick exactly one. All three are graded against the same rubric; they differ in what you point the method at. Choose the one whose question you actually want answered — the work is long enough that interest matters.

Track 1 — Comparison

Two real, currently-shipping SoCs from different vendors, compared on architecture, for one named workload that you state up front.

This track has a hard rule, and it is the rule the entire book is built on: you are not ranking vendors. You are comparing architectural choices and their consequences for your named workload. “Part X is better than part Y” is not an acceptable conclusion and will lose the integrity line. “For a sustained on-device inference workload under a phone-class thermal envelope, X’s wider memory interface matters more than Y’s higher peak clock, because ⟨evidence⟩ — and here is the condition under which that reverses” is exactly the acceptable form. Normalize what you can (§8.9’s table), and name every confound you cannot normalize, especially process node.

Track 2 — Workload study

One workload — an on-device model, a game frame, a camera pipeline, a video encode, a continuous sensing task — profiled and analyzed across CPU, GPU and NPU, with a defended placement.

You will not have access to every processor on every workbench, and that is expected. What is required is that you measure what you can measure on your own hardware, reason structurally about the rest, and state which is which on every line. A CPU-only control measurement plus a rigorous structural argument about the accelerator path beats a hand-wave about all three. If you do have accelerator access, get the per-operator assignment and check for the operator fallback Chapter 4 warned you about.

Track 3 — Design

A hypothetical SoC for a named product — an XR headset, a medical wearable, a field-service tablet, a body-worn camera, a portable ultrasound — against a stated power and thermal envelope, with every block defended.

Your measurements here support the reasoning, not the part: measure a representative kernel of your product’s workload on hardware you have, and use those measurements to argue about what the design must provide. State the envelope as a number and hold yourself to it. A design that quietly assumes a fan, or a battery nobody would carry, is a fantasy with a block diagram attached.

Normal Tier

Goal: one investigation that a working engineer could act on, with every number traceable to a measurement you took or a source you cited.

Required deliverables

1. soc-architecture-review.docx — finished, not restarted. Carry forward the document you began in Week 1’s SoC Teardown and complete it: the block inventory with each block’s purpose and what it contends for; the CPU cluster configuration; the memory configuration including the derived bandwidth (MT/s × bus bytes); the accelerator inventory; and the storage and radio picture at the level Chapters 3 and 6 taught. Track 3 authors this for their designed part instead, at the same depth. Mark every figure with its source.

2. A stated question. One paragraph at the top of report.docx: the exact question this investigation answers, the workload or product it answers it for, the constraint it answers it under, and — explicitly — what is out of scope. A capstone without a scope statement drifts into a survey, and a survey is not a decision.

3. Real measurements you took yourself. At least three distinct kernels or scenarios, each run with warm-up and at least 21 repetitions, using code/bench_harness.cpp or an instrument of your own that meets the same standard. At least one of the three must be a kernel you wrote or adapted for your subject rather than a stock kernel from the harness.

4. measurements.xlsx per Appendix C. Every run: what was measured, on what machine, at what optimization level, with how many warm-up and timed repetitions, on mains or battery, in what thermal condition, with the median and a dispersion figure. No bare single numbers appear anywhere in this submission, including in report.docx.

5. Counter analysis. For each measured kernel, report IPC and at least one miss rate, using code/counters.py on your own counter capture where you have counter access, or on code/sample-perf-stat.txt with the limitation clearly stated where you do not. Note any multiplexing you observed.

6. placement-decision.docx — the placement table. At least four workloads relevant to your subject, one row each: workload → chosen processor → why → what it costs. The “what it costs” column is the graded one; a row that names only a benefit is incomplete. At least one row must record a placement you rejected and the reason.

7. A bottleneck analysis under The Four Questions. For your primary workload, answer all four explicitly and separately, with a heading each: performance, energy, thermals, placement. Say which of your evidence bears on each, and where a conclusion rests on structure rather than measurement, say so in that sentence.

8. Peak and sustained handled honestly. Any headline number you report carries its measurement window and thermal condition. If you report a peak figure, report a sustained one beside it or state why you could not obtain one.

9. Primary-source citations. Every external figure — every clock, width, capacity, rate, score, or date that you did not measure — carries a citation to a vendor specification page, an architecture reference manual, a standards document, or a named published paper, with the retrieval date. “Not published by the vendor” is a correct and creditable answer when it is true.

10. “What I could not measure, and what I would need.” A required section of report.docx, and one of the most heavily weighted. Be specific. Not “I did not have a power meter,” but “I could not attribute power to the GPU; to do so I would need ⟨named instrumentation or platform counter⟩, and with it I would run ⟨named experiment⟩ to answer ⟨named question⟩.” Every honest investigation has this section. A submission without one is claiming to have measured everything, which is never true.

11. ai-usage.txt. Per Appendix D: which models you used, for exactly what, what you accepted, and what you rejected or had to correct. One honest paragraph beats a page of hedging.

Normal-tier rubric (out of 100)

The integrity line (pass/fail, applied over the rubric). A fabricated figure, a citation that does not resolve, a quotation nobody wrote, a measurement you did not take, or synthetic data presented as real, fails this project outright. This is not a deduction. It is the one rule in this course with no partial credit, because a measurement report is an instrument, and a crooked instrument is worse than no instrument at all.

Medium Tier (+up to 25% extra credit)

Choose one. Do it properly rather than doing both thinly.

M1. The mitigation cost, measured

Chapter 7 promised that mitigating a side channel costs performance, and that Chapter 8 would measure it. Collect on that promise.

Pick a real, named mitigation or hardening measure available on your workbench — a constant-time comparison replacing a data-dependent one, a bounds-checked build versus an unchecked one, a compiler hardening flag, a memory-safety option your toolchain offers. Measure the cost properly: same harness, same conditions, 31+ repetitions, median and IQR, with the confound list. Then answer the question a tool cannot: for your subject product and its threat model, would you enable it? Name the threat it addresses, the cost you measured, the residual risk it does not cover, and the condition under which your answer flips. A “yes, always” or “no, too slow” that is not grounded in your own measured number scores nothing.

M2. The metric-sensitivity analysis

Take your own primary conclusion and try to break it. Re-derive it under at least four different defensible metric or condition choices — different measurement window, different thermal condition, different summary statistic, different normalization, different weighting of your workloads. Present the four results in one table. Then state: which choices leave your conclusion standing, which reverse it, and — the graded sentence — which single choice most flatters your conclusion, and what your headline claim becomes when you reverse it. Do this well and you will have written most of Challenge Question 1 for the briefing.

Hard Tier (+up to 25% additional extra credit)

H1. The architecture memo (the judgment piece)

Write report.docx’s final section as a memo — one page, no more — to a decision-maker who will not read your measurements and cannot evaluate them. It must contain:

1. The recommendation, in one sentence, at the top, with its metric named before its conclusion.
2. The three pieces of evidence it rests on, each one line, each traceable to a numbered result elsewhere in your report.
3. The strongest argument against your recommendation, stated as well as its advocate would state it — not a straw version — and your answer to it.
4. The tradeoff you are accepting, named plainly. Every real architecture decision gives something up. If you cannot name what yours gives up, you have not made a decision; you have made a wish.
5. The condition under which you would change your mind, stated as something observable: a measurement, a threshold, a change in the workload or the envelope.
6. What you would fund next — the one experiment or instrumentation purchase that would most reduce the remaining uncertainty, and what question it would settle.

This is the heart of the Hard tier and the part no tool does for you. A model can summarize your measurements; it cannot decide, under uncertainty, with something at stake, what your organization should do — because it has nothing at stake and cannot be wrong in the way that matters. A memo that hedges every direction, or that recommends without naming what it gives up, fails the tier no matter how good the measurements behind it are.

Submission

Submit one link or one archive. It must contain exactly these five documents, at the top level, with these exact names:

1. report.docx — the investigation. Stated question and scope; method; results; counter analysis; the Four Questions section; peak-versus-sustained; “what I could not measure and what I would need”; conclusion; and (Hard) the memo.
2. measurements.xlsx — the measurement log, in Appendix C’s tabular form.
3. soc-architecture-review.docx — the Week 1 document, carried forward and finished.
4. placement-decision.docx — the placement table and its defense.
5. ai-usage.txt — the honest disclosure.

Put your raw harness CSV output and any counter captures inline in measurements.xlsx, or in a folder beside these five files. Do not invent a sixth deliverable document; if you need another section, it is a section of report.docx.

Open report.docx with this header block, filled in:

# Capstone SoC Investigation — <your title>

**Track:**              1 (Comparison) / 2 (Workload study) / 3 (Design)
**Question:**           (one sentence — the thing this answers)
**Subject:**            (the part(s), the workload, or the product)
**Constraint:**         (the envelope, deadline, or budget you held yourself to)
**Out of scope:**       (what you deliberately did not investigate)
**Headline result:**    (metric named first, then the finding, then the conditions)
**Confidence:**         (high / medium / low — and one line on why)
**Workbench:**          L / B / D, and the machine
**Tier targeted:**      Normal / Medium / Hard
**AI usage:**           (one line; full disclosure in ai-usage.txt)

Hints (Read Before You Begin)

• Pick the question before the subject. Students who start with “I’ll compare these two chips” write surveys. Students who start with “for a sustained on-device transcription workload under a phone-class envelope, does memory width or peak clock matter more?” write investigations. The second is a shorter document and a much better grade.
• Measure something small first, today. Build the harness, run one kernel, write the first row of measurements.xlsx. A capstone that begins with measurement stays honest; one that begins with a conclusion spends three weeks defending it.
• Your own kernel is where the marks are. Requirement 3 asks for at least one kernel you wrote or adapted. Take a real piece of your subject’s workload — a convolution, a colour transform, a hash, a tokenizer’s inner loop — and make it into a harness entry. It is a twenty-line change and it turns a generic report into yours.
• When a number surprises you, suspect the method. §8.4’s Coach’s Note is the single most useful reflex in this project. Multiplexed counter? Migrated core? Counted start-up? Optimizer removed the loop? Check all four before you write a sentence beginning “interestingly.”
• Never report a bare number. Median and dispersion, or do not report it. This is checked mechanically against your log.
• Write the “what I could not measure” section early, not last. Keep it open as you work and add to it every time you hit a wall. Written last, it is an apology; written throughout, it is a map of the frontier, and reviewers respect it more than anything else in the document.
• If you cannot find a primary source, say so. “The vendor does not publish the last-level cache size” is a true, useful, creditable sentence. Guessing is not, and asking a model to guess for you is the one thing that fails the project.
• Say the flattering choice out loud. Whatever your conclusion, one of your methodological decisions helped it. Find it and disclose it before your reader does. It costs you nothing and it is the difference between a report and an advertisement.

What Mastery Looks Like (Beyond the Rubric)

A great capstone reads like a document an engineering director would forward without editing. The question is narrow and stated in the first paragraph. The measurements are few, well chosen, and reported so precisely that a stranger could repeat them. The counter analysis names a mechanism instead of a symptom, and the mechanism connects to something the course taught, by number. The placement table has a real cost in every row and at least one honest rejection. The Four Questions are answered separately, so nobody can mistake a performance answer for an energy answer. The limits section is specific enough to be a work order. And the conclusion is a decision — with its metric named first, its tradeoff named plainly, and the condition that would reverse it stated in a form somebody could go and check.

The tell of an excellent one is subtler: nowhere in it does the author sound like they wanted a particular answer. The evidence goes where it goes. When something contradicted the expected result, the document says so and works out what it means. That posture is not modesty. It is the only thing that makes a measurement worth reading.

Coach’s Note — Somewhere in the next four weeks a result will come out the wrong way — against the part you liked, against the placement you had already argued for, against the design you had already drawn. That moment is the capstone. Everything up to it was arithmetic. What you do in the next hour is the profession. Follow the result, say what changed, and keep the run that broke your assumption in the log where a reviewer can see it. The strongest documents in any engineering organization are the ones that visibly changed their author’s mind.

When You’re Done

1. Open measurements.xlsx and confirm every row has a median, a dispersion figure, a machine, a power state, a thermal condition, and a repetition count. Fix anything missing before you look at anything else.
2. Search report.docx for every number. For each one, ask: did I measure this, or did I cite it? If measured, it points at a row in the log. If cited, it points at a source with a retrieval date. If neither — delete it. There is no third category.
3. Re-read your conclusion and check that it names its metric before it names its finding. If it does not, rewrite the sentence.
4. Confirm your synthetic-data sentences are present wherever you used code/soc-compare.csv or code/sample-perf-stat.txt.
5. Confirm the “what I could not measure and what I would need” section names instruments and experiments, not feelings.
6. (Track 1) Reread your comparison for any sentence that ranks a vendor rather than a choice. Rewrite each one to compare the architectural decision and its consequence for your named workload.
7. Read the exam document and check that you can answer all three challenge questions from what is already in your report. If you cannot, you have four weeks, and that is what they are for.
8. Submit.

A theological footnote. Proverbs 11:1 is about a merchant’s stone: “A false balance is an abomination to the LORD, but a just weight is his delight” (ESV). The reason the language is so severe is that a crooked scale does not deceive one buyer — it corrupts the whole basis on which people who cannot check each other’s claims nevertheless manage to trade honestly. That is exactly what a measurement report is. Almost nobody who reads yours will be able to rerun it; they will trust the instrument, and the instrument is you. Which means the honesty of this document is invisible to everyone relying on it — and that is precisely why it has to be a matter of character rather than of enforcement. There is a quieter promise in the verse too: the just weight is not merely tolerated but delighted in. Ordinary competent work, done truthfully, in a market nobody is auditing, is not filler between the things that matter. For the person called to it, it is the thing that matters. Carry one set of stones.

Go finish it well. See you at the briefing.