Appendix C — The Measurement Kit

This appendix exists to stop you making an honest measurement of the wrong thing.

Fraud is rare in this field and easy to condemn. What is common is to run a real program on real hardware, record a real number with real precision, and publish a claim the number does not support. Nobody lied. The instrument worked. The arithmetic was right. And the sentence at the top of the report is still wrong, because it is a sentence about a situation the measurement was never in.

Everything below prevents that, and it is deliberately narrow. There are no confidence intervals here and no hypothesis tests — not because those are bad, but because a measurement taken on a machine that rescales its own clocks does not satisfy their assumptions, and running them anyway is theatre that makes a weak result look strong. What you get is the small set of practices a hardware measurement actually needs, stated precisely enough that two students on two machines produce numbers a third person can compare.

Where this sits. Appendix A is the toolchain and the three workbenches. Appendix B is block-diagram literacy and dataset provenance. Appendix D is the grading contract, submission mechanics, and the AI-use policy. Appendix E is the glossary. This appendix owns methodology, statistics, the sustained-load protocol, and the six templates you are graded against. Every lab in the lab arc points here; every rubric line reading “per Appendix C” means the rules below.

C.0 — The Card

C.1 — What a Microbenchmark Can and Cannot Support

Both kinds of run are legitimate. Each licenses a different kind of sentence, and writing the wrong one is the commonest analysis error in this course after mistaking utilization for performance.

Precisely wrong looks like: a working set that accidentally fits in L2; a stride that lines up with the cache’s associativity; a compiler that vectorized the loop you thought was scalar; a pointer chase timed for a fixed hop count rather than a whole lap, so it touched a twelfth of the structure you believed. Each produces a tight, repeatable, low-dispersion number, which is why it fools people. Low dispersion is evidence of a stable experiment, not of a correct one.

The professional loop — and you must say which step produced each number:

1. Application benchmark — establish that it is slow, and how slow it feels.
2. Counters — hypothesize why: IPC plus a miss rate, per Chapter 8.
3. Microbenchmark — confirm the mechanism where nothing else can be responsible.
4. Application benchmark again — confirm the fix moved a number a person can feel.

Skipping step 1 is how you make a kernel four times faster and the application zero percent faster, because the kernel was two percent of the frame. Get its share of the whole first; Chapter 2’s amdahl.py exists so you have no excuse.

Coach’s Note — Read your sentence aloud and ask what would falsify this? “The accelerator path is 3× faster” is falsified by a stopwatch. “The accelerator path is 3× faster because every operator mapped” is falsified only by a per-operator assignment dump — so if you did not get one, you may not write the second half. Most bad engineering writing is a microbenchmark result wearing an application benchmark’s grammar.

C.2 — Latency Against Throughput

Latency is how long one operation takes end to end when the next thing depends on its result. Throughput is how much work completes per unit time with many operations in flight. They are not reciprocals, and you may not derive one from the other by division. The bridge is Little’s Law:

items in flight = throughput × latency

68e9 bytes/s × 100e-9 s  =  6800 bytes in flight
6800 ÷ 64 bytes/line     ≈  106 outstanding line requests

That is Chapter 3’s memory case: a phone-class 64-bit interface at 8533 MT/s gives roughly 68 GB/s of theoretical peak, and at a DRAM latency on the order of 100 ns you must keep about a hundred cache lines in flight, permanently, to reach it. A dependent pointer chase gives you one. That is why a program can idle a 68 GB/s memory system, and why “we have bandwidth headroom” is not an answer to a latency problem.

The harness ships the pair: seq walks sequentially — high memory-level parallelism, prefetchable, so it runs at bandwidth — while chase follows a random cycle where each load’s address is the previous load’s result, so it runs at latency. Same array, same loads, same instruction count; the gap between the curves is your prefetcher and your memory-level parallelism.

Row three matters more on mobile than anywhere: a renderer hitting its budget on the median frame and missing one in twenty produces a visible stutter, and the median is silent about it.

C.3 — Warm-Up, and the Two Steady States

There are two. They arrive on different timescales and push the number in opposite directions, which is why students get contradictory results from one machine in one afternoon.

So “throw away the first run” is necessary and not sufficient: it fixes the first column and does nothing about the second. A benchmark warm microarchitecturally and cold thermally is both correct and misleading — every mechanism trained, the device at an operating point it cannot hold. Which is why every number in this course carries its thermal condition as well as its repetition count.

Choosing a warm-up count. The harness defaults to three. Test that rather than trusting it: run with --warmup 0 and a high repetition count, compare the first timed repetition against the median of the rest, and raise the warm-up until repetition one is indistinguishable from repetition ten. Anything with an interpreter, runtime or driver in the path needs more than you expect. Record the number — a warm-up count is part of the method.

Coach’s Note — Warm-up is where honest and flattering diverge quietly. There is a count that makes your favourite build look best, reachable by raising the number until the answer stops improving, and nobody will catch it. The defence is procedural: fix the warm-up count before you look at the comparison, write it in the log, use the same one on both sides.

C.4 — Why the First Run Is Always Wrong

Not “usually” — always, in one direction or the other, and often both at once.

That last row is why best-of-N is wrong on a thermally limited device. On a fixed-clock machine with nothing else running, reporting the minimum is defensible. On a phone or a fanless laptop the fastest run is the coldest run, and coldness is not a property of the silicon — it is a property of the last four minutes. Best-of-N there measures your patience between runs. Report the median; and when min sits well below it while max sits far above, that spread is the environment talking, not noise to discard (C.8.3).

C.5 — Defeating the Compiler That Optimized Your Benchmark Away

A C++ compiler may transform your program any way that preserves observable behaviour. A benchmark’s observable behaviour is characteristically nothing at all, so deleting the loop is the obviously correct thing to do. You then measure, with impressive precision, the cost of an empty loop.

The portable barrier, from Chapter 8’s bench_harness.cpp:

static volatile std::uint64_t g_sink = 0;
static inline void do_not_optimize(std::uint64_t v) { g_sink ^= v; }

A read-modify-write of a volatile object is an observable side effect the compiler may not remove, so the value handed in must genuinely be computed and everything it depended on must genuinely run. It is portable — no inline assembly, no compiler-specific attribute — which is why the harness uses it. And it costs a load and a store per call, which is why the harness calls it once per repetition, not once per iteration: the sink write sits outside each kernel’s loop. Put it inside your own hot loop and you are measuring your barrier.

The stronger barrier, from Chapter 3’s memory programs:

inline void consume(std::uint64_t v) {
#if defined(__GNUC__) || defined(__clang__)
    asm volatile("" : : "r"(v) : "memory");
#else
    g_sink = v;
#endif
}

The empty asm volatile emits no instructions, so it costs nothing. The "r"(v) constraint says the value must exist in a register there, so the loads that produced it cannot be deleted. The "memory" clobber says memory may have changed, so loads cannot be hoisted out of the loop or across repetitions — the failure a plain volatile sink does not prevent. When memory behaviour is the subject, use this one.

Three sanity checks, in order. (1) Doubling — double the input; a linear kernel should take about twice as long, and if it does not, the loop is gone or hoisted. (2) The clock check — convert to operations per cycle; for a dependent chain more than one per cycle is impossible, and Chapter 3’s cautionary “0.02 ns per access on a 3 GHz machine” is about sixteen dependent accesses per cycle, which is a deleted loop rather than a fast machine. Careful with the converse: an independent summation loop can legitimately retire several elements per cycle once vectorized, so know which kind you have before calling a number impossible. (3) -O0, as a diagnostic only — confirm the shape survives, then discard those numbers. Never report -O0: nobody ships it, and it flatters memory-bound code by drowning it in instruction overhead.

Whatever you did, record the compiler and the exact flags. A benchmark without its build flags is not a result.

C.6 — Isolating a Measurement From a Machine That Rescales Its Own Clocks

Your machine is not a stable instrument. It is a control system with its own goals — finish the work quickly, do not get hot, do not drain the battery — and it will change its clock underneath you several times during your experiment.

On the default path you control almost nothing, and that is fine — this course grades a defensible number on the machine you have.

1. Interleave, do not batch. A B A B A B …, never AAAA BBBB. Drift, a governor decision, or a neighbour waking hits both arms equally when interleaved and only the second when batched. Batched comparisons are where a “regression” comes from when nothing regressed.
2. Repeat in the opposite order. If A B A B and B A B A disagree by more than your dispersion, order is a confound and you say so rather than picking the run you like.
3. Report cycles as well as wall time where counters exist. A cycle count is clock-invariant; wall time is not. A run whose wall time moved while its cycle count did not is a run whose clock changed, not whose work changed.
4. Fix the power source and name it. Never compare a mains run against a battery run. The commonest missing field in student submissions.
5. Settle the machine, and say for how long. Ten minutes idle, lid open, in its final position — then do not move it, close it, or set it on a blanket halfway through.

Never run half an experiment on Monday and half on Wednesday and present the difference as a result. If one must span sessions, repeat a common control condition in each and report the control’s drift; if the control moved more than your effect, you measured the calendar.

C.7 — Controlling Placement When You Compare

Chapter 2 is why this exists: on a heterogeneous cluster all cores run the same instruction set and differ in microarchitecture. A thread that migrates mid-run has changed machines, and your median is now an average of two machines weighted by an accident of scheduling. The signature is bimodality — run times clustering into two groups rather than scattering around one centre — and a median hides it perfectly.

When you cannot pin: say so in the conditions line (“no placement control available on this platform” is complete and creditable); raise the repetition count so one migration cannot dominate the median; report min and max and comment on bimodality if you see two clumps; and do not attribute a difference to microarchitecture you could not control. If build A ran mostly on performance cores and build B mostly on efficiency cores, you measured the scheduler. Chapter 2’s migrate.cpp reports whether affinity control was available on your platform, which is what a good instrument does: it tells you what it could not do rather than silently pretending.

Pin the inputs too. The harness shuffles its chase order with std::mt19937(12345u) — a fixed seed, deliberately, so the order is identical between runs and machines. Same discipline in your own kernels: fixed seed, fixed input size, fixed file, all three recorded. A comparison whose arms saw different random data is not a comparison.

C.8 — The Statistics You Actually Need

Nothing here needs more than a sorted list and a calculator. That is deliberate.

C.8.1 — Repetitions: how many, and why odd

Use an odd count. The median of an odd sample is an observed run rather than an interpolation, so you can point at the row of raw output that produced it — with 31 repetitions, median_ns is exactly the sixteenth-fastest run. The iqr_ns is interpolated, because the 25th and 75th percentiles do not land on sample boundaries at n=31; that is fine, since dispersion is a band rather than a claim about one run.

Was 31 enough? Go to 51 and see whether the median moves by more than the difference you intend to claim. If the IQR is more than about a tenth of the median, extra repetitions will not help — the problem is the environment, not sampling. Quiet the machine, or state the noise as a limitation and shrink your claims.

C.8.2 — Median against mean

Five real timings of one kernel, in milliseconds: 12.4 12.5 12.4 12.6 24.9. Median 12.5; mean 15.0. The fifth run is not a property of your code but of whatever else the operating system did for eleven milliseconds.

Noise in a timing measurement is one-sided. A run can be delayed by a context switch, a page fault, a migration, an interrupt, a neighbour on a shared host. Nothing makes a run mysteriously faster than the machine can go. So the sample has a long right tail and the mean is dragged into exactly the events you are excluding. The median is not.

Row three is the honest exception — total energy over a shift, total time for N tasks — because the slow runs really happened and really cost you.

One reconciliation you will notice: throttle_analysis.py summarizes the soak’s delivered-work score with a mean over its burst and tail windows while summarizing clock and power with a median. That is defensible for the shipped dataset, whose plateaus are flat enough that the two agree. Your own soak has dropouts that drag a mean down, so report the median for your own soak (C.9.5), and if you quote the tool’s output, say its score summary is a mean.

C.8.3 — Dispersion, and the decimal point you cannot defend

The interquartile range is the 75th percentile minus the 25th. Half your runs fell inside that band — the whole definition and the whole interpretation. Not the standard deviation: it assumes a shape the data does not have, it is dragged by the same one-sided outliers the mean is, and it does not answer the reader’s actual question, if I ran this again, roughly where would I land? Report min and max too when you have room; the gap between the 75th percentile and the maximum is where the environment lives.

The rounding rule: quote the dispersion to one or two significant digits — never more — and round the median to that same decimal place.

The instrument may print ten digits. What you publish is what you can defend.

C.8.4 — When a difference is too small to claim

A convention, chosen so two students reach the same verdict on the same data. Say in your report that you used it. Let A and B have medians mA, mB, interquartile ranges iA, iB, and d = |mA − mB|.

This is not a statistical test. Your repetitions are not independent draws from a stationary distribution — the machine drifts thermally, the governor’s state carries between repetitions, a background process affects consecutive runs together — and those are precisely the assumptions a t-test or confidence interval needs. Computing one anyway produces the appearance of rigour and none of the substance; if a reviewer asks for a p-value, explaining why the data cannot support one is the stronger answer.

“No difference I can resolve” is a result — often the most useful sentence in a report, because it stops an organization shipping a change on the strength of noise. And percentages need a base: write B is 1.22× A, or 22% below A, name the direction, and name the metric before the winner.

C.8.5 — The honest way to write a result

Long form — measurements.xlsx:

matmul ikj, N=512
  12.4 ms, IQR 0.3, min 12.1, max 19.8, n=31 (warm-up 3)
  laptop, 8 logical CPUs (heterogeneity unknown), macOS
  clang++ -O2 -std=c++17
  mains power, idle 10 min before, ambient ~22 °C, hard desk, nothing else running
  no placement control available on this platform
  instrument: bench_harness.cpp, steady_clock
  provenance: MEASURED

Short form — a sentence of report.docx: Reordering from ijk to ikj cut the median from 41.7 ms to 12.4 ms — medians of 31 repetitions after 3 discarded, IQR 0.9 and 0.3, on mains power after ten minutes idle, clang++ -O2. Checksums identical.

Spoken form — the briefing: Three point four times faster — median of thirty-one runs, plugged in, after a ten-minute idle.

The conditions travel with the number, and in speech in the same breath, because there is no footnote in a spoken sentence.

Coach’s Note — Students ask how much of this a lab really needs. Wrong question. The right one: could a stranger with your machine reproduce this row from the log alone? Every field above exists because somebody once could not, and the missing field was always the boring one — the power source, the flags, the browser playing video. Write the boring fields. They are the only reason anyone believes the interesting ones.

C.9 — The Sustained-Load (Thermal Soak) Protocol

Every thermal claim in this book is graded against this protocol. Project 5 requires you to run it on hardware you have; Project 8 requires a stated thermal condition on every number. It is written to be unambiguous, so two students on two laptops produce comparable numbers. Where a threshold looks arbitrary, it is: the values were chosen so everyone agrees, not because physics selected them. Changing one is allowed; changing one silently is not. No phone is needed — laptops throttle, cloud instances throttle, and the shape is the same.

C.9.1 — Before you start: write it down

In measurements.xlsx, before the first run: the duration, the sampling interval, the opening and tail windows, the onset threshold, and your prediction — will the rate fall, at roughly what second, and to what fraction of its opening value. Two committed numbers. Choosing the tail window after seeing which value flatters your conclusion is the exact failure Chapter 8 exists to prevent, and it is invisible in a finished report. The defence is that you wrote it first.

C.9.2 — The load

Any load you can describe precisely enough for a stranger to reproduce is acceptable — a video encode, a compile in a loop, a SIMD kernel. “I ran some benchmarks” is not.

C.9.3 — Duration, sampling, cool-down

One reconciliation you must know. throttle_analysis.py defaults to --window 60, because the shipped dataset’s burst lasts about seventy seconds. Your laptop may begin throttling at forty seconds, in which case a sixty-second opening window already contains throttled samples and understates your peak. That is why this protocol uses thirty. If you feed your own log to the tool, pass --window 30 so the two agree, and record that you did.

C.9.4 — What to record

Emit a CSV with exactly these six columns, in this order — the schema throttle_analysis.py reads, so your own log can be analysed with the shipped tool rather than by hand:

t_seconds,clock_mhz,power_w,soc_temp_c,skin_temp_c,score

Two warnings about placeholders. The loader converts all six columns to floating point, so every column must be present and numeric even when you have nothing for it. And throttle onset is derived from the clock column — the first row whose clock is below the run’s peak clock — so a constant in clock_mhz makes the tool correctly report “throttle onset: none observed — the run never left its peak point” even when your delivered rate collapsed. That is the tool saying it had no clock data. Find the onset from score by hand using C.9.5, and say in the log which columns were placeholders. A placeholder must never appear in your report as an observation.

Everything that is not a column goes in # comment lines at the top of the CSV — the loader skips them — and in measurements.xlsx: the machine, OS and logical CPU count; the power source; the ambient temperature and how you got it; the physical setup (surface, lid, case, external cooling); what else was running and whether the host is shared; the pre-run idle duration; and the load’s worker count, chunk size and calibrated chunk time.

C.9.5 — The three numbers, defined exactly

Report peak, knee and sustained separately. They are three different claims, and collapsing them into one is the whole failure this protocol prevents.

Report all six. A soak reported as one number has thrown away the only thing it was run to find out. The knee’s duration is informative too: short means the platform stepped hard and settled, long means it hunted its way down.

The thresholds — 30 s, 300 s, 0.95, 0.03, three consecutive samples — are conventions, so your ratio and your classmate’s mean the same thing. If your machine’s behaviour makes one wrong, change it, state the new value and why, and report under both if you can. Silently choosing the threshold that flatters your curve is what this course fails people for.

C.9.6 — Reading the shape

Every soak curve has three phases, named in Chapter 5. The book’s thermal-soak.csv — synthetic teaching data, modeled on published behaviour and not a measurement of any product — is the reference shape:

Running throttle_analysis.py on that dataset prints the burst, an onset at t = 80 s, a sustained clock and power far below peak, and an “honesty line” contrasting what a sixty-second benchmark would have claimed with what fifteen minutes delivered. Never cite that dataset as a measurement of anything — cite your own soak, and the dataset only as a modeled shape, per Appendix B.

C.9.7 — The negative result

“My machine did not throttle in ten minutes” is a valid result, graded the same. Report P, S, the ratio (near 1.00) and the tail IQR; state “no onset observed under the stated threshold within the stated window”; give the two most plausible reasons — genuine cooling headroom at this load, or a load too light to reach the envelope — and say what would distinguish them: raise the worker count or the chunk weight, or extend the run, and which you would do first.

On a shared cloud host, add one more sentence and it is not optional: you cannot separate your own throttling from a neighbour’s load, because you cannot see the neighbour. Concealing that is the only thing that would weaken the report. What earns nothing is quietly re-running until the curve looks good — if you ran the soak four times and are reporting the fourth, the log says so and reports all four.

C.9.8 — The optional device path (Workbench D)

Never required; every graded requirement in this book is completable without a device. With an Android device and the debug bridge, adb shell dumpsys thermalservice shows the platform’s own view. The application-facing surface is PowerManager.getCurrentThermalStatus() and addThermalStatusListener(), reporting the ladder THERMAL_STATUS_NONE, LIGHT, MODERATE, SEVERE, CRITICAL, EMERGENCY, SHUTDOWN; plus getThermalHeadroom(int forecastSeconds), a normalized forecast where 1.0 is the throttling threshold. Log the status alongside your score samples: a chart showing the score falling and the status climbing has explained itself, where one showing only the falling score has reported a mystery. Documentation is under https://developer.android.com/.

Record the device, ambient, charging state, whether it was in a case, and the soak duration — a phone in a case on a warm desk is a different thermal system from the same phone in a hand. And never report a device’s numbers as a general fact about that product: you have one measurement, of one unit, under one set of conditions.

C.9.9 — Comparing two soaks

Everything in C.9.3 identical for both, plus the same ambient (or state the difference and treat it as a confound), the same physical position and surface, the same power source, a full cool-down between them, and interleave if you can — A, cool-down, B, cool-down, A again. If the two A runs disagree by more than your tail IQR, the day drifted and you must say so. Then compare the three numbers separately: two devices can share a peak and have completely different sustained states, and reporting only the ratio, or only the peak, discards the finding.

C.10 — Reporting: What Must Appear Beside Every Number

A number that is none of the four does not belong in the document. Delete it — always correct, never expensive. “The vendor does not publish the last-level cache size” is a true, useful sentence that earns credit; an invented figure is an integrity failure rather than a deduction, per Appendix D.

Charts. The measurement window and the thermal condition go on the chart, not in a caption and not in a spoken aside — charts get screenshotted out of reports and the conditions must travel with the pixels. Label axes and units. Anything from a synthetic dataset carries the word synthetic on the chart. If the y-axis does not start at zero, say so on the chart.

The flattering-choice disclosure. Every investigation contains one decision that helped its conclusion — a measurement window, a summary statistic, a cool-down policy, which build, which cores, which workload got weighted most. It is rarely wrong, which is what makes it dangerous. Name it before your reader does: “The choice that most helps my conclusion is ⟨X⟩; it was defensible because ⟨Y⟩; reversed, my headline becomes ⟨Z⟩, and my conclusion ⟨does / does not⟩ survive.” It costs nothing on any rubric, it is a named line on several, and it is the difference between a report and an advertisement.

C.11 — The Templates

Six document stems, fixed course-wide. Do not invent a seventh: if your work needs another document, it is a section of report.docx. The site renders these six into the formats your section submits, so renaming them breaks the submission.

Each comes three ways: blank to copy, worked with a small illustrative example, and badly — the version you are most likely to write under deadline, and the one worth studying hardest. Worked examples use your own machine’s measurements or the book’s fictional, synthetic datasets, so no sentence here can be mistaken for a claim about anybody’s real silicon.

C.11.1 — report.docx

Blank

# ⟨Project N — Title⟩

**Tier targeted:**   Normal / Medium / Hard
**Workbench:**       L / B / D — and what that limited
**Machine:**         ⟨CPU, logical CPUs, RAM, OS⟩
**Build:**           ⟨compiler and exact flags⟩
**Conditions:**      ⟨power · thermal state · ambient · other load · shared host?⟩
**Headline result:** ⟨metric named first, then the finding, then the conditions⟩
**Confidence:**      high / medium / low — and one line on why

## 1. The question       ⟨and what is out of scope⟩
## 2. Method             ⟨what you ran, warm-up + timed reps, instrument⟩
## 3. Results            ⟨median, dispersion, n, conditions, tag — every number⟩
## 4. Interpretation — mechanism, not symptom   ⟨tied to a chapter by number⟩
## 5. The Four Questions
### Performance
### Energy
### Thermals
### Placement
## 6. Provenance summary | Category | Count | Notes |   ⟨MEASURED / DERIVED / CITED / MODELED⟩
## 7. Limitations, and the flattering choice
## 8. What I could not measure, and what I would need
⟨quantity · instrument or access required · the experiment · the question it settles⟩
## 9. Conclusion   ⟨metric before finding; tradeoff named; observable reversal condition⟩
## 10. Citations   | Figure | Value | Source (URL) | Retrieved | Tag |

Worked — a small illustrative extract, on the student’s own machine

# Project 3 — The Memory Wall Lab

**Workbench:**  L. No counter access, so no MPKI this week.
**Machine:**    laptop, 8 logical CPUs (heterogeneity not reported), 16 GB, macOS
**Build:**      clang++ -O2 -std=c++17
**Conditions:** mains · idle 10 min before each sweep · ambient ~22 °C · hard desk
**Headline:**   Measured as median ns/access over 5 internal trials × 3 runs, the
                dependent chase costs 24× the strided walk at 64 MiB — the gap is
                prefetching and memory-level parallelism, not raw memory speed.
**Confidence:** medium — three clear plateaus, but I cannot distinguish the third
                from a system-level cache without associativity evidence.

## 3. Results
| Experiment | Working set | Median | Dispersion | n | Tag |
|---|---|---|---|---|---|
| cache_walk, stride 128 B | 64 MiB | 4.05 ns/access | range 3.91–4.40 over 3 runs | 5×3 | MEASURED |
| pointer_chase | 64 MiB | 96.2 ns/access | range 91.0–104.8 | 5×3 | MEASURED |
| matmul ijk, N=512 | — | 41.7 ms | IQR 0.9 ms | 31 (warm-up 3) | MEASURED |
| matmul ikj, N=512 | — | 12.4 ms | IQR 0.3 ms | 31 (warm-up 3) | MEASURED |

Checksums for both matmul orders agree to six decimals (`raw/matmul.txt`); raw
sweeps committed unmodified. The ijk/ikj difference is 29.3 ms against IQRs of
0.9 and 0.3 — larger than their sum — so I claim it: **ikj is 3.4× faster.**

## 7. Limitations, and the flattering choice
I cannot say whether the third plateau is an L3 or a system-level cache, and I
did not read the assembly, so I can name locality as a mechanism but cannot
apportion the win between locality and vectorization.

The choice that most helps my conclusion is building at -O2 rather than -O3. It
was defensible — Chapter 3 warns some compilers interchange loops at -O3 and
erase the effect — but it is the best case for my argument. At -O3 the ratio
falls to 1.9×: the direction survives, the magnitude does not. Both are logged.

## 8. What I could not measure, and what I would need
Per-level miss rates: no counter access here. On Linux with `perf` I would
capture cycles, instructions, L1-dcache-load-misses and LLC-load-misses across
each working-set size and compute MPKI, confirming my boundary hypotheses
directly rather than by inference from timing.

Badly

# Project 3

I ran the cache walk and the pointer chase and matmul.

The cache walk showed a nice staircase with L1 at 32K, L2 at 512K and L3 at 8MB.
The pointer chase was much slower, about 100ns. The ikj version was 3.4x faster
than ijk, which shows that locality matters.

Phones would be slower because they have less cache. The L2 on a phone is
usually about 4MB per core.

Conclusion: memory is the bottleneck.

The bad version is not lazy in the sense of being short. It is confident where the good one is specific: every sentence would pass a casual read, and not one could be checked.

C.11.2 — measurements.xlsx

The measurement log, the artifact this appendix is named for. Tabular, written as you go. Open it before your first run.

Blank

# Measurement Log — ⟨Project N⟩

## Environment
| Field | Value |
|---|---|
| Machine / OS | |
| Compiler + flags | |
| Workbench | L / B / D — shared host? |
| Placement control | available / not available on this platform |
| Counter access | available / not available |
| Power source | mains / battery |
| Ambient | ⟨°C, and how you know⟩ |
| Physical setup | ⟨surface, lid, case, position⟩ |
| Pre-run idle | ⟨minutes⟩ |
| Other load | ⟨what else was running⟩ |

## Predictions — recorded BEFORE any run
| # | Experiment | What I predict | Why |
|---|---|---|---|

## Runs
| # | What | Command | n (warm-up + timed) | Median | Dispersion | Min / Max | Conditions | Raw file | Tag |
|---|---|---|---|---|---|---|---|---|---|

## Soak runs
| # | Duration | Interval | P | t_onset | t_stable | S | S/P | Tail IQR | Conditions | Raw file |
|---|---|---|---|---|---|---|---|---|---|---|

## Notes and anomalies
⟨Everything that happened. A run you discarded and why. A machine that woke up.
A result you cannot explain. This is evidence of good faith, and it is read.⟩

Worked — illustrative, soak section only

## Predictions — recorded BEFORE any run
| # | Experiment | What I predict | Why |
|---|---|---|---|
| P1 | Own soak, 10 min | falls, onset ~t=180 s, S/P ≈ 0.75 | fanned laptop: later and shallower than a phone |

## Soak runs
Protocol per Appendix C.9. Fixed **before** running: duration 600 s, interval
10 s, opening window 30 s, tail 300 s, onset threshold 0.95 P held 3 samples.

| # | Duration | Interval | P | t_onset | t_stable | S | S/P | Tail IQR | Conditions | Raw file |
|---|---|---|---|---|---|---|---|---|---|---|
| S1 | 600 s | 10 s | 2.41 chunks/s | 220 s | 300 s | 1.88 chunks/s | 0.78 | 0.06 (3.2% of S) | mains, 8 workers, ~22 °C, hard desk | raw/soak-1.csv |
| S2 | 600 s | 10 s | 2.39 chunks/s | 210 s | 290 s | 1.85 chunks/s | 0.77 | 0.05 (2.7% of S) | as S1, after 12 min cool-down | raw/soak-2.csv |

S/P differs by 0.01 across the two runs, inside the tail IQR, so I report
**S/P = 0.78, run-to-run spread ±0.01.** Prediction P1 was wrong on onset
(predicted 180 s, observed 210–220 s): more cooling headroom than I assumed.

## Notes and anomalies
- S1's sample at t=340 s dips to 1.42 chunks/s. It does not meet the
  three-consecutive-sample onset rule and I did not treat it as one. A system
  process ran then; I could not identify it.
- I discarded a first attempt at S1 because I closed the lid at t≈120 s out of
  habit. Recorded here rather than silently omitted.

Badly

# Measurements

cache_walk: 0.3 ns at 4K, 4 ns at 64M
chase: 100 ns
matmul ikj: 3 runs — 12.4, 12.6, 24.9 ms — average 16.6 ms
soak: throttled to about 78%

Ran on my laptop.

The tell of a bad log is that it was written afterwards, from memory: the numbers survived and every condition evaporated, because conditions exist only at the moment of the run.

C.11.3 — soc-architecture-review.docx

Opened in Project 1, extended as each chapter teaches a new block, finished in the capstone.

Blank

# SoC Architecture Review — ⟨part or product⟩

**Subject:** ⟨the part, or "designed part" for a Track 3 capstone⟩
**Workbench:** L / B / D   **Started:** Week 1   **Last extended:** Week ⟨N⟩
**Primary sources used:** ⟨count⟩   **Figures I could not source:** ⟨count⟩

## 1. Block inventory
| Block | What it is for | What it shares | What it contends for | Confirmed? | Source |
|---|---|---|---|---|---|
⟨`Confirmed?` is `confirmed` (a primary source says this block is on this part) or
`inferred` (reasoning from the general shape of a mobile SoC). Both are fine.
Labelling an inference as a confirmation is not.⟩

## 2. CPU cluster configuration | Field | Value | Tag | Source |
⟨core classes and counts · cluster arrangement · private cache sizes⟩
## 3. Memory configuration      | Field | Value | Tag | Source |
⟨generation · interface width · data rate · peak bandwidth, DERIVED, with
`bytes/s = MT/s × bus bytes` shown⟩
## 4. Accelerator inventory     | Block | For | What it does NOT do | Tag | Source |
## 5. Storage and radios
## 6. The Four Questions, applied to ⟨N⟩ blocks  ⟨four sentences each, not adjectives⟩
## 7. Bottleneck predictions
⟨Per workload: which blocks run · which single shared resource becomes the
constraint and why that one · which of The Four Questions binds first and what a
user would observe · what would make you wrong, and what you would measure.⟩
## 8. What I could not find
| Figure I wanted | Why it matters | What kind of source would have it |
|---|---|---|

Worked — built on SoC-C, a fictional part from the synthetic soc-compare.csv

# SoC Architecture Review — SoC-C

**Subject:** SoC-C, a FICTIONAL part from soc-compare.csv, used only to show this
document's shape. Every figure below is MODELED. A real review names a real part
and cites vendor documentation.

## 3. Memory configuration
| Field | Value | Tag | Source |
|---|---|---|---|
| Interface width | 128 bits = 16 bytes | MODELED | soc-compare.csv |
| Data rate | 7500 MT/s | MODELED | soc-compare.csv |
| Theoretical peak bandwidth | **120 GB/s** | DERIVED | 7500e6 × 16 = 120e9 bytes/s |
| Private cache sizes | not specified by the dataset | — | — |

I will not supply the cache sizes the dataset does not carry. On a real part that
row reads either with a vendor citation or "not published."

The comparison the arithmetic exposes: SoC-A in the same file runs a *higher*
data rate (8533 MT/s) on a 64-bit bus and reaches 68.3 GB/s. SoC-C is not using
newer memory — it is using **twice the width**. Width buys bandwidth, and costs
package pins, board area, controller area and power, which is why a phone-class
part usually does not have it. Arithmetic, not a claim.

## 7. Bottleneck predictions
**Workload: sustained on-device token generation.**
- Blocks running: NPU (or GPU) streaming weights · memory controller · DRAM ·
  CPU orchestrating · display if the UI is live.
- Constraint: **DRAM bandwidth**, then the thermal envelope. Chapter 3's ceiling
  is `tokens/s ≤ bandwidth ÷ weight bytes`, and it does not move when you add
  multiply-accumulate units.
- Binds first: **thermals**. This part's sustained/peak ratio in the dataset is
  0.85 and its sustained power is the highest of the four — a *form-factor
  statement*, not a virtue. Nothing dissipating that much sits on a phone-class
  thermal path.
- I'd be wrong if the runtime is dominated by per-token overhead rather than
  weight streaming. To find out: measure tokens/s at two quantization levels —
  halving bytes-per-weight should roughly double a bandwidth-bound rate and
  barely move an overhead-bound one.

## 8. What I could not find
| Figure I wanted | Why it matters | What kind of source would have it |
|---|---|---|
| Last-level / system cache size | Decides how much traffic reaches DRAM at all | Vendor architecture documentation; frequently not published |
| Sustained thermal envelope | Every claim in §7 depends on it | Vendor thermal guidance, or my own soak on real hardware |

Badly

# SoC Review

The SoC has a CPU with 8 cores at up to 3.2 GHz, a GPU, an NPU rated at 45 TOPS,
16 MB of system cache, and LPDDR5X memory with 77 GB/s of bandwidth. It is built
on a 3nm process.

The GPU is very powerful and the NPU makes AI fast. The main bottleneck will be
the CPU when running heavy applications.

Note what the bad review has that the good one does not: more numbers. Six confident unsourced figures are worth strictly less than two cited figures and four honest “not published” rows, because the second can be built on and the first cannot.

C.11.4 — placement-decision.docx

The placement table: workload → chosen processor → why → what it costs. The rule it applies — the cheapest processor that meets the deadline — comes from Chapter 2 and Chapter 4; this is the format, not the reasoning.

Blank

# Placement Decision — ⟨project / product⟩

**The requirement, stated before any choosing:**
⟨The latency budget, or per-workload budgets, justified from the product. A
60 fps frame budget is 1000/60 ≈ 16.7 ms. A wake-word response is a different
number. A background draft is a third. Say which and why.⟩
**Energy ceiling, if any:** ⟨and where it comes from⟩
**Data provenance:** ⟨name the dataset; if synthetic, say so HERE, not in a footnote⟩

| Workload | Processor | Why (and what it beat) | What it costs |
|---|---|---|---|

⟨Rules, all graded: every workload gets a row, including any whose answer is
"none of these"; Why names at least one **rejected** alternative and why it lost;
What-it-costs is a **number with a unit**, never an adjective; and at least one
row shows energy scaled to a real **duty cycle** — per-unit energy × how often it
runs, multiplication shown.⟩

## Defense   ⟨one paragraph per row, under all Four Questions⟩
## What this table does not model
## What I would measure to confirm it
⟨quantity · processor · tool · baseline · the result that would change a row⟩

Worked — illustrative, using the six workloads of the synthetic placement-bench.csv

# Placement Decision — field clinic handheld

**The requirement, stated before any choosing:** a split budget. Anything the
clinician waits on: **50 ms**. Background work: **2000 ms**. The 50 ms is the
product's responsiveness target for a hand-held capture flow — looser than a
16.7 ms frame budget because nothing here is a continuous animation.
**Data provenance:** every figure below is **MODELED**, from placement-bench.csv
— synthetic teaching data, not a measurement of any product. The *ratios* are the
lesson; the absolutes are not a claim.

| Workload | Processor | Why (and what it beat) | What it costs |
|---|---|---|---|
| `image_classification` | NPU | 6.5 ms, lowest energy. Beat GPU (11.5 ms, 26.0 mJ — 3.7× the joules) and CPU (48.0 ms, over budget) | 7.0 mJ per inference |
| `video_encode_4k30_1s` | **None — needs a block absent from the dataset** | Best listed option is the DSP at 1450 ms for one second of video: it cannot keep up with real time, so capture is impossible. The NPU is not slow but architecturally unable — entropy coding is serial and bit-exact, not a tensor graph | A fixed-function encoder, costing die area that is dark whenever nobody records |
| `game_frame` | GPU | 9.4 ms, inside a 16.7 ms budget. NPU and DSP UNSUPPORTED — no rasterizer, no texture units | 44.0 mJ/frame → **at 60 fps: 44.0 × 60 = 2640 mJ/s = 2.64 W** for rendering alone — the arithmetic that decides whether this product can run a 3D viewer |
| `wake_word` | DSP (always-on sensor hub) | 2.4 ms, **0.06 mJ**. Beat the NPU, faster at 1.9 ms but 0.42 mJ — 7× the energy, because the main-SoC NPU forces the whole SoC out of its deepest idle state. The latency difference is invisible; the energy difference runs all day | 0.06 mJ per 1 s window → **86 400 × 0.06 = 5184 mJ ≈ 5.2 J per 24 h** |
| `llm_token_gen` | NPU | 24.0 ms, 9.5 mJ. GPU is faster (18.0 ms) at 21.0 mJ — 2.2× the joules for 6 ms nobody perceives | 9.5 mJ/token → a 200-token note is 1.9 J |
| `photo_pipeline` | DSP | 45.0 ms, 31.0 mJ. **The NPU is the trap**: 190 ms and 240 mJ, because two custom operators are unsupported, the graph splits, and three CPU round trips follow — **4.2× the latency and 7.7× the energy** | 31.0 mJ per capture |

## What this table does not model
Every figure is a per-unit number with no statement of thermal steady state.
`game_frame` is the most exposed — 2.64 W for rendering alone, continuously, on a
sealed handheld is a thermal event, and the placement that is right for one frame
may be wrong in the tenth minute. It also models each workload alone; nothing
accounts for two contending for one memory controller.

## What I would measure to confirm it
For `photo_pipeline`: run the graph on the accelerator path and on a CPU-only
control over the same 200 inputs, report median and IQR per Appendix C, and ask
the runtime which operators it actually placed. If more than one falls back, the
row stays on the DSP. That question — *which operators actually ran there* — is
the one an activity meter will never answer.

Badly

# Placement Decisions

| Workload | Processor |
|---|---|
| image_classification | NPU |
| video_encode | NPU |
| game_frame | GPU |
| wake_word | NPU |
| llm_token_gen | NPU |
| photo_pipeline | NPU |

The NPU is the AI accelerator so AI workloads go there. The GPU handles graphics.
Every workload above meets its latency target, so this assignment is optimal.

Four of six rows say NPU, and the prose says why: “the NPU is the AI accelerator so AI workloads go there.” That is placement by the name of the block rather than the shape of the work — the exact reasoning Chapter 4 exists to break.

C.11.5 — threat-model-report.docx

Chapter 7’s deliverable. The graded core is the residual risk column — what each mechanism does not cover. A test of nerve rather than knowledge: you already know the answers, and the difficulty is writing them into a document somebody is paying for.

Blank

# Threat Model — ⟨device⟩ in ⟨scenario⟩

**Bad outcome for this user:** ⟨one sentence — it differs by scenario⟩
**Adversaries modeled:** ⟨three or more⟩   **Assets:** ⟨five or more⟩
**The single largest residual risk:** ⟨one sentence, no hedging⟩
**Verdict:** ⟨one sentence, to the decision-maker⟩
**Synthetic data used:** ⟨name it⟩

## 1. The engagement  ⟨what the device must do; what counts as a bad outcome⟩
## 2. Assets
| Asset | Why it has value **to the adversary** | What its exposure would cost |
|---|---|---|
⟨At least one asset is not data — a capability, an identity, or a credential that
lets the holder act as the device.⟩
## 3. Adversaries
| Adversary | Capabilities (explicit) | Not capable of |
|---|---|---|
⟨At least one has physical possession. That is what makes this a *mobile* threat
model rather than a server one.⟩
## 4. Mechanisms and residual risk
| Asset | Defending mechanism(s) | Which adversary it stops | **Residual risk — what it does not cover** |
|---|---|---|---|
⟨Every residual-risk cell is specific and falsifiable, naming *this* asset,
*this* mechanism, *this* adversary. "Some risk remains" earns zero.⟩
## 5. Chain analysis  ⟨injected faults, raw output, attacker gain, data provenance⟩
## 6. Measurement     ⟨the side-channel measurement per Appendix C, tied to this model⟩
## 7. Verdict         ⟨one sentence. If it takes three, you have not decided⟩

Worked — illustrative and structural; no product named, no figure invented

# Threat Model — warehouse wrist scanner, fleet deployment

**Bad outcome for this user:** a unit that goes home in a coat pocket becomes a
working credential against the warehouse management system, and nobody notices
for a week.

## 2. Assets (extract)
| Asset | Why it has value **to the adversary** | What its exposure would cost |
|---|---|---|
| Fleet credential (**not data — a capability**) | It authenticates *as the fleet*; possession is the ability to act | Fleet-wide re-issue; unbounded fraudulent transactions until detected |
| Scan history on the device | Reveals inventory movement and staffing patterns | Commercial; low per unit, high in aggregate |

## 4. Mechanisms and residual risk (extract)
| Asset | Defending mechanism(s) | Which adversary it stops | **Residual risk — what it does not cover** |
|---|---|---|---|
| Fleet credential | Hardware-backed key store; the application asks for an *operation*, never for the key | The opportunist with an hour and ordinary tools — they cannot extract key material | **It does not stop that adversary from using the credential.** The key never leaves the secure subsystem, and a possessor of an unlocked, un-revoked unit can still ask it to sign. The defence against *use* is revocation, an operational control with a detection delay we have not measured |
| Scan history | Full-disk encryption, key released after user authentication | An adversary acquiring a powered-off unit | Does not protect a unit taken while unlocked and in use, nor data already synced to the WMS, nor backups |
| Device identity | Verified boot with rollback protection; attestation to the server | An adversary installing a modified or older image | Verified boot proves what *started*. It does not prevent a live exploit against a running, verified system from acting with that identity for as long as the session lasts |

## 7. Verdict
If a unit leaves the site and is not recovered, its stored scan history is
protected against everything short of die-level attack, and its fleet credential
is protected against extraction but **not** against use until revocation
completes — so the security of this deployment is the speed of your revocation
process, not the strength of the silicon.

Badly

# Threat Model

The device uses TrustZone, secure boot, and encryption, so it is secure.
Attackers include hackers and thieves. Data is protected by hardware.
Some risk always remains but the device implements multiple layers of
hardware-backed protection.

The failure here is writing toward reassurance, the professional temptation of the whole discipline. Your value is not making people feel safe. It is telling them accurately what they are safe from, which is the only information that lets them decide anything.

C.11.6 — ai-usage.txt

Appendix D owns the policy and defines the note; this is the shape it takes on the page. Graded on honesty and specificity, not abstinence — “I did not use AI” is complete and acceptable when true. The rule, in five words: use it to explain, never to source.

Blank

# AI Usage — ⟨Project N⟩

**Models used:** ⟨name and version, or "none"⟩

**What I used them for:** ⟨specific — "explain why the IQR is preferred to the
standard deviation for timing data"; "review the prose in §4"; "check my algebra"⟩

**What I did NOT use them for:** No figure in this submission came from a model.
Every number is measured by me and logged in `measurements.xlsx`, derived by
arithmetic shown in `report.docx`, cited to a primary source I opened, or drawn
from a dataset shipped with this book and labelled synthetic where used.

**Where I overrode it, or caught it being wrong:** ⟨the part people skip, and the
part read most carefully. Be specific.⟩

**Signed:** ⟨your name⟩

Worked

# AI Usage — Project 3

**Models used:** ⟨assistant name and version⟩.

**What I used them for:** three things, all explanatory. (1) I did not follow why
a power-of-two stride is pathological, and asked for a walk-through of set
indexing; the explanation matched §3.3. (2) I asked it to review §4 for clarity
and cut two paragraphs on its suggestion. (3) I asked it to check my Little's Law
arithmetic, which it did correctly.

**What I did NOT use them for:** no figure here came from a model. Every number
is measured by me and logged with its conditions, derived by arithmetic shown in
the report, or drawn from `model-memory.csv`, labelled synthetic where used.

**Where I overrode it:** as an experiment for Rep 10 I asked for the L2 cache
size of a named shipping part. It answered with a specific figure, confidently,
with a plausible generation name attached. I could not find that figure on the
vendor's own page, and the vendor does not appear to publish it at all. I
therefore report "not published" and use my own measured plateau as evidence. I
could not have told from its tone that the answer was unreliable — which is the
point of the rep.

**Signed:** ⟨your name⟩

Badly

# AI Usage

I used AI for research and to help with some of the writing.

An honest disclosure protects you. A vague one makes a grader guess about your process — and a grader guessing about your process will read your numbers the same way.

C.12 — The Pre-Submission Audit

Ninety seconds per deliverable, every week, and the capstone will not surprise you.

measurements.xlsx

• Every row has a median, a dispersion figure, and a repetition count split into warm-up and timed.
• Every row states the machine, the build flags, the power source, and the thermal condition.
• Every row names its raw output file, and those files are committed unmodified.
• Predictions are recorded, unedited, before the results they precede.
• Every soak reports P, t_onset, t_stable, S, S/P and the tail IQR — six numbers, not one.
• The anomalies section names every run you discarded and why.
• A stranger with your machine could reproduce any row from the log alone.

report.docx

• Search for every number. Each is MEASURED, DERIVED, CITED or MODELED. Anything else is deleted, not softened.
• No bare single numbers anywhere, including in prose.
• Every median is rounded to the place its dispersion supports.
• Every claimed difference passes C.8.4’s test, or is explicitly labelled unresolvable.
• Every chart carries its measurement window and thermal condition on the chart.
• Synthetic data is labelled synthetic in the sentence where it is used, with what that limits.
• The Four Questions are answered separately, under their own headings.
• The conclusion names its metric before its finding.
• The flattering choice is named, defended, and the headline restated with it reversed.
• “What I could not measure and what I would need” names instruments and experiments, not feelings.

Everything else

• Exactly the required filenames, spelled exactly. No seventh document.
• Every external figure has a URL and a retrieval date, and you clicked each link.
• ai-usage.txt names models, uses, and at least one override — or plainly states there was none.

Coach’s Note — The audit finds the same thing every term, and it is never the interesting number. It is the power source. The compiler flag. The run you discarded on Tuesday and forgot to mention. Those are the fields that turn a defensible measurement into an anecdote, and they are boring, which is precisely why they are missing. Do it before you are tired, not after.

C.13 — Where to Go From Here

• “It will not build / what is Workbench B?” → Appendix A.
• “Where did this dataset come from, and what may I claim from it?” → Appendix B.
• “How is this scored, and what exactly is the AI policy?” → Appendix D.
• “What does this word mean?” → Appendix E.
• The mechanisms behind the methodology → Chapter 3 for why memory decides most results, Chapter 5 for why the soak protocol exists, and Chapter 8 for counters, comparison, and the argument this appendix is the operational form of.
• The tool landscape — Arm Performance Studio and Streamline for Arm targets (https://developer.arm.com/), Perfetto for Android and Linux system tracing (https://perfetto.dev/), simpleperf and the Jetpack benchmark libraries on Android (https://developer.android.com/), perf on Linux, Instruments and xctrace on Apple platforms, and portable C++ steady_clock everywhere.

Nobody who reads your report will rerun your experiment. That is not cynicism about your colleagues; it is arithmetic about their time. They will read a number and act on it, and the entire chain between what happened on your machine and what your organization decides is you. Every rule above exists to make that chain inspectable by somebody who cannot rerun it: the conditions attached to the number, the dispersion attached to the median, the provenance tag attached to the figure, the flattering choice named before anyone finds it.

State the conditions before you state the number. Report the median with its dispersion. Tag every figure. Name the choice that helped you. Do those four things every week for eight weeks and they stop being a checklist and start being how you think — which is the only version of this that survives a deadline.