Chapter 4 — Reps

Conditioning, not grading. This week’s reps train one motion: pricing an architectural choice in bytes and joules instead of in adjectives — divergence, tile traffic, quantization error, memory-bound token rates, and placement under a stated budget.

Ground rules:

1. Type every command yourself. No copy-paste from the chapter. You should be able to build code/tile_cost.cpp and drive code/partition.py from memory by Friday, because the practical does not come with a cheat sheet.
2. Run everything. Every program in code/ runs on Workbench L and on Workbench B — a C++17 compiler and a Python 3 interpreter is the entire requirement. No phone, no admin rights, no third-party packages. If you are unsure which workbench you are on, Appendix A sorts it out.
3. Predict before you measure. Every rep that produces a number: write your prediction down first, then run it, then write the gap. The gap is the rep. A prediction you did not record is a memory you will rewrite after the fact.
4. AI policy — use it to explain, never to source. Ask an assistant what a lockstep group is, why per-channel scales exist, what a delegate does. Do not ask it for a group width, a cache size, a TOPS rating, an API status, or any other figure. Every number in a submitted artifact needs a primary source: a vendor specification page, an architecture reference manual, this book’s grounded text, or your own run. End every AI-touching rep with a one-line honest AI usage note.
5. Say when the data is synthetic. code/placement-bench.csv is modelled teaching data. Its first line says so. Any sentence you write from it says so too.

Reps 1–3: The Shape of the Work

Rep 1 — Sort six workloads by the shape of their parallelism

No tools. Paper, or a text file, and §4.2’s three questions: how wide is the parallelism, how regular is the control flow, how much data moves per unit of arithmetic?

For each of these six, answer all three questions in one line each, then name the processor you expect to win and the processor you expect to be the surprising near-miss:

image_classification     one 224x224 inference of a small INT8 CNN
video_encode_4k30_1s     one second of 4K30 video, encoded
game_frame               one rendered frame at a 60 fps target
wake_word                one 1-second audio window scanned for a keyword
llm_token_gen            one decoded token from a small on-device model
photo_pipeline           one burst-fusion still capture, end to end

Keep this file. You will check it against code/placement-bench.csv in Rep 9, and being wrong now is free. Write one sentence on which of the six you found hardest to classify and why.

Rep 2 — Price a divergent branch inside a SIMT group

Arithmetic, by hand. Take a lockstep group of 32 lanes running this fragment:

if (is_edge(x, y)) {
    heavy_edge_filter();     // 120 cycles
} else {
    cheap_interior_path();   // 20 cycles
}

Predict first, then compute, for each case: how many cycles does the group take, and what is the group’s useful throughput as a fraction of peak?

1. All 32 lanes take the else.
2. 16 lanes take the if, 16 take the else.
3. One lane takes the if, 31 take the else.
4. A different branch entirely, where each of the 32 lanes takes a different one of 32 equal-cost paths.

Then answer the question that matters: in case 3, what were the other 31 lanes doing during the 120-cycle branch, and what would a utilization dashboard have reported while they did it? Two sentences. This is the seed of Chapter 8’s “utilization is not performance.”

Rep 3 — Verify one accelerator claim against a primary source

Pick one claim from this chapter that is deliberately hedged — the SIMT group width on a named mobile GPU, whether a specific SoC’s video block does AV1 encode as well as decode, or which processors a named runtime will actually target on a named device.

Now source it properly. Find the vendor’s own specification page or architecture reference manual, and record: the exact claim, the URL, the date you retrieved it, and whether the primary source actually answers the question or merely gestures at it.

Then do the control experiment. Ask an AI assistant the same question, cold. Save its answer verbatim. Compare. Write three sentences: what the primary source said, what the assistant said, and whether you could have told the difference without checking. AI usage: required — name the model and version. This rep is the reason the AI policy exists.

Reps 4–6: The Tile-Based Argument, in Bytes

Rep 4 — Run tile_cost.cpp and read the overdraw sweep

Build it and run the default configuration:

g++ -std=c++17 -O2 -o tile_cost code/tile_cost.cpp
./tile_cost

Predict first: at 2400 × 1080 with an overdraw factor of 3, roughly how many megabytes reach DRAM per frame under immediate mode? Under tile-based? Write both numbers down before you look.

Now sweep the overdraw factor with the geometry held fixed:

./tile_cost 2400 1080 32 1.0 150000
./tile_cost 2400 1080 32 2.0 150000
./tile_cost 2400 1080 32 4.0 150000
./tile_cost 2400 1080 32 8.0 150000

Record the immediate-mode figure, the tile-based figure, and the ratio at each point. Write two sentences: why does the tile-based number not move at all across that sweep, and what does that tell you about which resource tile-based rendering was designed to protect?

Rep 5 — Price a gratuitous framebuffer readback

Same tool, third output row. At the default configuration, record the tile-based traffic with and without one full-frame readback, and compute what fraction of the tiling advantage the readback destroys.

Then push it:

./tile_cost 2400 1080 32 1.0 150000

At overdraw 1.0, the “tile-based + one readback” row goes worse than immediate mode. Confirm that on your own output. Write two sentences: describe a concrete piece of application code that would cause this — a post-processing effect, a screenshot, a UI compositor step — and state the restructuring that avoids it. This is the single most valuable sentence a mobile graphics engineer says in a code review.

Rep 6 — Find the geometry crossover where tiling stops paying

Hold overdraw at 1.0 and sweep the triangle count instead:

./tile_cost 2400 1080 32 1.0 150000
./tile_cost 2400 1080 32 1.0 300000
./tile_cost 2400 1080 32 1.0 600000

Predict first: does tile-based rendering ever lose? Then find, by bisection, the triangle count at which the two architectures produce equal DRAM traffic at overdraw 1.0.

Write two sentences: name the assumption in the model that produces the crossover (look at the assumptions line the program prints), and say what real-world scene would sit on the losing side of it. Then one more sentence on why this does not mean tile-based rendering was a mistake.

Reps 7–8: Quantization You Can Check

Rep 7 — Hand-compute the INT8 memory saving, then run quantize.py

No tools first. A weight matrix of 4 channels × 32 weights. By hand, compute: the FP32 footprint in bytes, the INT8 footprint in bytes, the exact ratio, and then the overall ratio once you add one FP32 scale for per-tensor and four for per-channel. Write all four numbers down.

Now check yourself:

python3 code/quantize.py

Then scale it up and confirm your reasoning generalizes:

python3 code/quantize.py --rows 8 --cols 64 --seed 7

Write two sentences: why is the 4× exact while the overall ratio is not, and what happens to the scale overhead as the channels get longer? Then one more: for a 3-billion-parameter model, how many bytes does moving FP32 → INT4 save, and does that change which device the model fits on?

Rep 8 — Break per-tensor quantization with one loud channel

Run code/quantize.py again and read only the second table — the hostile matrix, where channel 0 carries values roughly forty times larger than the rest.

Predict first: which column will show the damage — max error, mean error, or mean error on the quiet channels? Commit to an answer before you look.

Now read all three columns for both schemes and compute the per-tensor-to-per-channel ratio on the mean err, quiet ch. column. Write three sentences: why the max-error column barely distinguishes the two schemes, why the quiet-channel column does, and what an aggregate accuracy metric would have hidden about this model. Then the professional question in one line: what would you have to measure to know whether this mattered for your task?

Reps 9–11: Placement, Defended

Rep 9 — Build a placement table under a stated latency budget

Run the placement tool at two budgets and read both carefully:

python3 code/partition.py --budget-ms 50
python3 code/partition.py --budget-ms 20 --verbose

Predict first: which of the six workloads changes processor between the two budgets? Write your guess before running the second command.

Now build the artifact. In a table with four columns — workload → chosen processor → why → what it costs — write one row for each of the six workloads at the 50 ms budget. The “why” must name the alternative you rejected and the reason. The “what it costs” must be a number with a unit.

Write two sentences on the workload whose assignment flipped between budgets, explaining what changed. Then one sentence stating plainly that the dataset is synthetic and what that limits about your table.

Rep 10 — Walk into the operator-fallback trap on purpose

In code/placement-bench.csv, find the row whose notes begin OPERATOR FALLBACK. Read the whole workload’s four rows before you do anything else.

python3 code/partition.py --budget-ms 500 --workload photo_pipeline --verbose

Write down, in order: which processor an engineer would assign this workload to on intuition alone; what the fallback row’s latency and energy actually are; the multiple by which the intuitive choice is worse than the winner; and — the real question — what you would run, on real hardware, to detect that this had happened to you. Name the specific thing you would ask the runtime for. Two sentences on why a dashboard reporting “NPU busy” would not have told you.

Rep 11 — Show that token generation is memory-bound

Arithmetic, by hand, using §4.8’s method. For a 3-billion-parameter model at INT4, batch size 1:

1. Bytes moved per token.
2. Operations per token (roughly two per weight).
3. Arithmetic intensity, in operations per byte.
4. The time-per-token floor at an assumed 50 GB/s achievable bandwidth, and the resulting tokens per second.

Now the interesting part. Redo step 4 twice: once assuming the MAC array doubles and bandwidth does not, and once assuming bandwidth doubles and the MAC array does not. Write two sentences on which change moves the number and why. Then compute the KV cache for that hypothetical 28-layer, 8-KV-head, head-dimension-128 configuration at 2048 and at 8192 tokens of context, and state in one line what that does to the total footprint.

Label every assumption as an assumption. The 50 GB/s is not a measurement and you may not present it as one.

Done? One Last Thing.

This is the practical in miniature — one placement, fully defended.

Pick a single workload from code/placement-bench.csv. Then, in about 300 words:

1. State the requirement. A latency budget and, if you like, an energy ceiling. Without a stated budget the rest of this is decoration.
2. Assign it. Name the processor, using code/partition.py to check yourself.
3. Defend it against the alternatives. For each of the other three processors, one sentence on why it lost — over budget, more joules, or not architecturally expressible.
4. Name what it costs. In millijoules, and in whatever else the choice spends: die area, thermal headroom, an operator-coverage risk.
5. State the honest limits. The dataset is synthetic; say so, and say what that limits. Then name the one measurement you would take on real hardware that would most change your confidence in this row — a specific tool, a specific counter, a specific comparison against a specific baseline.

Keep this write-up. It is one row of the placement-decision.docx the practical asks for, and you have already drafted it — five more rows and a defence and you are done.

Up next: the Week 4 Placement Practical, issued in Canvas — the take-home worth 8% of your grade. Week 4’s graded knowledge check is the cumulative auto-graded checkpoint covering Weeks 1–4 (10%); the Week 4 practice quiz is ungraded, and there is no ordinary weekly lab this week. Methodology and how to report a number honestly live in Appendix C; dataset provenance is in Appendix B.