Chapter 12 — Reps

The keyboard is the gym, and this week the gym has a meter on the wall. Every rep here trains the one skill that separates a cloud admin from a cloud tourist: predicting the bill before it arrives, then reading it honestly when it does.

Ground rules

• Type it yourself. No copy-paste. The muscle memory of computing a token bill by hand is the point; you will do it in front of a stakeholder one day with no calculator.
• Run everything. Where a rep gives a script, run it. Where it gives a price, re-verify it against a live vendor console — every price in this book is a mid-2026 snapshot and will have drifted.
• Predict before you measure. Every cost rep asks you to write down the number you expect before you run the calculator or read the bill. The gap between your prediction and reality is the lesson. Honor it.
• AI policy (Phase 2, Week 12): AI is part of the work and increasingly agentic. You may use a copilot to draft a billing query or a script. You must do the arithmetic and the crossover reasoning yourself first, verify any price/unit the AI gives you against a live console, and keep an honest agent-log.txt of what you delegated and where it was wrong. The human owns the verdict.
• No real secrets. Use placeholder keys ($PROVIDER_API_KEY) and never commit a key. A leaked key with model-invoke rights is a credit card with no limit.

Reps 1–3: Read the Meter

Rep 1 — One prompt, three tiers, by hand

A single request is 1,000 input tokens and 500 output tokens. Using the mid-2026 snapshot prices from §12.3, compute the per-request cost on Claude Opus 4.8 ($5/$25 per 1M), Claude Haiku 4.5 ($1/$5), and Gemini 3 Flash ($0.50/$3). Write down all three by hand first.

cost = (in_tokens / 1e6 × in_price) + (out_tokens / 1e6 × out_price)

Then run code/token_cost.py to check yourself:

python3 code/token_cost.py --in 1000 --out 500 --in-price 5 --out-price 25

Reflect (3–4 sentences): What is the ratio between the most and least expensive tier for this identical request? You predicted before you computed — how close were you, and which number surprised you?

Rep 2 — Project to volume, where the money actually is

Take your Rep 1 per-request costs and multiply each by 1,000,000 requests/month. Write the three monthly figures.

python3 code/token_cost.py --in 1000 --out 500 --in-price 5 --out-price 25 --requests 1000000

Reflect: A per-request cost that reads as “free” (under two cents) becomes a five-figure monthly line item. Which tier would you actually ship for a high-volume support-ticket-classification task, and why is shipping the flagship here a stewardship failure, not just a budget one?

Rep 3 — The output-token tax

Re-run Rep 1’s Opus request but flip the shape: 500 in / 1,000 out, then 1,000 in / 1,000 out, then 2,000 in / 200 out. Predict which is most expensive before computing.

python3 code/token_cost.py --in 500 --out 1000 --in-price 5 --out-price 25

Reflect: Output tokens cost ~4–8× input. State one concrete prompt-design change (e.g., “ask for a one-line answer,” “cap max_tokens”) that cuts the bill, and estimate the saving on the 1,000-out case.

Reps 4–6: Taxi, Lease, or Own

Rep 4 — Find the reserved-vs-serverless crossover

Your workload is projected at a steady 250M tokens/month on a mid-tier model. Using the third-party rule-of-thumb break-even (~150–200M tokens/month for a mid-tier model — flag it as third-party, not vendor-official), decide whether serverless or reserved is the call, and write one sentence of why the rule of thumb might be wrong for your case (duty cycle, spikiness, region sell-out risk).

Reflect: Reserved capacity buys a guarantee as well as a price. Name a workload where you would pay for reserved even slightly below the dollar break-even, and say what you are buying.

Rep 5 — Self-host vs API for sensitive data

A Christian K-12 school wants an AI tutor that touches student records (FERPA-sensitive). Volume is modest and business-hours-only. Fill in this decision table for the school, then write your recommendation in two sentences.

Reflect: The cheapest-dollar option and the correct option may differ. Which did you choose, and which non-dollar column decided it? (Hint: the “rent the GPU, self-host the model” third option from §12.5 is a live candidate — but watch idle GPU-hours.)

Rep 6 — Idle GPU math

A cloud H100-class instance you keep running 24×7 for a workload that is actually busy only 6 hours on weekdays. Assume an illustrative $4/GPU-hour (re-verify live). Compute the monthly cost as-run vs. the cost if you scheduled it to run only during the busy window (~6h × 5 days × ~4.3 weeks).

Reflect: What is the wasted spend, in dollars and as a percentage? Write the one-line cron/scheduler policy you would put in place so this never bills you for idle silicon again.

Reps 7–9: Wire the Controls

Rep 7 — Decode the abstract units

You need ~60,000 tokens/minute of guaranteed throughput. Without converting between them (you can’t cleanly — that’s the lesson), write down what you would have to size separately for each platform: Azure PTU, Bedrock model unit, Google GSU. You don’t need exact numbers; name the unit and the consequence.

Reflect: Why is the non-convertibility of these units itself a lock-in mechanism? What’s the one architectural choice (§12.2) that preserves your ability to leave?

Rep 8 — Stand up FinOps controls

Open code/finops_alerts.sh and read it end to end. It sketches a daily spend pull, a per-tag breakdown, and a threshold alert. Adapt the threshold and tag keys to a hypothetical project, and dry-run it:

bash code/finops_alerts.sh --budget 5000 --threshold 0.8 --tag-key project

Reflect: List the four FinOps controls from §12.6 (tagging, budgets/alerts, hard caps, anomaly detection) and state which one stops runaway spend vs. which only warns. Which would you wire first, and before or after deploying the workload?

Rep 9 — Cost the agent cascade

An agentic feature turns one user question into: 1 planning call (2k in / 500 out), 3 retrieval/tool calls (1k in / 200 out each), 1 guardrail call (1k in / 50 out), and 1 synthesis call (4k in / 800 out) — all on Sonnet 4.6 ($3/$15). Compute the cost of one user question end to end. Predict first.

Reflect: Compare it to a single non-agentic call (4k in / 800 out) on the same model. What’s the multiplier? Where in the cascade would you swap a cheaper tier to cut cost without hurting quality?

Reps 10–11: Read a Real Bill

Rep 10 — Reconcile prediction vs. actual

Use code/sample-bill.csv — a small, synthetic cloud AI billing export (per-line: service, model, tokens, tag, cost). Total it, break it down by tag, and identify the single most expensive line.

python3 - <<'EOF'
import csv, collections
by_tag = collections.Counter()
total = 0.0
with open('code/sample-bill.csv') as f:
    for row in csv.DictReader(f):
        c = float(row['cost_usd']); total += c
        by_tag[row['tag']] += c
print(f"total ${total:,.2f}")
for tag, c in by_tag.most_common():
    print(f"  {tag:<16} ${c:,.2f}")
EOF

Reflect: Which tag/project is the cost driver? Is the most expensive line justified (a high-value workload) or waste (an idle instance, a flagship doing trivial work)? A FinOps copilot would flag the idle line — but only you can tell waste from insurance. Which is it here, and how do you know?

Rep 11 — Catch the runaway

In code/sample-bill.csv there is one line whose cost is anomalous versus the daily baseline. Find it. Write the budget-alert threshold and the hard cap you would have set to catch or stop it, and say which control would have prevented the spend vs. merely reported it after the fact.

Reflect: Tie this to Chapter 5’s behavioral baseline and Chapter 11’s preparedness. Anomaly detection on the meter is the same idea as anomaly detection on the wire — what’s the analogous “normal baseline” for spend, and what triggers the alert?

Done? One Last Thing.

A relief organization wants to deploy an AI assistant that drafts donor thank-you letters from a database that includes donor PII. Expected volume: ~400,000 requests/month, fairly steady, 24×7. They have a small but competent IT team and a tight budget.

In one page, do the whole chapter in miniature — this is P12 in rehearsal:

1. Pick a billing model (serverless per-token vs. reserved) and justify it against the ~150–200M-token rule of thumb, given the request volume and a realistic token-per-request estimate. Show the arithmetic.
2. Pick a model tier and defend it on cost and fitness for “draft a warm thank-you letter.”
3. Make the build-vs-buy call — API, self-host, or rent-GPU-self-host-model — weighing the donor-PII residency requirement, the steady volume, and the team’s capacity. Name the deciding column.
4. List the FinOps controls you would wire before launch: the tags, the budget, the 80% alert, the hard cap.
5. Write the one paragraph you would put in the architecture memo answering the week’s question: where is this org’s treasure, and what does it cost — in dollars and in dependence — to keep it elsewhere?

Predict the monthly bill before you compute it. Then compute it. Keep your agent-log.txt if you used any AI to draft the memo.

Up next: Project 12 — Project 12: Deploy in the Cloud, Read the Bill.