Chapter 12 · Week 12

Cloud Systems Administration and Counting the Cost

Where is your treasure, and what does it cost to keep it elsewhere?

Chapter 12 — Cloud Systems Administration and Counting the Cost

“The cloud is just someone else’s computer.” — industry adage (sticker-and-mug provenance, true anyway)

“For where your treasure is, there your heart will be also.” — Matthew 6:21 (ESV)


Why This Matters

For eleven weeks you have administered machines you could, in principle, walk over and touch. Even when they were virtual, even when they were containers on someone’s Kubernetes node, there was a floor of ownership under your feet: you sized the GPU, you carved the partition, you built the ark. This week the floor moves. In the cloud you rent the machine, you rent the model, and — this is the part that catches good administrators off guard — you rent them by the unit the vendor invented for the express purpose of being hard to compare. Azure bills AI inference in one abstract unit, AWS in another, Google in a third. The model is often the same model. The bill is the vendor.

That is the whole chapter in one sentence: in the cloud, the engineering question and the financial question are the same question. You cannot architect a cloud AI system competently without reading the bill, and you cannot read the bill without understanding the architecture. A junior admin spins up a GPU instance to “test something” and forgets it over a long weekend; on Monday the org has spent four figures on idle silicon. A team ships a chatbot on a frontier model’s flagship tier when a model a tenth the price would have answered every question users actually ask. These are not accidents of carelessness. They are failures of stewardship — and stewardship is the spine of this week.

The dual-AI thread runs straight through the meter. AI as the tool you wield: the cloud sells you AI as a managed service — call an endpoint, get an answer, never own a GPU — and it sells you AI to manage the cloud, the FinOps copilots that watch your spend and recommend right-sizing. AI as the workload you run and govern: AI is now the single most expensive line item a cloud admin signs off on, and the one most prone to silent cost explosions, because reasoning and agent stacks turn one user question into a cascade of billed model calls. You will learn to forecast that cascade before it bills you.

And so the week’s question, which an LCMS reader will recognize and a skeptical one should still find sharpening: Where is your treasure, and what does it cost to keep it elsewhere? “The cloud is just someone else’s computer” is a joke until you notice it is also a doctrine of the heart. What you keep on someone else’s computer — your data, your models, your dependence — reveals what you have decided to treasure and what you have decided to entrust. Jesus’ line in Matthew 6 is not about servers, obviously. But it tells the truth about them: where your treasure is, there your heart will be also. Move your treasure into the cloud and your attention, your money, and your trust follow it there. This week you count that cost with open eyes.

Coach’s Note — The single most useful sentence I can give a graduate cloud admin: quota is not capacity, and a price sheet is not a bill. You will be granted the right to spin up far more than the cloud can actually give you on demand, and you will be quoted prices that bear no resemblance to what you owe at month’s end. Both gaps are where careers get dented. Read on.


12.1 — The Cloud Service Models, Honestly

You know the canonical stack from your professional baseline, so I will state it once, precisely, and move to where the money and the AI live.

  • IaaS (Infrastructure as a Service): you rent compute, storage, and network and run everything above the hypervisor. A GPU VM you SSH into is IaaS. You patch the OS, you leave it running over the weekend, you eat the idle cost.
  • PaaS (Platform as a Service): you rent a managed runtime and deploy code or models into it; the platform owns the OS and scaling. A managed inference endpoint is PaaS-shaped.
  • SaaS (Software as a Service): you rent the finished application. A frontier model’s hosted chat product is SaaS.

A fourth shape now dominates AI: the model API, which behaves like PaaS for billing (you pay per call, the platform scales) but is really a new category — you rent cognition by the token. Most of this chapter lives there.

Layered under all of it are the structural facts you must hold:

ConceptWhat it isWhy the admin cares
RegionA geographic cluster of datacentersData residency, latency, and which models are even available — model availability differs by region
Availability Zone (AZ)Isolated datacenter(s) within a regionSurvive a single-datacenter failure without leaving the region
Shared ResponsibilityThe vendor secures of the cloud; you secure in itMisconfiguring an S3 bucket is your breach, not theirs
Cloud IAMRoles, policies, service principals, scoped keysLeast privilege here is the difference between a leaked key and a leaked company

The shared responsibility model is the one graduate admins most often recite and least often honor. The provider guarantees the hardware, the hypervisor, the physical security, the managed-service control plane. You own identity, configuration, data classification, and — critically for this book — how much you spend. No cloud will stop you from leaving a GPU running. The meter is your responsibility, in the cloud, by contract.

Coach’s Note — Treat every cloud IAM key like a credit card with no limit, because that is functionally what a key with bedrock:InvokeModel or model-deployment rights is. The first cost incident most teams suffer is not an attacker — it is a committed key in a public repo that a bot uses to mine crypto or hammer an expensive model endpoint for a weekend.


12.2 — The Big Four AI Clouds: One Shape, Four Dialects

The administrator’s job is rarely to pick the “best” AI cloud in the abstract. It is to map this workload onto the platform the org already lives in, and to read each platform’s idiosyncratic billing honestly. As of mid-2026, four platforms matter, and the most important thing to learn is that they share one architecture and four incompatible vocabularies.

The common shape, true across all four:

  1. A model catalog — first-party models plus partner models (you can call Anthropic’s Claude on more than one of these clouds).
  2. A pay-per-token serverless default — the on-demand meter, billed separately for input and output tokens.
  3. A reserved-capacity tier — billed by time, in a vendor-specific abstract unit, that only pays off above high sustained volume.
  4. A managed agent runtime — a hosted place to run tool-using agents.

Here is the catalog as of mid-2026 — date everything; these move weekly:

PlatformName note (as of 2026)Reserved unitAgent runtimeNotable catalog
OpenAI PlatformResponses API + Conversations API + Agents SDK; Assistants API sunsets Aug 26, 2026(priority processing tiers)Agents SDKGPT-5.5 / 5.5-pro / 5.4 family
Microsoft FoundryRenamed from Azure AI Foundry Jan 1, 2026 — but “Azure OpenAI” service name persistsPTU ($/PTU/hr)Foundry Agent ServiceOpenAI models + partners
Amazon BedrockAgentCore GA Oct 13, 2025model unit ($/hr; no-commit / 1-mo / 6-mo)AgentCoreClaude, Llama 4, Amazon Nova 2, Mistral, Cohere
Google Vertex AIRebrand to “Gemini Enterprise Agent Platform” announced Apr 22, 2026 — name persists in URLs/SDKsGSU (generative AI scale unit, via burndown)Vertex agentsGemini 3 Pro / Flash, partners

Notice the trap the vendors built on purpose: Azure PTU, Bedrock model unit, and Vertex GSU are three different opaque units for the same thing — reserved throughput. None of them converts cleanly to the others, and that non-convertibility is the lock-in. Once your capacity planning is denominated in PTUs, leaving Azure means re-deriving everything. A graduate admin names this out loud in the architecture memo; a junior one signs the reserved contract because the rep drew a nice graph.

The model API is, mercifully, the one place the industry standardized. The OpenAI-compatible /v1/chat/completions HTTP shape is the lingua franca — on most of these platforms (and every self-hosted engine from Chapter 7) you change the base_url and a key and the same SDK code runs. That portability is your single best hedge against lock-in, and you should architect to preserve it.

Coach’s Note — When someone says “we’re an Azure shop, so we use Azure’s AI,” push once. The right answer is usually “we use the OpenAI-compatible endpoint Azure exposes, written so we could repoint it,” not “we wrote our app against a proprietary SDK we can never leave.” Portability is cheap to keep on day one and ruinous to retrofit on day four hundred.

Two more catalog facts separate a current answer from a stale one, and both are deprecations — the cloud’s quietest cost. OpenAI’s Assistants API hard-sunsets August 26, 2026, migrating to the Responses + Conversations APIs and the Agents SDK; an app built on the old surface is a migration bill waiting to land. And OpenAI announced (May 7, 2026) it is winding down self-serve fine-tuning (last training jobs ~Jan 6, 2027). The lesson is the one from the whole book: architect on a provider’s roadmap, not just its current menu. The model you build on this quarter has a retirement date, and reading the deprecation notice is part of counting the cost.


12.2b — Identity, Region, and Residency: The Non-Dollar Constraints

Before the meter, two structural facts decide more architectures than price ever does.

Cloud IAM is the perimeter now. In the cloud there is no firewall you can stand behind — there is only identity. Every call to a model endpoint, every read of a bucket, every deploy is authorized by a role, a policy, a service principal, or a scoped key. The least-privilege discipline you learned in Chapter 3 is more consequential here, because a single over-scoped key is a company-wide blast radius. A key carrying bedrock:InvokeModel or model-deployment rights is, functionally, an uncapped credit card; the first cost incident most teams suffer is not an attacker breaching a server but a committed key in a public repo that a bot finds and abuses against an expensive endpoint over a weekend. Scope keys to the one model and the one action they need, rotate them, and put a spend cap behind them — least privilege and FinOps are the same control wearing two hats.

Region is not just latency — it gates which models exist. A frontier model GA in us-east may be preview-only, or absent, in the EU region your data-residency rules force you into. So the residency requirement (where the data and the inference are legally allowed to happen) can override the model choice entirely: the best model you may not use is worse than the second-best model you may. For a Christian school holding student records, a relief org holding donor PII, or any org under the EU AI Act, “which region, and what is allowed to leave it” is the first question, asked before the cost question — and it frequently pushes you toward the self-host path of 12.5 regardless of price. Write the residency constraint at the top of the architecture memo, not the bottom.

Coach’s Note — Decide residency before you decide model. I have watched a team fall in love with a model’s benchmark, build against it, and only then discover it wasn’t offered in the region their compliance team required — a rebuild that a single question on day one would have prevented. The data tells you where it may live; the model list narrows to what’s available there; then you count cost.


12.3 — How a Token Becomes a Bill

This is the section that pays for the chapter. Master it.

Serverless model APIs bill per token, input and output priced separately, per million tokens. A token is roughly ¾ of a word in English. The two facts that destroy naive cost models:

  1. Output tokens cost roughly 4–8× input tokens. A model that quotes “$5 / $25 per million” is five times more expensive to talk than to listen. Verbose system prompts are cheap; verbose model answers are not.
  2. Reasoning and agent stacks multiply call count. One user question, in an agentic app, becomes: a planning call, several tool/retrieval calls, a guardrail/moderation call, and a final synthesis call — each billed. The token meter does not bill questions; it bills model invocations, and modern apps invoke a lot.

A snapshot of frontier per-token prices, mid-2026, per 1M tokens (input / output — snapshots, re-verify at the console):

Model (as of 2026)InputOutputCached input
GPT-5.5~$5~$30~$0.50
GPT-5.5-pro~$30~$180
GPT-5.4~$2.50~$15
GPT-5.4-mini~$0.75~$4.50
GPT-5.4-nano~$0.20~$1.25
Claude Opus 4.8~$5~$25
Claude Sonnet 4.6~$3~$15
Claude Haiku 4.5~$1~$5
Gemini 3 Pro~$2~$12
Gemini 3 Flash~$0.50~$3

Three levers cut a serverless bill without touching the architecture:

  • Cached input. Repeated prompt prefixes (a long system prompt, a fixed knowledge block) can bill at a fraction — note GPT-5.5’s $0.50 cached vs. $5 uncached input, a 10× cut on the cached portion. Order your prompts so the stable part is the prefix.
  • Batch. Non-interactive jobs (classify a million documents overnight) commonly take a ~50% batch discount because the vendor schedules them when convenient.
  • The right tier. This is the big one. The model ladder — Haiku/Flash/nano for cheap simple work, Sonnet/mid-tier for balance, Opus/pro for hard agentic reasoning — exists precisely so you can match cost to difficulty. Sending a “classify this support ticket” task to a $30/$180 flagship is the cloud-era equivalent of provisioning a 70B model to answer “what time is it.”

Work one bill by hand, the way you will on the project. A 1,000-token-in / 500-token-out request on Claude Opus 4.8:

input  : 1,000 tokens / 1,000,000 × $5  = $0.0050
output :   500 tokens / 1,000,000 × $25 = $0.0125
total per request                        = $0.0175  (~1.75 cents)

One request is nothing. Now multiply by a million requests a month and you are at $17,500/month — for the same prompt that cost under two cents once. Then notice that the identical request on Haiku 4.5 ($1/$5) costs $0.001 + $0.0025 = $0.0035five times cheaper, $3,500/month. The model-tier decision is, at scale, a five-figure monthly decision. That is the engineering-and-money-are-one-question thesis made arithmetic. The code/token_cost.py calculator does this math for any tier; run it before you ship anything.

Coach’s Note — Always quote AI cost per thousand requests at expected volume, never per request. “1.75 cents” sounds free and ends careers; “$17,500 a month, or $3,500 if we drop to Haiku for this task” starts the conversation that should have happened in design review.


12.4 — Reserved Capacity: The Taxi and the Lease

Serverless per-token billing is a metered taxi — you pay per mile, you pay nothing when parked, and the meter surges (a region/model can “sell out” and throttle you under load). Reserved capacity is leasing a private car by the month — a fixed bill, guaranteed availability, and you pay the same whether you drive it or leave it in the garage.

Each platform sells the lease in its own unit:

PlatformReserved unitCommitment termsThird-party $ range (NOT vendor-confirmed)
Microsoft FoundryPTU (Provisioned Throughput Unit)hourly / monthly / yearly~$6–$60 / PTU / hr
Amazon Bedrockmodel unitno-commit / 1-mo / 6-mo~$21–$50 / hr per model unit
Google Vertex AIGSU (Generative AI Scale Unit)via burndown rates~$42–$158 / GSU-hr

I give those dollar ranges with a loud disclaimer: they are third-party estimates, not confirmed vendor pages, and they vary by model, region, and term. Defer to the live console/calculator for any number you will commit money against. The shape is what you must internalize, not the digits.

The decision rule: reserved only wins above high, sustained volume. A widely-cited third-party rule of thumb (illustrative, not official) is that an Azure reserved tier breaks even somewhere around ~150–200M tokens/month for a mid-tier model — below that the taxi is cheaper; above it the lease is. Three caveats that separate a graduate answer from a glib one:

  1. Quota does not guarantee capacity. You can hold quota for a model in a region and still be told, under real load, that the capacity is sold out. Reserved capacity is partly buying the guarantee, not just the price.
  2. Reserved units denominate your lock-in (12.2). Pricing your roadmap in PTUs makes leaving Azure a re-architecture, not a config change.
  3. Sustained is the operative word. Spiky, business-hours-only traffic wastes most of a 24×7 lease. Match the billing model to the shape of your demand curve, not its peak.

The Interactive Lab below lets you find the crossover point for your own numbers, which is the only crossover point that matters.


12.5 — Self-Host vs. API: The Crossover That Decides the Architecture

Now braid this week back into the whole book. You spent Chapters 4, 6, and 7 learning to own AI infrastructure — size the model, carve the GPU, ship it in a container. This week asks the opposite: should you own it at all, or rent cognition by the token? The honest answer is a crossover curve, and a graduate admin can sketch it.

The API (rent): zero capital, instant scale, vendor handles the GPUs and the on-call. You pay per token, forever, and your data leaves your boundary. Cheap at low and bursty volume; brutal at high sustained volume; a data-residency problem when the data is sensitive.

Self-host (own): you provision GPUs (cloud IaaS or on-prem) and serve with vLLM/SGLang/NIM from Chapter 7. Large fixed cost, near-zero marginal cost per token, full data control. The economics invert exactly opposite the API: brutal at low volume (a mostly-idle H100 is pure waste), excellent at high sustained volume, and the only option when the data must not leave.

The crossover, qualitatively:

FactorFavors APIFavors self-host
VolumeLow, spiky, unpredictableHigh, sustained, predictable
Data sensitivityTolerable to send outMust stay in your boundary
Ops maturitySmall/no GPU teamYou can run vLLM + GPUs at SLA
Model needsWant frontier, always-latestA good open-weight model suffices
CapitalPrefer opex, no commitCan amortize capex

There is a quiet third option that is often the right one for a Christian school or a relief org on a budget: rent the GPU, self-host the model. Spin up a cloud GPU instance (IaaS), run an open-weight model under vLLM on it, and you get data control and no capital outlay — at the price of paying for that GPU 24×7, idle or not. That only beats the API if you keep the GPU busy. Idle GPU-hours are the most common self-host cost mistake there is, the exact mirror of the “forgot a VM over the weekend” sin from 12.1.

Make the crossover concrete with an illustrative (re-verify the rates) worked case. Say a workload is 5M requests/month at 1,000 in / 500 out, served well by a small open-weight model.

OptionCost driverIllustrative monthly costNotes
API, Haiku-class ($1/$5)per-token~$17,500(0.001 + 0.0025) × 5M — scales linearly with volume, no capital
Rent-GPU self-host, kept busyone GPU 24×7 @ ~$4/hr~$2,900~$4 × 730 hrif you keep it saturated; data stays in-boundary
Rent-GPU self-host, 20% utilizedsame GPU, mostly idle~$2,900same bill, one-fifth the work — the idle-GPU trap

The arithmetic is brutal and clarifying: at high sustained volume a busy self-hosted GPU can undercut the API several-fold — but the same GPU left mostly idle costs the same and serves a fifth as much, which is how a “cheaper” self-host quietly becomes more expensive than the API it replaced. The crossover is real, and it lives at utilization, not at the sticker price. Compute yours in the Interactive Lab before you commit a dollar of capital.

Coach’s Note — The crossover is never only dollars. I have recommended the more expensive option many times because the cheaper one would have shipped a school’s student records to a third party, or because the team that would have to run the GPUs at 2 a.m. did not exist. Put the non-dollar columns in the memo. The cheapest correct architecture is the one you can actually operate and are actually allowed to run.


12.6 — FinOps: Watching the Meter Before It Bites

You do not get to find out what AI cost at the end of the month. FinOps — the practice of operationalizing cloud spend as an engineering concern — gives you the controls to know, attribute, and cap it as it happens. Four controls every cloud AI admin must wire up on day one:

1. Tagging. Every resource and, where supported, every API key carries cost-allocation tags (team, project, env). Untagged spend is unaccountable spend — you literally cannot tell whose chatbot cost $9,000. Enforce tagging in policy, not in good intentions.

2. Budgets and alerts. Set a monthly budget per project with alert thresholds (e.g., 50% / 80% / 100% of expected). The alert at 80% on day 12 of the month is the single highest-leverage control you can configure — it turns a month-end surprise into a mid-month decision.

3. Spend caps / hard limits. Where the platform supports it, a hard cap that throttles or stops spend protects against the runaway loop — the agent stuck in a retry storm, the leaked key being abused. Budgets warn; caps stop. Use both.

4. Anomaly detection. A baseline of normal daily spend plus an alert on deviation catches the cost spike the way Chapter 5’s behavioral baseline caught the traffic spike — same idea, the meter instead of the wire.

The code/finops_alerts.sh reference script sketches a daily spend pull, a per-tag breakdown, and a threshold alert you can adapt to any provider’s billing export.

Coach’s Note — Wire the 80% budget alert before you deploy the workload, not after the first scary bill. I have never once regretted a budget alert. I have several times regretted not setting one.


12.7 — AI Watching the Meter: FinOps Copilots and Their Limits

Here is the tool-side of the dual-AI thread, turned on the bill itself. The 2026 cloud consoles ship AI-assisted cost optimization: copilots that read your billing and utilization telemetry and recommend right-sizing an over-provisioned instance, buying reserved capacity for a steady workload, killing idle resources, or moving a workload to a cheaper region or model tier. Used well, a FinOps copilot does the tedious cross-referencing — which of my 200 instances are under 10% utilized and have been for two weeks — far faster than you would by hand.

And here is the same caution this book repeats every week, because it is always true: the copilot recommends; you decide. A cost recommendation is a confident, fast, sometimes-wrong partner like every other AI in this book. “Downsize this instance to save $400/month” is excellent advice unless that instance is your warm DR standby (Chapter 11) sized for failover load it only uses during a disaster — in which case the copilot just recommended you cripple your recovery to save lunch money. The copilot sees utilization. It does not see why. The “why” is your vocation.

The human stays in the loop where the judgment lives. The FinOps copilot can flag every idle resource in seconds. It cannot know which idleness is waste and which idleness is insurance. You own that distinction, and you own the consequence of getting it wrong.

The same goes for the AI-on-the-bill’s mirror image — the AI bill itself. This is where you turn the FinOps discipline back onto AI as a workload. The most volatile line item in a 2026 cloud bill is increasingly the AI calls, and the AI consoles meter in their own units the same way reserved capacity does. Recall from the observability world (Chapter 9) that even operating an AI ops agent now bills in abstract consumption units — Azure’s SRE Agent in Azure Agent Units (AAU), the separate Observability Agent in Azure Agent Credit (AAC) — and that a single deep AI investigation can be capped at hundreds of those credits. Do not conflate the two units; that confusion has put real overage on real invoices. FinOps for AI means budgeting tokens, GPU-hours, and these agent-consumption units, each with its own cap.


12.8 — Fine-Tuning, Provisioned Throughput, and the Costs Nobody Forecasts

Two AI-specific cost surfaces blindside teams, so forecast them explicitly.

Fine-tuning has three distinct cost phases, and people only budget the first: the training cost (per-token over your dataset, sometimes per GPU-hour), the hosting cost (a fine-tuned model frequently requires provisioned/reserved capacity to serve — you cannot always get cheap serverless rates for your custom model), and the lifecycle cost (when the base model is deprecated, your fine-tune may become inference-only or need re-training — a real, recurring bill). Note the platform churn here, too: OpenAI announced (May 7, 2026) it is winding down self-serve fine-tuning, with last training jobs around Jan 6, 2027. Architect a fine-tuning strategy on a provider’s roadmap, not just its current menu.

Provisioned throughput for your own models is the reserved-capacity story (12.4) applied to a model you customized: you pay for guaranteed tokens-per-minute whether you use them or not. Forecast the utilization, not the peak.

And the perennial one, the IaaS sin that predates AI and now costs more than ever: idle GPU instances and forgotten resources. A cloud H100-class instance left running does not care that you stopped using it. Egress — the cost of moving data out of the cloud — is the other silent line item; pulling a large model or a vector store back out of a provider can cost more than you expect, and it is a lock-in mechanism as much as a fee. Tag it, cap it, and schedule it to stop.


12.x — Interactive Lab: Cloud AI Cost Estimator

Below this chapter on the site is the Cloud AI Cost Estimator — your hands-on instrument for everything in 12.3 through 12.5. Do not skip it. Reading the cost math is not the same as feeling the crossover move under your hands.

Use it like this. First, pick a workload — a monthly token volume at a model tier (say, 1M requests/month at 1,000 in / 500 out), a block of GPU-hours, or a fine-tuning job. Watch the estimated monthly cost update, and read the provider comparison beside it: the same workload priced on OpenAI, Microsoft Foundry, Bedrock, and Vertex tiers, so you can see — concretely — that the model is the same and the bill is the vendor. Then do the experiment that teaches the most: hold the workload fixed and drop the model tier from a flagship to a mid-tier to a small model, and watch the five-figure monthly number collapse. That collapse is the single most important intuition a cloud AI admin owns.

Second, find your crossover points. Push the volume up and watch the estimator show where reserved capacity overtakes serverless per-token (the taxi-vs-lease line from 12.4), and where self-hosting a GPU overtakes the API (the curve from 12.5). The exact crossover depends on your numbers — your volume, your tier, your duty cycle — which is the entire point: there is no universal “the cloud is cheaper” or “self-hosting is cheaper,” only a crossover you compute. The estimator teaches you to compute it before you commit, which is what counting the cost actually means.

Bring one real or realistic workload from your own world — your org, your project, the scenario in Project 12 — and estimate it three ways before you read another line. The widget rewards a real number more than a toy one.


12.9 — The Steward and the Treasure Elsewhere

“The cloud is just someone else’s computer.” Sit with that joke long enough and it stops being a joke. It is a statement about where your treasure is.

Jesus says, “For where your treasure is, there your heart will be also” (Matthew 6:21, ESV). He is teaching about ultimate things — about the difference between treasure that moth and rust destroy and treasure that endures. But the line tells the plain truth about cloud architecture too, and the truth runs in the direction you might not expect: you do not move your treasure to where your heart is; your heart moves to where you put your treasure. Put your organization’s data, its models, its operational dependence onto someone else’s computer, and your attention, your money, and your trust follow them there. The architectural decision is, quietly, a decision about the heart’s allegiance — what you will depend on, what you will entrust, and to whom.

This is why “count the cost” (the discipline this whole book inherits from Luke 14:28) is not only arithmetic. The dollar cost is real and you must master it — that is most of this chapter. But there is a second ledger the spreadsheet does not show: the cost of dependence. What does it cost to keep your treasure elsewhere? It costs the egress fee, yes. It also costs a measure of sovereignty — the vendor’s lock-in unit, the vendor’s deprecation schedule, the vendor’s region that can sell out under load, the vendor’s terms about your data. A faithful steward counts both ledgers. The LCMS tradition calls this stewardship and vocation: you are entrusted with resources you did not create — the org’s money, its data, the people whose records live in your buckets — and “it is required of stewards that they be found faithful” (1 Corinthians 4:2, ESV). Faithfulness here is not asceticism — the cloud is often exactly right, and refusing a good tool out of false piety is its own failure of stewardship. Faithfulness is counting honestly: naming the lock-in, sizing the dependence, choosing with open eyes, and being able to give an account of why you put the treasure where you put it.

The skeptic at the next desk does not need to share the theology to use the lesson. Strip the language and the engineering is identical: every cloud decision is a treasure decision, so count both ledgers — the bill and the dependence — before you move your heart. That is the sharpened point. The believer holds it with a particular gravity, knowing the resources are entrusted and the accounting is owed to Someone. But the discipline is sound for anyone signing the invoice.


12.10 — Common Pitfalls

Pitfall: Quoting AI cost per request instead of per thousand-at-volume. Example: “It’s only 1.75 cents a call” — which is $17,500/month at a million calls. Fix: Always project to expected monthly volume and present the monthly figure with the tier alternative beside it.


Pitfall: Treating quota as capacity. Example: You hold quota for a frontier model in a region, deploy, and get throttled under load because the region sold out. Fix: For guaranteed availability under load, buy reserved capacity; treat serverless quota as best-effort and test under realistic load before you depend on it.


Pitfall: Conflating the vendors’ abstract billing units. Example: Sizing a workload in Azure PTUs and assuming Bedrock model units or Vertex GSUs convert at the same rate — or confusing Azure’s AAU with its AAC. Fix: Treat each unit as platform-specific and non-convertible; re-derive capacity per platform from a tokens-per-minute target, and never mix the units on one spreadsheet.


Pitfall: Shipping every task on the flagship model. Example: Routing “classify this support ticket” to a $30/$180 reasoning model. Fix: Match model tier to task difficulty; route simple work to the cheap tier (Haiku/Flash/nano), reserve the flagship for genuinely hard reasoning, and measure quality at each tier before deciding.


Pitfall: Forgetting idle resources — the GPU left running, the data egressing. Example: A “quick test” GPU instance runs all weekend; a model pull racks up egress on the way out. Fix: Tag everything, set budget alerts and hard caps, schedule non-prod resources to auto-stop, and forecast egress before you commit to a provider.


Pitfall: Trusting a FinOps copilot’s right-sizing recommendation blindly. Example: Accepting “downsize this idle instance” when the instance is your warm DR standby sized for failover. Fix: Treat every cost recommendation as a hypothesis; verify why a resource is idle before acting; the copilot sees utilization, you see purpose.


Pitfall: Architecting an AI app against a proprietary SDK you can never leave. Example: Writing directly to one vendor’s bespoke API so that switching providers is a rewrite. Fix: Code against the OpenAI-compatible /v1 endpoint where possible so base_url is the only thing that changes; keep portability as a day-one requirement.


12.11 — Reps

The reps for this week live in the exercises. Type every command yourself; do the cost math by hand before you run any calculator, because the whole skill is predicting the bill, not reading it after the fact. A preview of what’s waiting:

  • Hand-compute a serverless bill for a real prompt across three model tiers and project it to monthly volume.
  • Find the reserved-vs-serverless crossover for a given workload using the rule-of-thumb and the estimator.
  • Sketch the self-host-vs-API crossover for a sensitive-data workload and write the recommendation.
  • Wire FinOps controls — tags, a budget, an 80% alert, a hard cap — from code/finops_alerts.sh.
  • Decode the abstract units — convert one tokens-per-minute target into PTU, model unit, and GSU, and name the lock-in.

AI policy for the reps (Phase 2, Week 12): AI is part of the work this week, and increasingly agentic — but it cannot finish these for you, because the judgment is the point. You may use an AI copilot to draft a billing query or a cost script; you must do the cost arithmetic and the crossover reasoning yourself first, verify anything the AI tells you about prices against a live console (prices in this chapter are mid-2026 snapshots and will have drifted), and keep an honest agent-log.txt of what you delegated and where it was wrong. The human owns the verdict.

A short Check Your Reps quiz sits at the bottom of this page on the site — five questions to confirm the concepts landed before you spend real money.


12.12 — This Week’s Project

This week’s project is P12 — “Deploy in the Cloud, Read the Bill,” specified in Project 12. You will deploy a real AI workload to a cloud platform, instrument it with FinOps controls, drive measured traffic through it, and then — the actual deliverable — read the bill and write the analysis a steward could defend.

At a high level: Normal tier deploys a serverless model-API workload, tags it, sets a budget and alert, runs a fixed request volume, and reconciles the predicted cost against the actual bill. Medium tier adds a provider comparison (the same workload priced across at least two of the big-four catalogs) and a reserved-vs-serverless crossover analysis for the workload’s projected volume. Hard tier demands the judgment an agent cannot supply: a build-vs-buy / self-host-vs-API recommendation memo for a sensitive-data scenario, weighing dollars against data residency, lock-in, and operability — a defensible architecture call with a number and a reason, signed by you.

It is Phase 2, so an agent-log.txt is required: where you used AI, where it was confidently wrong about a price or a unit, and where you overrode it.


12.13 — Coach’s Final Word

This week you learned that in the cloud the engineering question and the money question are one question, and you learned to answer it like an administrator instead of a tourist: per-token vs. reserved, serverless vs. self-host, the right tier for the task, the budget alert wired before the workload ships. You learned that the vendors invented their units to be hard to compare, that quota is not capacity, that a price sheet is not a bill, and that a FinOps copilot is one more confident, fast, sometimes-wrong partner whose recommendations you verify and own.

But the thing I most want you to carry out of Week 12 is the second ledger. The spreadsheet shows the dollars. It does not show the dependence — the lock-in unit, the deprecation schedule, the data that left your boundary, the heart that followed your treasure to someone else’s computer. “For where your treasure is, there your heart will be also.” The cloud is a good and often-right tool, and refusing it out of false piety is no virtue. Faithfulness is counting both ledgers with open eyes and being able to give an honest account of why you put the treasure where you put it. That is stewardship. That is the job.

Read the reps, ship the project, and read the bill like it tells you something about your heart — because it does. The administrator wields the cloud; the administrator does not let the cloud quietly own the org’s treasure without anyone counting the cost.

See you next week.


Up next: Read the exercises and do every rep with the meter in mind, then build Project 12 — Project 12: Deploy in the Cloud, Read the Bill. Lean on Appendix A for the lab environment, Appendix B for local-vs-cloud AI, and Appendix C for the agentic-AI rules your agent-log.txt must honor. When the bill is read, go on to Chapter 13 — DevOps, SRE, and Work That Lasts. Previous week: Chapter 11.

Interactive Lab — Week 12
Cloud AI Cost Estimator

The same model can be billed three ways. Estimate the monthly cost of calling a hosted API, renting your own GPU, or fine-tuning — then see where the lines cross.

Monthly API cost $0
Crossover:

⚠ All figures approximate, as of 2026 — verify current vendor pricing before you budget.

Try: On API tokens, push requests/month up until the crossover note flips to "a dedicated GPU is now cheaper." That volume is where owning your inference starts to pay off.
Check Your Reps

Check Your Reps: Cloud Systems Administration and Counting the Cost

Question 1 of 5
In serverless model-API billing, roughly how do output tokens compare to input tokens in cost?
Why: Output tokens are billed at roughly 4-8x the input rate, which is why capping output length and asking for concise answers cuts the bill.
Question 2 of 5
Azure PTU, Amazon Bedrock model units, and Google Vertex GSU are best described as:
Why: Each platform invented its own opaque reserved-capacity unit that does not convert cleanly to the others, and that non-convertibility is precisely how the lock-in works.
Question 3 of 5
A FinOps copilot recommends downsizing an idle instance to save money. What is the correct administrator response?
Why: The copilot sees utilization but not purpose; an idle resource may be a warm DR standby, so the human verifies why before acting and owns the consequence.
Question 4 of 5
According to the chapter, reserved capacity (vs. serverless per-token) generally wins when:
Why: Reserved capacity is like leasing a car by the month: it pays off only above high, sustained volume, and it also buys a capacity guarantee that serverless quota does not provide.
Question 5 of 5
For a workload handling sensitive data (e.g., donor PII or student records), which factor most strongly pushes the architecture away from a third-party model API?
Why: Data residency is a non-dollar constraint that can override price and model choice, often pushing toward self-hosting so sensitive data never leaves the organization's boundary.
YOU FINISHED. NICE WORK.