Chapter 10 — Security Operations: The Adversary Who Disguises Himself

“Amateurs hack systems; professionals hack people.” — Bruce Schneier

“Be sober-minded; be watchful. Your adversary the devil prowls around like a roaring lion, seeking someone to devour.” — 1 Peter 5:8 (ESV)

Why This Matters

For nine weeks you have built things. You sized storage, carved GPUs, shipped a model in a box, generated infrastructure and verified it, and last week you learned to keep watch over a running system. This week the system has an enemy, and the enemy is using your own best tool against you.

Here is the shape of 2026. The same large language model that drafts your runbooks drafts the attacker’s spear-phishing email — and it drafts it without the spelling mistakes and the broken grammar that used to be the tell. The same agent that can read your logs and propose a root cause can be turned, by a single crafted email, into an exfiltration tool that walks your data out the door with no human clicking anything. That is not a hypothetical. In June 2025 a vulnerability called EchoLeak (CVE-2025-32711, CVSS 9.3) did exactly that to Microsoft 365 Copilot — the first widely reported zero-click prompt-injection in a production AI product. One email. No click. Data gone. Microsoft patched it server-side and there was no exploitation in the wild, but the lesson stands like a road sign: the AI you deployed to help is now part of your attack surface.

So this week’s dual-AI thread is unusually sharp, because both threads are weapons. AI as the tool you wield is AI as defender — ML-driven detection, behavioral analytics, automated response (SOAR), Microsoft Security Copilot reasoning over your alerts. AI as the workload you govern is, this week, AI as attacker and AI as target: phishing and malware and reconnaissance at machine scale on one side, and on the other the new class of vulnerabilities that exist only because you put an LLM in production — the OWASP Top 10 for LLM Applications, MITRE ATLAS, prompt injection, data poisoning, model theft. You have to administer both sides of that line, and the line moves.

The discipline that holds it together is old and unglamorous: security operations. A SOC, a SIEM, detection engineering, incident response, the kill chain, threat intel. None of that is new. What is new is that every one of those functions now has an AI accelerant bolted to it — and a confident, fast, sometimes-wrong AI accelerant on a security team is powerful and dangerous in exactly equal measure. An AI that auto-contains the wrong host has just caused your outage. An AI that summarizes an incident wrong has just sent your responders down a false trail at 3 a.m. The human stays in the loop where the judgment lives. This week, more than any other, the loop is where you live or die.

Coach’s Note — Every other chapter, AI is a productivity story with a footnote about risk. This chapter, AI is the threat model. Read it that way. The attacker has the same copilot you do, runs it cheaper, and does not have a change-management process.

And here is the week’s question, and it is a serious one. Scripture does not describe the enemy as a brute. It describes him as a deceiver: “your adversary the devil prowls around like a roaring lion” (1 Peter 5:8, ESV), and elsewhere, “even Satan disguises himself as an angel of light” (2 Corinthians 11:14, ESV). The defining feature of the adversary is not strength. It is disguise. That is precisely the security problem AI has handed you — the malicious email that looks legitimate, the synthetic voice that sounds like your CFO, the helpful document that hides an instruction to your agent. How do you stand against an enemy who disguises himself? Hold that question. We will answer it in engineering, and then in earnest.

10.1 — Security Operations: The Function You’re Defending With

Before the AI, the function. A Security Operations Center (SOC) is the team and the tooling that detects, investigates, and responds to threats against your systems. You do not need a glass room with wall monitors to have a SOC; a two-person ministry IT shop with a good SIEM and a written incident-response plan is a SOC. The job decomposes into a handful of disciplines you must be able to name:

Two mental models do most of the work. The first is the kill chain — the idea that an intrusion is a sequence, not an event: reconnaissance, weaponization, delivery, exploitation, installation, command-and-control, and actions-on-objectives (the Lockheed Martin formulation). You defend a kill chain by breaking any link; you do not have to catch the first move to win. The second is MITRE ATT&CK, the encyclopedic catalog of the tactics and techniques (TTPs) real adversaries use, which gives your detection engineers a shared vocabulary: not “something weird happened on db1” but “T1078 Valid Accounts followed by T1048 Exfiltration Over Alternative Protocol.”

Look at code/auth.log. It is sixteen lines of authentication events with one real attack chain buried in routine traffic. Map it to the kill chain yourself before you read further: the impossible-travel login is delivery/exploitation (T1078, valid stolen account), the sudo with no change ticket is privilege escalation, the lateral SSH to db1 is lateral movement (T1021), the bulk SELECT is collection, and the curl to an external IP is exfiltration (T1048). That is a textbook kill chain, and you broke it in your head in thirty seconds because the dataset is small. The whole problem of security operations is that in production the dataset is not sixteen lines. It is sixteen million.

Detection engineering is the discipline that bridges that gap. A detection is a rule — a query, a Sigma signature, a YARA pattern — that turns the firehose of events into the trickle of alerts worth a human’s attention. Good detections are written, version-controlled, tested against known-bad samples, and tuned against known-good traffic exactly like application code; the mature SOC treats detections as a codebase with a CI pipeline, not as a pile of clicked-together console rules. Threat intelligence feeds that pipeline: indicators of compromise (a malicious IP, a file hash) are the perishable, low-level signal, and TTPs — how an adversary operates — are the durable, high-level signal. You want detections written at the TTP level, because an attacker swaps an IP in an afternoon but changes their playbook over months. That is the whole reason ATT&CK exists: it lets you detect the behavior (T1078 valid accounts, T1021 lateral movement) rather than chasing the disposable indicator.

And the spine of all of it is incident response — the rehearsed sequence you run when an alert turns out to be real: prepare (write the plan and the contacts before you need them), detect and analyze (confirm it’s real and scope it), contain (stop the bleeding without destroying the evidence), eradicate (remove the foothold), recover (restore service), and learn (the blameless postmortem). The order matters, and the discipline matters more than the tooling. A two-person shop with a rehearsed IR plan beats a glass-walled SOC that has never run the drill.

10.2 — Detection Without AI: Build the Baseline First

This is a Phase 1 instinct carried into Phase 2: do it by hand first, then judge the AI. Before you let any model triage that log, you write the dumb, explainable rule engine yourself. code/triage_logins.py is exactly that — five hand-written rules (failed-auth bursts, logins from outside the home geography, off-hours privilege escalation, sudo without a change ticket, egress to a hostile IP). Run it:

python3 code/triage_logins.py code/auth.log

It flags the chain. It also produces false positives and it cannot explain intent — a sudo without a ticket might be a real emergency, and the rule does not know. That is the point. A deterministic detector is precise, auditable, and blind. It catches exactly what you told it to catch and nothing else, and it never hallucinates a finding — but it never notices the attack pattern you did not anticipate. Hold that property in your mind. It is the exact complement of what the AI gives you, and the complement is the architecture.

This is why the layered SOC is not “AI instead of rules” but “AI and rules,” each covering the other’s blind spot. The deterministic layer is your known-knowns: the attacks you have seen, written down as rules that fire forever, fast, cheap, and explainable in a courtroom. The ML/AI layer is your unknown-unknowns: the novel behavior no one wrote a rule for, surfaced as a hypothesis for a human to investigate. A rule you can read is a rule you can defend to an auditor and tune with confidence; an anomaly score you cannot read is a lead you must verify before you trust. Build the readable layer first — not only because it’s good practice, but because it is the yardstick against which you will grade everything the AI tells you. You cannot judge a detector you have never tried to write.

Coach’s Note — A detection rule you wrote by hand is a contract: this pattern, and only this pattern, is an alert. When the AI flags something your rules missed, that is a candidate for a new rule. The AI is a hypothesis generator; your rule engine is where the verified judgment gets written down so it fires deterministically forever after.

10.3 — AI as Attacker: The Adversary’s New Toolkit

Now the workload you must reckon with — because the attacker is running it. As of 2026, four things have changed, and you must understand each as an administrator, not as a headline.

Phishing at human-expert quality, at machine scale and cost. The historical tells of phishing — bad grammar, generic greetings, clumsy urgency — are gone. An LLM writes a grammatically perfect, context-aware, personalized lure for fractions of a cent, and writes ten thousand of them before lunch. The economics inverted: the expensive part of spear-phishing used to be the craft; now the craft is free and the only cost is the target list.

Deepfake-enabled social engineering. Schneier’s epigraph — “professionals hack people” — is now industrialized. The canonical case is January 2024, Hong Kong: a finance worker at the engineering firm Arup paid out roughly US$25.6 million across about fifteen transfers after a video call in which every “colleague,” including a fake CFO, was a deepfake (reported by CNN and CFO.com). Voice cloning trivially defeats caller-ID and voiceprint “authentication.” Write this on the wall: recognizing a voice or a face is no longer an authentication factor.

Reconnaissance and exploitation at scale. AI agents can enumerate your attack surface, summarize a target’s public footprint, and draft working exploit code faster than a human analyst. “Malicious LLMs” sold as a service — GhostGPT, Xanthorox (first seen early 2025) — package this for buyers; the earlier WormGPT/FraudGPT brands are now defunct, but the category is permanent.

The AI you deployed is itself a target. This is the part that is genuinely new to administration, and it has its own catalog. We turn to it next.

Sit with what the economics do to your threat model. Defense has always been asymmetric — the defender must close every hole; the attacker needs one. AI widens that asymmetry by collapsing the attacker’s cost and skill floor. A lone actor with a malicious-LLM subscription now drafts convincing lures in a dozen languages, summarizes your org chart from public sources, and iterates an exploit faster than your patch cycle — capabilities that used to require a team. The defensive AI you are about to deploy is the only thing that rebalances the equation, which is exactly why the rest of this chapter is about wielding it and governing it. You cannot opt out of the AI arms race by declining to use AI. You can only choose to be the side that uses it carelessly or the side that uses it soberly.

Coach’s Note — The dead tell was the typo. For twenty years we trained users to “spot the bad grammar.” That training is now actively harmful — it teaches people that a well-written message is trustworthy, which is exactly backwards in 2026. Retrain the reflex: the question is never “does this look professional?” It is “does this ask me to bypass a control, act without a callback, or trust an appearance?” Polish is not provenance.

10.4 — Securing the AI Itself: The OWASP Top 10 for LLM Applications

When you put an LLM in production, you added a component with failure modes no firewall rule anticipates. The OWASP Top 10 for LLM Applications 2025 (from the OWASP GenAI Security Project, released November 2024 and still current as of June 2026) is the canonical list. Learn all ten by their IDs; in a security meeting you will be expected to speak them fluently.

Three of these deserve to live in your bones, because together they form the dominant real-world risk.

LLM01 — Prompt Injection is the one with two flavors. Direct injection is a user typing “ignore your instructions” into the chat box. Indirect injection is the dangerous one: the malicious instruction lives in content the model retrieves — a web page, an email, a document in your RAG store — and fires when the model reads it, with no malicious user present. Open code/rag_notes.txt. It is a help-desk knowledge-base note that looks entirely benign, and it contains a hidden instruction telling the assistant to export the donor table to an attacker’s URL. A naive agent retrieving that note follows it. That is indirect prompt injection, and it is architecturally unsolvable within current LLM design — the model cannot reliably tell its instructions apart from the data it reads, because to the model they are both just tokens.

LLM05 — Improper Output Handling and LLM06 — Excessive Agency are the multipliers. Injection by itself is a parlor trick; it becomes a breach when (a) you pipe the model’s output straight into a shell, a database, or an apply step without validation (LLM05), and (b) you gave the model a tool with enough permission to do real harm — write access, the ability to delete infrastructure, an outbound HTTP call (LLM06). The fix for LLM05 is classic output validation/encoding. The fix for LLM06 is classic least privilege.

Coach’s Note — Prompt injection is the SQL injection of the AI era. You did not fix SQL injection with a smarter database; you fixed it by never trusting input as code — parameterized queries, validated output. Same move here. There is no filter that makes an LLM safe to read untrusted content and act with full privilege. The fix is architectural: trust boundaries, least privilege, output verification, red teaming.

10.5 — The Lethal Trifecta, EchoLeak, and MITRE ATLAS

The cleanest way to reason about whether an AI deployment is dangerous is the “lethal trifecta” (named by Simon Willison): an AI system is exploitable for data theft when it combines (1) access to private data + (2) exposure to untrusted content + (3) an exfiltration vector (a way to send data out). Hold any two and you are probably fine. Hold all three and a single piece of crafted content can walk your data out the door. Meta’s “Agents Rule of Two” says the same thing prescriptively: an unsupervised agent should satisfy at most two of {untrusted input, sensitive-data access, ability to change state / communicate externally}. Need all three? Then a human must be in the loop.

EchoLeak is the trifecta made real. Trace the kill chain and label each step:

1. Delivery — an attacker sends a crafted email to a Microsoft 365 Copilot user (untrusted content: trifecta leg 2).
2. Ingest — Copilot retrieves the email into context (it already has private-data access: leg 1).
3. Injection — the email’s hidden instruction bypasses the XPIA cross-prompt-injection classifier using reference-style Markdown (LLM01).
4. Exfiltration — a reference-style Markdown image auto-fetches a URL carrying the data, routed through a Teams proxy the content-security-policy allowed (leg 3).

Zero clicks. All three legs of the trifecta present. In MITRE ATLAS — the adversarial-ML companion to ATT&CK, organized as AML.T#### techniques — this maps to LLM prompt-injection and exfiltration techniques. ATLAS is your second framework this week: as of early 2026 it carried roughly 16 tactics, ~84 techniques, and ~42 case studies, and in v5.2.0 (January 30, 2026) it added a block of agentic techniques (AML.T0096–T0101). It moved to date-based versioning with v2026.05 (May 27, 2026). The counts shift every release — quote the live figure from atlas.mitre.org, not a number you memorized. ATLAS is the operational catalog of attacks; pair it with NIST AI 100-2e2025 (March 24, 2025), which is the adversarial-ML taxonomy. Catalog and taxonomy — do not conflate them.

10.6 — Poisoning the Well: Supply Chain, Data, and Embedding Attacks

Prompt injection is the loud risk. The quiet ones are worse, because they corrupt the model before it ever sees a hostile prompt, and they are squarely the administrator’s problem.

LLM03 — Supply Chain. Your AI system is assembled from parts you did not build: a base model pulled from a hub, a fine-tune dataset, a vector-store extension, a plugin. Any of them can arrive compromised — a backdoored model that behaves normally until a trigger phrase appears, a dependency that exfiltrates on import, a “helpful” community model card that lies about provenance. The countermeasure is the one you already know from package management: pin versions, verify signatures and hashes, prefer signed artifacts from registries you control, and maintain an AIBOM (an AI bill of materials — CycloneDX ML-BOM or SPDX) so you can answer “what is actually inside this system?” when the next backdoored-model advisory drops.

LLM04 — Data and Model Poisoning. If an attacker can influence your training, fine-tuning, or — most accessibly — your RAG corpus, they can bend the model’s behavior without ever touching the model weights. The accessible version for most organizations is retrieval poisoning: plant a document in the knowledge base the assistant retrieves from, and you have changed its answers for everyone. That is LLM08 — Vector and Embedding Weaknesses in its operational form, and it is why code/rag_notes.txt is dangerous twice over: it is an injection payload and an example of corpus poisoning. The administrator’s controls are access control on who can write to the corpus, provenance metadata on every document, and review before ingestion — the same hygiene you’d apply to any privileged write path.

Coach’s Note — Notice the pattern across LLM03/04/08: the attack is not against the model’s reasoning, it is against the model’s inputs — its training data, its dependencies, its retrieval store. You secure those the way you secure any data supply chain: control the write path, verify the source, inventory what you shipped. Old skills, new asset class.

10.7 — AI as Defender: ML Detection, UEBA, and Governed SOAR

Now the tool in your hand. The defensive AI thread has three layers, each older than the LLM hype and each now supercharged.

ML-based anomaly detection replaces static thresholds with learned baselines. The most important instance for an administrator is UEBA — User and Entity Behavior Analytics — which in 2026 has shifted from “alert if logins > N” to deep-learning, per-identity-and-device baselines that flag anomalous action sequences: impossible-travel logins, a sudden burst of file access, the rapid mass-read that means lateral movement. A 2026 theme worth your attention: UEBA is being extended to cover AI agents themselves as non-human identities, because an agent with credentials is an entity whose behavior you must baseline like any other.

SOAR — Security Orchestration, Automation, and Response — is where AI takes action, and therefore where governance matters most. Look at code/soar_playbook.yml. It encodes the one rule that keeps automated response from becoming an automated outage: reversible containment may be automated; destructive or external actions require a human gate. Revoking a session and blocking an IP for an hour are reversible — let the machine do them at 3 a.m. Disabling a user account or notifying donors is high-impact — those wait behind an approval_gate with a named approver and a timeout that escalates rather than auto-approves. That is OWASP LLM06 (Excessive Agency) answered in YAML, and it is EU AI Act Article 14 (human oversight) answered in YAML. Same idea, two vocabularies.

Here is UEBA made concrete on the data you already have. The legitimate jvargas in code/auth.log logs in from US-IL in the morning and never touches db1. At 13:22 an actor using jvargas’s credentials logs in from Romania, escalates privilege without a ticket, SSHes to db1, and bulk-reads the donor table. A threshold rule asks “more than N failures?” and might catch the spray. A UEBA model asks a different and better question: does this sequence of actions match this identity’s learned baseline? — and the answer is a screaming no, because jvargas has never, in ninety days, logged in from Europe, escalated without a ticket, or queried the donor table. The value of the learned baseline is that it flags the novelty of the behavior, not the crossing of a fixed threshold an attacker can stay under. The cost — and there is always a cost — is that it will also flag jvargas’s first legitimate business trip to Europe, and you must decide what that false positive is worth.

The AI SOC assistant is the natural-language layer over all of it. Microsoft Security Copilot (GA April 1, 2024) reasons over your alerts, drafts incident timelines, and answers “what is the blast radius of this account compromise?” in English. It is billed in Security Compute Units (SCUs) — as of 2026, roughly $4/hour provisioned and $6/hour overage, with Microsoft 365 E5 customers getting an allocation (about 400 SCUs/month per 1,000 paid licenses, capped, phasing in mid-2026). Treat every SCU figure and every “sub-100ms autonomous containment” vendor claim as a snapshot to verify, not a fact to teach. Microsoft Agent 365 (GA May 1, 2026, ~$15/user/month) is the adjacent governance plane — it inventories and secures AI agents across Entra/Defender/Intune, and can even import agents from AWS Bedrock and Google Gemini Enterprise.

Coach’s Note — The Security Copilot incident summary is a first draft of a hypothesis, exactly like the SRE agent’s root cause from Chapter 9. It is fast, it is usually directionally right, and it will occasionally invent a confident, wrong narrative that sends your responders down a rabbit hole. You verify before you act. The AI compresses the triage; it does not own the decision to disable an account.

Here is the defender’s stack as a sizing-and-governance table:

10.8 — The False-Positive Economy

One number governs whether your AI defense helps or hurts: the false-positive rate, multiplied by your event volume. An anomaly detector that is “99% accurate” on a million daily events still hands your analysts ten thousand false alarms a day, and a SOC that cries wolf ten thousand times trains its own humans to ignore the alert that matters. This is not a model-quality problem you can buy your way out of; it is an operations problem you must design around — tuning thresholds, suppressing known-benign patterns, and correlating alerts (the Chapter 9 noise-reduction pillar) so that one cascading failure becomes one incident, not five hundred. The AI that floods you is failing exactly as badly as the AI that misses the attack. Alert fatigue is LLM10 (Unbounded Consumption) wearing a human face: the consumed resource is your analyst’s attention.

The math is unforgiving and worth doing once explicitly. Take the base rate: real attacks are rare relative to events. If 1 in 100,000 events is malicious and your detector is 99.9% accurate, the vast majority of its alerts are still false positives, because the false-positive count (0.1% of the millions of benign events) dwarfs the true-positive count (a handful of real ones). This is the base-rate fallacy, and it is why “accuracy” is the wrong metric for a SOC. You care about precision (of the things it flagged, how many were real) and you care about it because every false flag spends a scarce, non-renewable resource: a human’s trust in the system. Design your AI defense to be quiet and right over loud and sensitive, and measure it on the bill it sends to your analysts’ attention, not on a vendor’s accuracy slide.

10.x — Interactive Lab: Phishing Defender Drill

Below this chapter on the site is an interactive panel: the Phishing Defender Drill. Use it now — this is conditioning, not reading.

The drill presents a stream of messages — some legitimate, some AI-generated phishing, and at least one that carries an indirect prompt-injection payload aimed not at you but at an AI assistant that would process the message. Your job is the defender’s job: classify each message as legitimate or malicious before you reveal the answer. Predict first, then check. When you reveal, the drill shows you the tells — the spoofed-but-plausible sender, the urgency-without-recourse, the link that does not match its anchor text — and, critically, the LLM-specific risks: where a message looks harmless to a human but contains an instruction that would hijack an agent reading it.

What the drill teaches is the thing this whole chapter is about: the modern lure has no spelling mistakes. You are not training your eye to spot bad grammar — that tell is dead. You are training your judgment to ask the questions that still work: Does this ask me to bypass an out-of-band verification step? Does it want me to act now, without a callback? If an AI assistant read this, does it contain text shaped like an instruction? Run the drill until you can articulate, for each message, why — not just that — it is or isn’t an attack. The “why” is the muscle. Then carry it into Project 10, where you build both the attacker and the defender yourself.

10.9 — The Adversary Who Disguises Himself

We owe the week’s question a real answer. How do you stand against an enemy who disguises himself?

Notice first what Scripture does not say. It does not say the adversary is strong and you are weak, so despair. It says the adversary is a deceiver — “your adversary the devil prowls around like a roaring lion” (1 Peter 5:8, ESV), and “Satan disguises himself as an angel of light” (2 Corinthians 11:14, ESV). The peril is not raw power. It is counterfeit. The roar is meant to panic you; the disguise is meant to fool you. And the instruction that precedes the warning is not “be strong.” It is “be sober-minded; be watchful.” Clear judgment and steady attention. That is the posture, and it is — precisely — the posture of a security operations center.

This is why the engineering of this chapter is more than analogy. Every defense we built is a defense against counterfeit. Out-of-band verification answers the deepfake CFO because it refuses to trust the appearance — the voice, the face — and demands a channel the impersonator does not control. Phishing-resistant authentication (FIDO2 passkeys) answers the perfect phishing email because it refuses to trust a thing the human can be tricked into typing. The human approval gate in the SOAR playbook answers the hijacked agent because it refuses to trust an actor that looks authorized but cannot be held accountable. In every case the move is the same: do not authenticate on appearance. The lion’s roar and the angel’s light are both appearances. Sobriety is the refusal to be governed by them.

There is a second, harder line in the LCMS register, and it is about your own tools. The most dangerous deception this year is not the attacker’s email. It is your own AI telling you, in fluent and confident prose, an incident summary that is wrong — the angel of light wearing your copilot’s face. The administrator who has surrendered judgment to the machine because the machine is usually right has made himself easy to deceive in exactly the way Scripture warns against, because he has stopped being watchful. Sober-mindedness toward your own instruments is a security control. You verify the AI’s hypothesis not because you distrust the vendor, but because watchfulness is the job, and the moment you outsource it you become the finance worker on the deepfake call — recognizing a familiar face and wiring the money.

And there is a comfort here that is not naïve. The same passage that names the prowling lion ends, a verse later, in confidence: God “will himself restore, confirm, strengthen, and establish you” (1 Peter 5:10, ESV). The vocation of the watchman is not to be the savior of the system — it is to keep faithful watch over what he did not make and cannot ultimately secure, trusting that the keeping does not finally rest on him. That frees you to do the work soberly instead of frantically. You will miss things. You build a recovery plan (next week) precisely because you are not omniscient. Watchfulness is a duty, not a guarantee — and a steward is judged faithful by the watching, not by the impossible promise that nothing will ever get through.

10.10 — Common Pitfalls

Pitfall: Treating the AI’s incident summary as a finding instead of a hypothesis. Example: Security Copilot drafts “root cause: compromised service account svc_bkp,” and the analyst disables it — but the real attacker used jvargas, and now the nightly backups are broken and the breach is still live. Fix: Verify every AI-generated conclusion against the raw events before any action. The summary compresses triage; it does not replace it.

Pitfall: Giving a tool-using AI agent the lethal trifecta. Example: A help-desk agent has read access to the donor database (private data), retrieves a poisoned KB note (untrusted content), and has an outbound HTTP tool (exfiltration). One crafted document — like code/rag_notes.txt — and it ships the data out. Fix: Apply the Rule of Two. Remove one leg: scope the data access, sandbox the retrieval, or remove the outbound vector. If you truly need all three, require a human gate.

Pitfall: Authenticating on appearance — voice, face, or a familiar email tone. Example: A caller who sounds exactly like the CFO authorizes a wire. (This is the Arup case, ~US$25.6M.) Fix: Out-of-band verification: call back on the number on file, use pre-shared code words, require dual approval for wires. Recognition is not authentication.

Pitfall: Letting SOAR automate destructive or external actions. Example: A playbook auto-disables any account flagged for impossible travel; a false positive on a traveling executive locks out the CEO during a board meeting. Fix: Automate only reversible containment (revoke session, time-boxed IP block). Gate disable/delete/notify behind a named human approver — see code/soar_playbook.yml.

Pitfall: Tuning the detector for sensitivity and ignoring the false-positive bill. Example: A new UEBA model is set to maximum sensitivity; it generates 8,000 alerts a day; within a week the team mutes the channel and misses the real one. Fix: Measure false-positive rate × event volume. Correlate and suppress. An alert nobody reads is worse than no alert.

Pitfall: Confusing the frameworks. Example: A report cites “MITRE ATLAS” as the source for an adversarial-ML taxonomy category, or quotes a memorized ATLAS technique count. Fix: ATLAS is the operational catalog (AML.T####, counts shift monthly — read the live figure); NIST AI 100-2e2025 is the taxonomy. OWASP LLM Top 10 is the application risk list. Cite the right one.

10.11 — Reps

Open the exercises for the full set. This is Phase 2: agentic AI is ON for the project, but the reps are still hand-built — you cannot judge an AI defender if you have never hand-written a detection rule or hand-traced an injection. A short Check Your Reps quiz sits at the bottom of this page; take it before you move on.

A preview:

• Rep 1 — Map code/auth.log to the kill chain and to MITRE ATT&CK technique IDs, by hand.
• Rep 2 — Run code/triage_logins.py, separate true from false positives, and write the rule it’s missing.
• Rep 4 — Find the indirect prompt injection in code/rag_notes.txt and explain which trifecta leg removal would neutralize it.
• Rep 6 — Label every step of the EchoLeak kill chain with an OWASP LLM ID and an ATLAS technique.
• Rep 9 — Turn code/soar_playbook.yml into a defensible autonomy policy and justify each gate.

Type every line. Run every script. Predict before you reveal.

10.12 — This Week’s Project

You’re ready for Project 10 — Attacker and Defender, in Project 10.

You will play both sides. As the attacker, you will (in a sandbox, against your own systems only) build an AI-assisted phishing lure and an indirect-prompt-injection payload, and document the kill chain. As the defender, you will build the detection-and-response pipeline that catches it: hand-written rules plus an AI triage layer plus a governed SOAR playbook, with every automated action classified reversible-or-gated. The Medium tier maps your whole scenario to OWASP LLM + MITRE ATLAS + the kill chain. The Hard tier is the architect’s deliverable — a written autonomy-and-trust memo deciding exactly how much your AI defender may do without a human, defended against the lethal trifecta and the Rule of Two. That memo is the judgment an agent cannot supply.

This is Phase 2, so an agent-log.txt is required: every task you delegated, what the agent did, where it was wrong, and where you intervened. The project is shaped so the agent cannot finish it alone — it can draft a playbook, but it cannot decide what your organization is willing to let a machine do unsupervised.

10.13 — Coach’s Final Word

Nine weeks you spent building. This week you learned that everything you build has an enemy, and the enemy has your tools. That is not a reason to despair; it is a reason to be sober. The administrator who understands AI as both the most powerful defender and the most dangerous new attack surface is the one who can actually be trusted with production — because she does not over-trust the copilot that drafts her incident timeline, and she does not under-estimate the copilot in the attacker’s hands.

Carry one sentence out of this week: do not authenticate on appearance. It is the whole chapter in five words. The deepfake, the perfect phishing email, the poisoned document, the confident-wrong AI summary — every one of them is an appearance asking for your trust. The disciplines we built all refuse it: out-of-band verification, phishing-resistant auth, output validation, the human gate, the verified hypothesis. Sober-minded and watchful. That is the posture against an enemy who disguises himself, and it was good advice nineteen centuries before it was good security engineering.

You will not catch everything. No watchman does. That is why next week you build the ark before the flood. But the watching is still the work, and the watching is still yours.

See you next week.

Up next: Complete every rep in the exercises. Then build Project 10 — Attacker and Defender. Reference Appendix A for the lab, Appendix B for local + cloud AI, and Appendix C for the agentic-AI rules your SOAR governance must satisfy. Then read Chapter 11 — Backup, Recovery, and the Ark You Build Before the Flood, where you prepare for the day the watch is not enough.