Chapter 10 — Reps

Conditioning for the watchman. Detection, injection, and governed response — by hand first, then with the AI, then judge the AI.

Ground rules:

1. Type every line yourself. This is Phase 2, so the project turns agentic AI on. The reps do not. You cannot judge an AI defender’s output if you have never hand-written a detection rule, hand-traced a kill chain, or spotted an indirect injection with your own eyes. Build the baseline; then you have the standing to grade the machine.
2. Run every script. Point triage_logins.py at auth.log and read the flags. Read the false positives. The output is a hypothesis, not a verdict — you supply the verdict in writing.
3. Predict before you reveal. Before you run a detector or open the answer key, write down what you expect it to catch and what you expect it to miss. The gap between your prediction and the result is the lesson.
4. Never run an attack outside a sandbox you own. Every offensive rep here targets the provided sample files or your own lab only. Read Appendix A for the lab and Appendix C for the agentic-AI rules. Crafting a real phishing message to send to a real person is a crime, not a rep.
5. Cite the framework by ID. When you classify something, name it: OWASP LLM0x, MITRE ATLAS AML.T#### (live count from atlas.mitre.org), a kill-chain phase, or an ATT&CK T####. “It’s bad” is not an answer; “it’s LLM01 indirect injection, ATLAS prompt-injection technique” is.

The sample files for the week live in code/: code/auth.log, code/triage_logins.py, code/rag_notes.txt, code/soar_playbook.yml.

Reps 1–3: See the Attack Without the AI

Rep 1 — Map the kill chain by hand

Open code/auth.log. Do not run any tool yet. With pen and paper (or a plain text file), find the attack chain and label each line with (a) its kill-chain phase and (b) a MITRE ATT&CK technique ID where one fits (e.g., T1078 Valid Accounts, T1021 Lateral Movement, T1048 Exfiltration).

Write: the ordered chain, one line per step, and the single line you would alert on first if you could only alert on one. Defend the choice — breaking which link costs the attacker the most?

Rep 2 — Run the baseline detector and grade it

python3 code/triage_logins.py code/auth.log

Read every flag. For each, decide: true positive (real attack) or false positive (benign). Then find the gap: there is at least one real attack step the five hand-written rules do not flag.

Write: your TP/FP table, plus the new rule the engine is missing, expressed as a one-line condition (e.g., “ok-auth from a geo this user has never used in 30 days”). Why didn’t the existing rules catch it?

Rep 3 — Quantify the false-positive economy

Assume your real SIEM sees 2,000,000 auth events a day and a detector that is 99.5% accurate.

Write: the daily false-positive count, and one sentence on what that does to an analyst’s behavior over a week. Then propose one correlation or suppression rule that would cut it without raising the miss rate. Tie your answer to OWASP LLM10 (Unbounded Consumption) — why is alert fatigue the same failure mode wearing a human face?

Reps 4–6: The AI as Target

Rep 4 — Find the indirect injection

Open code/rag_notes.txt. It is a help-desk knowledge-base note that an AI assistant retrieves and acts on. Exactly one passage is an indirect prompt injection (LLM01).

Write: quote the injected instruction, name the OWASP ID, and state which of the three lethal-trifecta legs (private-data access / untrusted content / exfiltration vector) you would remove to neutralize it — and the concrete config change that removes it.

Rep 5 — Build and break a tiny agent (sandbox)

In your lab, point a local model (Ollama, see Appendix B) at a trivial agent loop that retrieves code/rag_notes.txt into context and is told it has a (fake, logging-only) send_data(url) tool. Observe whether it tries to call send_data with the attacker URL.

Write: what happened, then apply least privilege (remove the tool) and an output-validation step (reject any tool call to a non-allowlisted host) and confirm the injection no longer fires. Map your two fixes to LLM05 and LLM06.

Safety: the tool must be a stub that only logs. Never wire a real outbound call to a model reading untrusted content.

Rep 6 — Diagram EchoLeak

Reconstruct the EchoLeak (CVE-2025-32711) kill chain in four-to-six steps from the chapter. For each step, label one OWASP LLM ID and one MITRE ATLAS technique (use the live catalog at atlas.mitre.org; do not memorize a count).

Write: the labeled diagram, plus one sentence on which single architectural control (trust boundary / least privilege / output verification) would have broken the chain earliest, and why “a better classifier” was not enough.

Reps 7–9: The AI as Defender, Governed

Rep 7 — Hand-correlate an alert storm

Take your TP/FP table from Rep 2. Imagine the same chain produced 40 raw alerts (every denied auth, every sudo, every query its own alert).

Write: how you would correlate those 40 into one incident, and which field (user? source IP? time window?) is the correlation key. This is the Chapter 9 noise-reduction pillar applied to security — name it.

Rep 8 — Question the vendor

Using the chapter’s figures, write the Security Copilot cost of an analyst running it provisioned for a 6-hour incident, and what it would cost at the overage rate. Then find one vendor claim in this space (e.g., “sub-100ms autonomous containment,” “X% faster MTTR”) and write the one question you would ask the vendor to falsify it.

Write: the cost math (label every figure “as of 2026, verify”), and your falsification question.

Rep 9 — Write the autonomy policy

Read code/soar_playbook.yml. For each action (enrich, ai_summary, revoke_sessions, firewall_block, disable_user, notify_donors), state whether it should be auto, approval_gate, or human_only, and why — using the test “is it reversible?” and “does it touch external parties?”

Write: your six-row policy table with a one-line justification per row, and the one place you’d disagree with the shipped playbook. Tie the whole thing to OWASP LLM06 and EU AI Act Article 14.

Reps 10–11: Synthesis

Rep 10 — Spot the tells

Write three short messages of your own (in a text file, not sent anywhere): one legitimate, one AI-generated-style phishing lure, and one that looks benign to a human but contains an instruction aimed at an AI assistant.

Write: for each, the tell a human defender would use and the tell an AI/output-filter defender would use — and note where they differ. This is the muscle the Phishing Defender Drill trains; do it on paper too.

Rep 11 — One sentence per OWASP ID

Without looking, write all ten OWASP LLM Top 10 (2025) IDs and a one-line meaning for each. Then check against §10.4 and fix what you missed.

Write: the ten lines, and the two you got wrong (everyone gets two wrong the first time — which two, and why those?).

Done? One Last Thing.

Capstone Rep — The smallest possible Attacker-and-Defender

Rehearse Project 10 in miniature, in your lab, in under an hour:

1. Attacker: take code/rag_notes.txt as your injection payload and code/auth.log’s chain as your intrusion. Write a three-line kill-chain summary of the combined attack.
2. Defender: run code/triage_logins.py, add the one rule it was missing (from Rep 2), and confirm the chain is now fully flagged.
3. Governance: decide — in one sentence — the single action in code/soar_playbook.yml you would never let run without a human, and why.

Write a five-line recap.txt: the attack in one line, the missing rule you added, the OWASP IDs in play, the one human-gated action, and one sentence on where you (not the AI) supplied the judgment. If you can write those five lines crisply, you are ready for the project.

Up next: Project 10 — Project 10: Attacker and Defender.