Chapter 5 — Reps

This week’s reps move your research project forward: by the end you will have diagnosed your domain’s failure mode, walked the method ladder, named the baseline you intend to beat, and verified that the paper you anchor to actually exists. These are not warm-ups — they assemble directly into this week’s Domain-Specific AI Integration Proposal.

Ground rules.

• Work on your own chosen domain (the one you committed to in Week 2). Every rep produces an artifact you keep in your research portfolio.
• Use AI as a thinking aid, never as an author. If a model helps you draft, you own and verify every claim and every citation. Disclose per Appendix C.
• Verify every paper you cite against dblp or Semantic Scholar before it enters your bibliography. The 55%/18% fabrication figures from Chapter 5 are why.
• Commit each artifact to your Git repo as you finish it. A rep that isn’t committed didn’t happen.
• Cite time-sensitive facts as “as of 2026” and date any model name.

Rep 1 — Diagnose the failure

Write two sentences: what does your domain’s current best non-AI approach (or current AI baseline) get wrong, and is that error a knowledge failure (gets facts wrong), a behavior/form failure (right facts, wrong shape), or a can’t-elicit-it failure (it can do it but you can’t reliably get it to)?

Current approach: ____________________
What it gets wrong: ____________________
Failure type (knowledge / behavior / elicitation): ____________________

Reflect: The failure type points at a rung of the ladder before you’ve chosen any tool. Which rung does yours point to, and does that surprise you?

Rep 2 — Walk the ladder

For your problem, write one line each for Prompting, RAG, Fine-tune (LoRA/QLoRA), and (if relevant) Agent / RL / CV / Multimodal — stating whether each fits and why. Force yourself to argue for the lowest rung first.

Reflect: Did the lowest rung that could work turn out to be lower than the method you instinctively reached for in Week 2? Practitioners reach too high; check yourself.

Rep 3 — Pin your data shape

Fill this honestly. Your data is the binding constraint.

Labeled dataset?      yes / no  — size: ____   source: ____
Document corpus?      yes / no  — size/coverage: ____
Reward signal/sim?    yes / no  — what reward: ____
Only a few examples?  yes / no  — how many: ____

Reflect: Cross-reference with Rep 2 — does the method you want actually have the data it needs? If not, you’ve found this week’s most important problem early. Good.

Rep 4 — Name the baseline

Write the one sentence every method proposal reduces to:

"<Method X> beats <baseline Y> on <metric Z> by <margin M> under <conditions>."

Make Y a simple, often non-AI baseline (production heuristic, rule filter, base model with no retrieval, last published number).

Reflect: If you couldn’t fill in Y, you don’t yet have a research design — you have an enthusiasm. What’s the simplest honest Y you could measure this semester?

Rep 5 — Build the method-fit matrix

Open code/method-fit-matrix.csv and fill at least three candidate methods as rows: task shape, data shape, measured failure, baseline, primary failure mode. Then pick one and circle it.

Reflect: Seeing three candidates side by side, what made the winner win — fit, data availability, or just feasibility on your compute? Name it plainly.

Rep 6 — Anchor to a canonical paper, and verify it

Tie your domain to one canonical applied-AI paper from Chapter 5 §5.6 (e.g. SWE-bench/c-CRAB for SE, Aurora for networking, LLMSTEER for databases, Hawkeye/Glider for architecture, SAM 3 for vision). Then verify it exists with code/verify_citations.py against Semantic Scholar / dblp.

python code/verify_citations.py --query "Aurora reinforcement learning congestion control"

Reflect: Did the metadata you thought you knew (year, venue, authors) match what dblp says? Note any drift — this is the citation-hygiene muscle you’ll use all semester.

Rep 7 — Run the AI Method Matcher three ways

Use the AI Method Matcher widget on the chapter page. Run it (a) with the data shape you wish you had, (b) with the data shape you actually have (from Rep 3), and (c) one rung lower than your first instinct.

Reflect: Write 3–4 sentences on what changed between run (a) and run (b). The gap between your wish and your data is exactly where most projects fail — name yours.

Rep 8 — Instrument the right failure mode

If you chose RAG, write how you will measure retrieval recall/precision separately from answer accuracy. If you chose an agent, write what fields your trajectory log will capture. If RL, name the reward and the baseline policy. If fine-tune, name the catastrophic-forgetting check.

Reflect: Which single measurement, if you skipped it, would let a reviewer say “you blamed the model for a pipeline bug”? That’s the one you must build first.

Rep 9 — Govern every failure mode

List the failure modes for your chosen method (Chapter 5 §5.7). Give each a one-line mitigation. Citation hallucination → verify against dblp. Long-context fabrication → cap context / spot-check. Contamination → name a contamination-resistant benchmark.

Reflect: Which failure mode is most likely to actually bite your project, and is your mitigation real or aspirational?

Rep 10 — Draft the disclosure stance

Pick a plausible target venue family (ACM, IEEE, NeurIPS, ICLR, ACL, arXiv) and write the one paragraph stating how you will disclose AI assistance per that venue’s policy (Appendix C). Note where the disclosure goes (Acknowledgements vs experimental-setup vs submission form).

Reflect: Under your chosen venue’s rule, does the way you plan to use AI require disclosure, or is it assistive-only? Be honest — “the model did it” is never a defense.

Done? One Last Thing.

Capstone rep — Assemble the proposal core. Open code/proposal-skeleton.txt and code/baseline-card.yaml. Stitch Reps 1–10 into a single coherent draft: problem → diagnosed failure → method (ladder rung + justification) → data shape → baseline + metric (the baseline card) → canonical anchor (verified) → failure modes + mitigations → disclosure stance. It should read as one argument, not ten answers.

Then do the thing that separates a researcher from a tinkerer: read your draft as a hostile reviewer and write the single most damaging question they’d ask. If the answer is “you never showed it beats the simple baseline,” fix that now — before it’s in a submission. Commit the draft to your portfolio. This is the spine of Project 5.

Up next: Project 5