Chapter 8 — The Proposal, and Defending It

“No one ever made a great discovery without a great deal of preparation… and the preparation is mostly the asking of the right question, and then standing still long enough to be wrong out loud.” — adapted from the research-methods tradition; the durable lesson is older than any one source

“Iron sharpens iron, and one man sharpens another.” — Proverbs 27:17 (ESV)

Why This Matters

You are at the hinge of the course. Behind you are seven weeks of finding and framing: you read like a scientist, found a gap, built a literature review and a comparison matrix, matched a modern AI method to your chosen domain, sharpened a falsifiable hypothesis, and designed a reproducible experiment. Ahead of you are eight weeks of doing and defending: environment, pilot, full run, analysis, discussion, paper, peer review, symposium.

This week is the lock. The proposal takes everything from the first half — problem, literature, method, expected outcomes, validity threats — and freezes it into a plan you will commit compute and weeks to. And because a plan you never have to defend is a plan you never had to think through, you will defend it: a 10–15 minute talk, a peer-review session where classmates score your work and ask the one question that hurts, and a revised written proposal that answers them. This is your midterm — 25% of your grade (15% the written proposal, 10% the presentation).

Here is the thing about a proposal that nobody tells you early enough: the proposal is where research integrity gets cheap or expensive. Right now — before any data exists — committing to a baseline, a metric, a number of seeds, and a statistical test costs you nothing but discipline. After the data is in, every one of those choices becomes a temptation. The run that “didn’t work” begs to be dropped. The metric that didn’t move begs to be swapped for one that did. The proposal is where you bind your future self to honesty, while it is still free.

The AI thread runs through this week from both sides. As a tool, an LLM is a genuinely good rehearsal partner — it will draft a hostile reviewer’s questions, tighten a wordy slide, and stress-test your hypothesis faster than a human could. As a workload, your applied-AI method is the thing on trial; your proposal must say which model, which version, which baseline it beats, and how you’ll keep a closed, drifting model from making your results unreproducible. And the spine rule holds: the AI rehearses the defense, but you stand at the front of the room and answer for every claim. “The model wrote it” is never a defense — not in this room, and not at any venue you will ever submit to.

So the apologetic question for the week is the oldest one in this book about how knowledge gets made together: how does iron sharpen iron? Why does truth get stronger when it is examined by someone who is trying to find what’s wrong with it — and what does that demand of how you give criticism, and how you take it?

One more thing before we build. This week asks you to do something that feels backwards: to invite people to attack the thing you’ve spent eight weeks building. Every instinct says protect it. But a proposal is not a possession; it is a hypothesis about the future, and a hypothesis you won’t let anyone test is just a wish with citations. The researchers you admire are not the ones who were never wrong. They are the ones who set up their work so that if they were wrong, someone would find out — and who thanked the person who did. That posture is the whole job this week. Let’s build it.

8.1 — What a Proposal Actually Is

A research proposal is a pre-execution contract — with your committee, with your future self, and with the truth. It is not a summary of what you’ve read; you wrote that in Week 4. It is a precise, falsifiable statement of what you are about to do, why it is worth doing, and how you’ll know whether it worked.

A complete CS practicum proposal has seven load-bearing parts. You have built all seven across the first half of the course. Assembling them is the work of this week.

The proposal is the moment the comparison matrix pays off. The empty cell you found in Week 4 — the column no prior paper filled — is your problem statement. If you cannot point at a specific, cited absence, you do not yet have a proposal; you have an interest. Go back to the matrix.

There is a rhetorical move underneath all of this, and it’s worth naming because reviewers feel it whether or not they can name it. Swales called it CARS — Create A Research Space — and Move 2 is “establishing a niche”: you describe the territory (what’s known), then you show the gap in it (what’s missing), then you announce that you will occupy that gap. A proposal that does only the first move — “here’s a lot of related work” — reads as a literature review with no point. A proposal that skips to the third — “I will build X” — reads as a project with no reason. The middle move, the cited absence, is the one that turns reading into research. You’ll meet CARS again in Week 14 when you write the paper’s introduction; the introduction is the proposal’s first two sections, grown up.

Coach’s Note — A proposal that another researcher could disagree with is alive. A proposal everyone nods along to is usually too vague to be wrong, which means it is too vague to be research. If your committee can’t imagine the result coming out the other way, sharpen the claim until they can.

8.2 — Locking the Plan: Preregistration

The most consequential thing you can do this week takes twenty minutes and costs nothing. Preregister your analysis plan.

A preregistration is a time-stamped, read-only declaration of your method before you collect data. Two free, open registries are standard as of 2026: OSF (the Center for Open Science — time-stamped, public, read-only once posted) and AsPredicted.org (an eight-question form). The Center for Open Science also runs Registered Reports, where a venue grants in-principle acceptance on the plan itself, before any result exists — the purest form of “we judge your method, not your luck.”

Why bother? Because of two named failure modes you must be able to recognize:

• HARKing — Hypothesizing After the Results are Known (Kerr, 1998). You run an exploratory analysis, find something that crossed significance, and write it up as if you had predicted it. The hypothesis is now unfalsifiable theater.
• p-hacking / researcher degrees of freedom — Simmons, Nelson & Simonsohn, “False-Positive Psychology” (Psychological Science, 2011). Try enough metrics, subgroups, and stopping rules on noise and something will cross p < 0.05. They showed you can get a “significant” result for an effect that cannot possibly be real.

Preregistration kills both, because the public record fixes — with a date — what you predicted, what metric you’d use, how many seeds you’d run, and what test you’d apply. Start from code/preregistration.txt, fill it, and post it.

Coach’s Note — Preregistration is not a cage; it is a clearing. You can still explore — you just have to label exploratory findings as exploratory. The honesty is in the labeling, not in pretending you never looked.

8.3 — The Method Section a Reviewer Can’t Knock Down

Your method section is where the proposal stops being a wish. A reviewer reads it asking one thing: if I had your data and your code, could I get your numbers? Make the answer yes on paper before it has to be yes in practice.

Six commitments every applied-AI method section must make explicit:

1. Baselines. What do you beat, and is it a fair baseline? “We compare to no method” is not a baseline. For an LLM method, the prompt-only or retrieval-only version of your own system is often the most honest baseline.
2. Ablations. One ablation per claim. If you claim the retrieval step matters, you must run the version without it.
3. Data splits. Train/validation/test fixed in advance, and named. The classic leakage bug — scaling or feature-selecting before the split — is documented across 294 papers in 17 fields (Kapoor & Narayanan, 2023, Patterns). Commit to the split now.
4. Seeds / runs. Multiple seeds, reported as a distribution. Single-run “we beat the baseline” is statistically empty: deep-learning results vary substantially by initialization and data order (Bouthillier et al., 2021, MLSys).
5. Metric, operationalized. Not “accuracy” — which accuracy, computed how, on what. State whether you’ll report effect size and a confidence interval, not a bare p-value (ASA, 2019).
6. Model provenance. If a closed model is your instrument, name the model and version and date — closed models drift, and a result you can’t pin to a version is a result no one can reproduce.

The applied-AI seed examples from the course menu make this concrete:

Coach’s Note — Name your baseline on the first slide you draft the method, not the last. A method built to beat a baseline you chose afterward is a method built to win, not to learn. Choose the baseline that would hurt if you lost to it — that’s the one whose defeat means something.

A word on the closed-model trap, because it bites applied-AI proposals specifically. If your method calls a hosted, closed model — and as of mid-2026 most do — that model is an instrument whose behavior can change under you without notice. Frontier model names and versions churn weekly; a result you collected against one version may not reproduce against the next. So your method section must commit to pinning the exact model, version string, and date of every model call, and to logging the temperature and prompt. This is not paranoia; it is the difference between a result and an anecdote. We build the full reproducibility chain in Chapter 9, but the commitment to provenance belongs in the proposal, where it’s still free to make.

8.4 — Building the Talk: Slide Discipline

A 10–15 minute talk is a brutally short form, and that is the point. If you cannot say it in twelve slides, you do not yet understand it well enough to defend it.

The discipline is one idea per slide, roughly one minute per slide. Twelve is the ceiling. Use the skeleton in code/proposal-slide-outline.txt:

• Slide 1 is the research question in one sentence. If a stranger can’t restate your RQ after slide 1, the whole talk is at risk.
• Slide 4 is your comparison matrix, with you pointing at the empty cell. This is the rhetorical heart: the gap is visible, not asserted.
• Slide 7 is the hypothesis and the null — what result would refute you. A talk that can’t be wrong is a sales pitch.
• Slide 10 names your top two validity threats and their mitigations, plus your AI-disclosure plan.
• Slide 11 is the timeline: Weeks 9–16, with the pilot in Week 10 (catching a broken pipeline before you spend the compute budget).

Rules that separate a defensible talk from a pretty one:

• No paragraphs on slides. Bullets ≤ 7 words. The audience reads or listens, never both.
• Every figure caption stands alone. A reviewer who sees only the slide should still get it.
• Repeat the RQ. Put it on slide 1 and slide 6. People drift; repetition is mercy.
• Build back-pocket slides — full bibliography, extra ablations, the threats table — that you do not show but reach for when a question demands evidence.

Coach’s Note — Rehearse the talk out loud, on a timer, at least three times. The first time you’ll run long; the second you’ll find the dead slide; the third you’ll hear which sentence you don’t actually believe. That sentence is where a reviewer will go.

8.5 — Using AI to Rehearse the Defense (and Where It Fails)

An LLM is a strong sparring partner for a proposal — used inside the spine rule.

What it does well:

• Generate the hostile reviewer. Prompt: “You are a skeptical Reviewer 2. Here is my RQ, hypothesis, and method. Ask me the five hardest questions, and for each, name the validity threat behind it.” You will get a usable rehearsal set.
• Tighten slides. Paste a wordy slide and ask for it in seven words per bullet. This is assistive editing — generally outside disclosure requirements at most venues.
• Pressure-test falsifiability. Ask it to state the result that would refute your hypothesis. If it can’t, neither can your committee — fix the hypothesis.

Where it fails, and where the human stays in the loop:

• It will invent citations. Walters & Wilder (2023, Scientific Reports) found 55% of GPT-3.5 and 18% of GPT-4 citations entirely fabricated, and among the real ones, a quarter to a half had substantive metadata errors. Verify every reference against dblp or Semantic Scholar. A hallucinated citation in a proposal is the kind of thing that, at a venue like ICCV, gets a paper rejected without review.
• It will agree with you. An LLM is trained to be helpful, which often means agreeable. It will smooth over a weak baseline because you asked it to help, not to fight. The hostile-reviewer framing partly fixes this, but a human peer (8.6) will see the social and methodological holes a model won’t.
• It cannot own the claim. No major venue lets an LLM be an author — authorship requires accountability a tool cannot bear (the cross-venue consensus: ACM, IEEE, NeurIPS, ICLR, ICML, ACL, CVPR, arXiv all agree). When you disclose AI use, it goes in the Acknowledgements (ACM/IEEE/ACL) or the experimental-setup section (NeurIPS, if it’s a non-standard part of the method). Spell out which tool and which sections.

Coach’s Note — The right way to use an LLM here is as a whetstone, not a judge. It sharpens the question; you decide whether the answer is true. The moment you let it decide, you’ve handed away the one thing a researcher can’t delegate — accountability for the claim.

8.6 — The Peer-Review Session

This is where iron meets iron. In the peer-review session, classmates score your proposal on six dimensions and write the one question that improves it most — and you do the same for theirs.

The craft of a good review is itself a skill the course is teaching, because it is the same skill that runs every real venue. Top CS conferences use double-anonymous review; reviewers read for novelty, methodology, and clarity, and the best reviews are specific, cited to a line, and constructive. A few rules carry from the real world into this room:

• Score the lowest dimension first. If the method scores a 2, that’s the review — clarity polish is rearranging deck chairs.
• Make the question answerable. “What baseline would falsify your claim?” beats “Is this novel?” The first moves the work; the second just stings.
• Cite the evidence. “On slide 8 you scale before the split — that’s leakage” is a review. “Methods feel weak” is a vibe.
• Confidentiality is real even here. Do not paste a classmate’s unpublished proposal into a public LLM. That is exactly the breach NeurIPS and IEEE forbid their reviewers to commit — reviewers may not upload submissions to any LLM. Practice the discipline now.

Use code/proposal-peer-review-rubric.txt as the worksheet. The six dimensions — problem, novelty, method, feasibility, validity, clarity — plus an integrity check, score out of 35. The deliverable from your review of a peer is the score sheet and the one constructive question.

Here’s why this is worth taking seriously beyond the grade. The peer-review session is a scale model of the machinery that governs every venue you will ever submit to. When you read your classmate’s proposal looking for the weakest claim, you are doing exactly what a NeurIPS or CHI reviewer does with a real submission. The skills transfer one to one: reading for the unstated assumption, spotting the missing baseline, noticing that the metric doesn’t actually measure the construct the author claims it does (the live applied-AI question — does a high benchmark score measure reasoning or memorization?). A student who learns to review well in Week 8 is a student who, by Week 15, can review two peers’ full papers and read the reviews of their own without flinching. The session this week is where that muscle starts.

And a concrete shape for the “one question.” Suppose a classmate proposes an LLM agent for code review and claims it “improves” review quality, with the project’s existing linter as the baseline. The lowest-scoring dimension is probably construct validity: “review quality” is not defined. The single best question is not “is this novel?” — it’s: “What is your operational definition of ‘review quality,’ and what result on that metric would mean your agent is no better than the linter?” That question is specific, cited to the construct, answerable, and it forces the author to do the most valuable rewrite. That’s the shape you’re aiming for.

8.7 — Revising Under Critique

The talk and the review are not the deliverable. The revised proposal is. A proposal that doesn’t change after the peer-review session means one of two things: the review was empty, or you didn’t listen. Neither reflects well.

Triage every comment into three buckets — the same triage real authors use when reviews come back:

A comment you disagree with still demands a response: in your revision memo, say why, with evidence. “I kept baseline X because Y is not comparable for reason Z” is a researcher’s answer. Silence reads as either agreement or evasion, and a committee can’t tell which.

Coach’s Note — The hardest review to receive is the correct one. It is much easier to defend a flawed plan than to admit the reviewer found the flaw. Watch for the heat in your chest when you read a hard comment — that heat is almost always pointing at the thing that’s actually wrong.

8.8 — Feasibility: The Timeline That Tells the Truth

The most common way a good proposal dies is not a bad idea — it’s a plan that can’t be done in the weeks and the compute that remain. Feasibility is a real grading dimension, and it’s the one students flatter themselves about.

Build the timeline as a small Gantt over Weeks 9–16, and put the pilot in Week 10 as a hard checkpoint, not an afterthought:

Two feasibility questions a committee will ask, so answer them on the slide:

• Compute. Can this run on the hardware you actually have — a laptop, Colab, a Codespace, a shared cluster? If your method needs a GPU you don’t have, the proposal is not feasible as written. Scope it down to a pilot that fits, or name where the compute comes from. (See Appendix A for the cheapest paths that work.)
• Data. Is the dataset already in hand, or are you assuming access you don’t have? An assumed dataset is the silent killer of practicum timelines.

Coach’s Note — Halve your ambition and double your time estimate. Every researcher who has ever built a timeline has been optimistic, and every one has been wrong in the same direction. The proposal that scopes down to something it can actually finish beats the grand plan that stalls in Week 11.

8.9 — The Cumulative Midterm Review (Weeks 1–7)

This week also closes the methods half of the course. The Canvas midterm is a separate, auto-graded quiz over the research methods of Weeks 1–7 — it is not the proposal. Both are due this week; don’t let one eat the other.

Study from code/midterm-review-map.txt. One idea per week, one anchor to name from memory:

Coach’s Note — The quiz checks that the vocabulary is in your hands; the proposal proves you can wield it. Study for the quiz the way you’d warm up — quickly, to loosen what’s already there. Spend your real hours on the proposal.

8.10 — Interactive Lab: Proposal Peer-Review Rubric

Below this chapter on the website is the Proposal Peer-Review Rubric widget. Use it now.

You’ll be shown a sample proposal (problem, RQ, method, expected outcomes). Score it on the five core dimensions — problem, novelty, method, feasibility, clarity — then write the one constructive question that would improve it most. The widget shows you how a reviewer’s scores aggregate, flags when your question is too vague to be actionable, and reveals the “expert” review for comparison.

What it teaches: that reviewing is a skill distinct from writing, that the most useful score is the lowest one, and that the best question is specific, answerable, and aimed at the weakest part — not the easiest target. Do the lab before your live peer-review session; it is the rehearsal for the room.

8.11 — How Does Iron Sharpen Iron?

“Iron sharpens iron, and one man sharpens another.” (Proverbs 27:17, ESV)

Here is something strange about how knowledge gets made: a claim that has never been attacked is weaker than one that has survived an attack. Peer review — the whole apparatus of double-blind reviewers, hostile questions, and revision memos — exists because truth gets stronger under examination. That is not a slogan; it is the engineering principle underneath the entire scientific enterprise. We do not trust a result because its author is confident. We trust it because people who wanted to find it wrong tried, and couldn’t.

The proverb names the mechanism. Iron does not sharpen iron gently. Sharpening is friction — metal grinding metal, sparks, heat, the removal of material. It is, frankly, a little violent. And the iron being sharpened does not get to choose a softer stone. This is why a real peer-review session can feel like an attack even when it is a gift: the friction is the point. A reviewer who only praises you has left your blade dull.

But the proverb cuts both ways, and the second edge is the one students miss. You are also the iron doing the sharpening. The way you review a classmate’s proposal is a moral act, not just a technical one. You can grind to sharpen — specific, cited, aimed at the work — or you can grind to wound — vague, personal, aimed at the author. The same friction; opposite intent. The book’s frame for research is that it is the disciplined, humble search for truth, “the glory of kings to search things out” (Prov 25:2). Humility shows up most clearly in how you hand someone a hard truth: you assume they want the paper to be good as much as you do.

And it shows up in how you receive the hard truth. “Whoever loves discipline loves knowledge, but he who hates reproof is stupid” (Prov 12:1, ESV) — a verse you’ll meet again in Week 15. The instinct to defend a flawed plan is strong; it is the instinct to protect the self at the cost of the truth. Confessional honesty runs the other way: the reviewer who found your leakage bug did you a kindness, and the right response to a kindness is gratitude, then revision. The proposal you defend this week is not yours to protect. It is yours to make true, and that is a job too big to do alone. That is why we do it in a room full of iron.

There’s a quieter point here about the limits of the very tool this course keeps in your hand. An LLM will rehearse the hostile reviewer, but it cannot truly sharpen you the way a human can, because the friction that sharpens is the friction of another person who has their own stake in the truth and their own blind spots different from yours. The model agrees because agreeing is what it was trained to do; the human across the table disagrees because they see something you cannot. Proverbs 18:17 — the verse over the project — is precisely about this: the one who states his case first seems right, until the other comes and examines him. Not until a tool checks his grammar. Until the other comes. The examination that makes knowledge trustworthy is fundamentally relational, and that is not a bug in how science works — it is the design. We were not made to find the truth alone, and the proposal defense is a small, honest acknowledgment of that.

So receive the examination as a gift, and give it as one. When you sit in that room this week — scoring a classmate, fielding the hard question, writing the revision memo — you are practicing something older and larger than a grading rubric. You are practicing the humility that the whole search for truth runs on.

8.12 — Common Pitfalls

Pitfall: A proposal that asserts a gap instead of citing one. Example: “Little work has examined LLMs for SQL optimization” — with no citations showing the absence. Fix: Point at the empty cell in your comparison matrix. A gap is a cited absence: “Of the six systems in Table 1, none evaluate on schema-shifted queries.” Show the table.

Pitfall: An unfalsifiable hypothesis dressed as a research question. Example: “We investigate whether AI can improve code review.” (Improve compared to what? By how much? What would failure look like?) Fix: State H1, the null, and the refuting result. “H1: the agent finds more test-failing bugs than the linter; H0: no difference; refuted if the agent finds ≤ the linter across 5 seeds.”

Pitfall: No baseline, or a strawman baseline chosen to be beaten. Example: Comparing your RAG system only against “no method,” or against a deliberately untuned competitor. Fix: Choose the baseline whose defeat would mean something — usually a strong prior method or the prompt-only version of your own system. Name it on the first method slide, before the result exists.

Pitfall: Hallucinated citations from an LLM left unverified in the proposal. Example: A confident reference to “Smith et al. 2024, ICML” that does not exist, surfaced by a chatbot. Fix: Verify every citation against dblp or Semantic Scholar before it enters the document. Remember 55%/18% (Walters & Wilder, 2023). At real venues this gets papers rejected without review.

Pitfall: A timeline that ignores the pilot. Example: “Weeks 9–13: run all experiments.” No small run first. Fix: Schedule the Week 10 pilot explicitly. The pilot validates the pipeline — splits, metrics, seeds, compute budget — before you spend the full run. Catching leakage on 10% of the data is cheap; catching it after the full run is not.

Pitfall: Treating the peer review as theater — collecting comments and changing nothing. Example: A revised proposal that is identical to the draft except for a typo fix. Fix: Triage every comment (major/minor/clarification) and write a short revision memo. A comment you disagree with gets a reasoned response with evidence, not silence.

Pitfall: Pasting a classmate’s unpublished proposal into a public LLM “to help review it.” Example: Dropping the whole draft into a chatbot and asking it to grade it. Fix: Review with your own judgment. Confidentiality of unpublished work is a real obligation — the same one that forbids NeurIPS and IEEE reviewers from uploading submissions to any LLM. Practice it now.

8.13 — Reps

The reps for this week are in the exercises. They are not warmups — they are the proposal, built piece by piece. Preview:

• Rep 1 — assemble the seven-part proposal skeleton from your first-half artifacts.
• Rep 2 — write the falsifiable claim in four lines: RQ, H1, the null, and the result that refutes you.
• Rep 3 — preregister your analysis plan on OSF or AsPredicted from code/preregistration.txt.
• Rep 5 — build the 12-slide talk and rehearse it on a timer, out loud, three times.
• Rep 8 — run the LLM as a hostile Reviewer 2, then verify any citation it touched.

Do the on-page Check Your Reps quiz when you finish the chapter — five questions over this week’s material. Then go to the reps.

8.14 — This Week’s Deliverable (the Midterm)

This week’s deliverable is the midterm: Project 8 — the Proposal Presentation + Revised Written Proposal, worth 25% of the course (15% the written proposal, 10% the presentation). You will deliver a 10–15 minute talk, sit a peer-review session (giving and receiving), and submit a revised written proposal that answers the critique. The project file also carries the cumulative midterm-review section over Weeks 1–7; the auto-graded methods quiz on Canvas is separate.

This is the artifact that turns “a class project” into “the first half of a thesis.” Treat it that way.

8.15 — Coach’s Final Word

Eight weeks ago you had an interest. This week you have a plan you can defend — a cited gap, a falsifiable claim, a baseline you chose because beating it would mean something, a method another researcher could rerun, and the honesty, locked in by preregistration, to report what the data says instead of what you wished it said.

The defense will feel like an attack. It is supposed to. Iron sharpens iron, and the friction is the gift. Walk into that room assuming your reviewers want your paper to be good — and review theirs the same way. Give the hard truth kindly; receive it with gratitude, then revision.

You are halfway. The finding and framing are done. Now we go do the work.

See you on Monday.

Up next: the exercises builds the proposal rep by rep · Project 8 is the midterm itself · then Chapter 9 — where you build the reproducible environment the plan promised. Reference appendices: Appendix A (research environment), Appendix B (researcher’s toolkit), Appendix C (using AI responsibly), Appendix D (glossary). Previous: Chapter 7.