Chapter 2 — Reps

Every rep this week moves your own Research Opportunity Report forward. By the end you’ll have three domains, ten scored questions, and a gap writeup — the deliverable, assembled rep by rep. The keyboard is the gym; aim is the lift.

Ground rules

• Work in your portfolio. Everything lands in your Git repo (a week-02/ folder) and, where it’s a source, your Zotero library. Commit as you go.
• AI proposes, the literature disposes. You may use AI to brainstorm and rephrase. You may not let an AI-asserted gap or an AI-generated citation enter your report unverified. Verify every reference in dblp or Semantic Scholar.
• Tag honesty. Every gap is confirmed (you have a citation for the absence/limitation) or unconfirmed (a hunch you’ll chase in Chapter 3). No silent fudging.
• One domain, all semester. The domain you commit to in Rep 1 is the one you keep. Choose like it’s permanent, because it is.
• Templates live in code/. Use code/gap-analysis-template.txt, code/question-scorer.csv, and code/heilmeier-catechism.txt.

Rep 1 — Commit your three candidate domains

Pick three domains from the menu (AI/ML, security, networking, SE, databases, HCI, graphics, AR/VR, robotics, cloud, HPC, architecture, data science). For each, write one sentence: why is modern AI changing this domain right now?

Domain 1: Databases
  Why AI matters now: LLMs are being applied to query optimization and text-to-SQL,
  challenging cost-based optimizers that have been the standard for decades.
Domain 2: ...
Domain 3: ...

Reflect (3–4 sentences): Which of the three can you actually run given your access to data, compute, and a baseline? Rank them by the Access filter from §2.2.

Rep 2 — Find one anchor paper per domain

For each of your three domains, locate one recent, real anchor paper (use the §2.2 table or find your own via Google Scholar / Semantic Scholar / arXiv). Record full metadata and verify each exists in dblp.

Domain 1 anchor: Author(s), Year, Title, Venue, arXiv/DOI — [verified in dblp? Y/N]

Reflect: For one anchor, read only its Related Work and Future Work. What does it explicitly say it did not do? Quote one sentence.

Rep 3 — Mine five Future-Work sections into five problems

Open five recent papers in your favorite of the three domains. Read each Future Work section. Turn each into one candidate problem (not yet a question).

Paper 1 (cite): Future Work says "..." → Problem: current X methods are untested on Y.
Paper 2 (cite): ...

Reflect: Which gap shape (§2.3 — unverified claim / narrow benchmark / contamination-prone / untested transfer / missing baseline) does each problem fit? Label all five.

Rep 4 — Convert observations into questions

Take your three strongest problems and run each down the topic → problem → question staircase (§2.4): state it, generalize, make it measurable, make it comparative.

Problem: LLM code reviewers miss security bugs.
  Measurable: recall on OWASP-Top-10-labeled defects.
  Comparative: vs. a static-analyzer baseline.
  Question: "Does an agentic LLM reviewer detect more OWASP-class vulnerabilities
             on SWE-bench-Verified patches than a single-pass prompt and than Semgrep?"

Reflect: For each, name the thing you would count and the baseline you might lose to. If you can’t name both, the question isn’t measurable yet.

Rep 5 — The AI-citation drill (do this one slowly)

Ask any chatbot for five references on a niche corner of your domain. Then verify each in dblp and Semantic Scholar.

Ref 1: <as given by the model>  → dblp: FOUND / NOT FOUND ; metadata errors: ___
Ref 2: ...
Fabrication count: __ / 5      Metadata-error count: __ / 5

Reflect: Compare your fabrication rate to Walters & Wilder 2023 (55% GPT-3.5 / 18% GPT-4 entirely fabricated). What does this do to how you’ll treat AI output for the rest of the semester? Write the rule you’ll follow.

Rep 6 — Score all candidates with FINER

Put every question you’ve drafted (aim for ten across the three domains) into code/question-scorer.csv. Score each 1–5 on Feasible, Interesting, Novel, Ethical, Relevant.

question_id,domain,question,feasible,interesting,novel,ethical,relevant,total,notes
q01,databases,"Does RAG-augmented prompting...",4,4,3,5,4,20,"baseline = LLMSTEER"

Reflect: Which dimension killed the most questions? For most students it’s Feasible or Novel — say which, and why, in two sentences.

Rep 7 — Read one OpenReview thread

Find one paper in your domain on OpenReview (ICLR 2026 / NeurIPS 2026 / ICML). Read one full reviewer–author thread. Tabulate every objection of the form “the authors did not…” / “this is only shown on…” / “I’m concerned that…”.

Objection 1: "Only evaluated on one benchmark." → gap shape: narrow benchmark
Objection 2: "No comparison to a fine-tuned baseline." → gap shape: missing baseline

Reflect: Did any reviewer objection match a gap you’d already guessed? Reviewers are the field’s gatekeepers — a gap they name is confirmed. Promote it.

Rep 8 — Run your top two through Heilmeier

Take your two highest-scoring questions and answer the Heilmeier Catechism for each, using code/heilmeier-catechism.txt. Be honest on #2 (limits of current practice) and #4 (who cares).

Q: "Does an agentic LLM reviewer detect more OWASP-class vulns..."
1. Plain language: ...
2. Today / limits (cite!): ...
4. Who cares: secure-code teams; CI tooling vendors.

Reflect: Could you answer #2 with a citation, or only with “I think people just…”? If the latter, the gap is unconfirmed — flag it.

Rep 9 — Write the gap-analysis paragraph for your favorite

For your single top question, write the gap analysis as three CARS moves in code/gap-analysis-template.txt: territory (what’s known, cited) → niche (the cited absence) → occupy (what you’d do).

Territory: LLMs now rival cost-based optimizers on text-to-SQL (cite).
Niche: but their query-plan latency has not been measured against [baseline] on [benchmark] (cite the absence).
Occupy: I will compare RAG-prompted vs. fine-tuned plan selection on JOB.

Reflect: Mark each cited claim confirmed/unconfirmed. How many are still hunches? That number is your Chapter 3 to-do list.

Rep 10 — Tag your AI use and start the disclosure note

In your portfolio repo, create ai-use.txt. List every place you used AI this week and classify it assistive (no disclosure needed) or generative (disclose). This is the running note that becomes your paper’s Acknowledgements line.

- Brainstormed candidate domains (generative — disclose).
- Rephrased Q4 for clarity (assistive).
- Drafted gap paragraph wording (generative — disclose; verified all citations).

Reflect: Per §2.7 / Appendix C — why can an LLM never be an author, and why is “the model did it” never a defense? Two sentences.

Done? One Last Thing.

Capstone rep — assemble the Research Opportunity Report. Pull Reps 1, 6, 7, 8, and 9 together into a single document matching the Project 2 spec: three domains, ten scored questions (the CSV), the gap-analysis writeup for your top one or two with confirmed/unconfirmed tags, and your ai-use.txt note. Drop the gap statement, in rough, into your Week-1 acmart/IEEEtran skeleton — that paragraph is the first real sentence of your paper. Commit it all. Then read it as a reviewer would and ask the one question from §2.5 you’d hate to be asked: who cares? If your report answers it, you’re ready to submit.

Up next: Project 2