Chapter 2 — Finding a Problem Worth Solving

“The formulation of a problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill.” — Albert Einstein & Leopold Infeld, The Evolution of Physics (1938)

“What has been is what will be, and what has been done is what will be done, and there is nothing new under the sun.” — Ecclesiastes 1:9 (ESV)

Why This Matters

Last week you read like a scientist. You took three papers apart and saw the machine inside each one — the gap, the question, the claim, the evidence. This week you build the engine of your own paper. Everything you will do for the next fourteen weeks — the literature review, the experiment, the statistics, the draft, the talk — is downstream of one decision you make now: what problem are you going to solve?

Get this wrong and no amount of skill saves you. You can run a flawless experiment on a question nobody asked and produce a perfectly reproducible result that nobody needs. The reproducibility crisis gets the headlines — Baker’s 2016 Nature survey found more than 70% of researchers had failed to reproduce someone else’s experiment — but the quieter failure is more common and more fatal: a competent answer to a dead question. The keyboard is the gym, and this week the lift is aim.

Here is the trap, and it has a name this semester: Is there truly nothing new under the sun? Ecclesiastes says there is not. A first-year master’s student opens arXiv, sees the avalanche — hundreds of cs.* preprints a day — and concludes that everything worth doing is done, and done by someone with more compute. That feeling is real and it is wrong, and the difference between the feeling and the truth is exactly the skill this chapter teaches: gap analysis. Novelty is not a brand-new universe. Novelty is a specific, cited absence in the existing one.

The AI thread runs straight through it. You will use AI as a tool — to brainstorm directions, to summarize a subfield, to draft a question — and you will watch it fail in the most dangerous way it fails for a researcher: it will confidently hand you a “gap” that does not exist, citing papers that were never written. By Friday you will have a rule burned in: AI proposes, the literature disposes. The human stays where the judgment lives. And the work you choose is itself an instance of the thread — your job this semester is to investigate how modern AI advances your chosen CS domain, so the problem you pick this week is the problem you will live with until the symposium.

By the end of this chapter you will have three candidate domains, ten candidate research questions, and a gap-analysis writeup that backs your favorite with cited evidence. That package is your Research Opportunity Report — 5% of your grade, due this week, and the foundation of the proposal that comes at the midterm.

Coach’s Note — A problem is not a topic. “LLMs for SQL optimization” is a topic. “Does retrieval-augmented prompting reduce query-plan latency on the JOB benchmark versus a fine-tuned baseline?” is a problem. This chapter is the conversion of one into the other.

2.1 — Topic, Problem, Question: Know the Difference

Three words get used interchangeably by people who lose semesters. Pin them down.

The arrow goes topic → problem → question, and most of the work is making the arrow point downward and sharpen. A topic is where you start. A question is where you can put on running shoes. The middle term — the problem — is the one students skip, and it is the one that makes the question worth answering. The problem is the answer to “so what?”

Coach’s Note — If you cannot state your work as a question that could come back “no,” you do not have a research question. You have a plan to confirm what you already believe. That is not research; it is decoration.

2.2 — Choosing Your Domain (and Committing to It)

This is the semester-defining choice, and it happens now. You pick one CS domain and you stay in it through the symposium. The menu, from the course design:

• AI / ML methods
• Cybersecurity
• Networking
• Software engineering
• Databases
• Human-computer interaction (HCI)
• Computer graphics
• AR / VR
• Robotics
• Cloud / distributed systems
• High-performance computing (HPC)
• Computer architecture
• Data science

Whatever you pick, your charge is the same: investigate how modern AI advances that domain. You are not (necessarily) doing core AI research. You are doing applied AI research — AI as the method, your domain as the territory. To seed your thinking, here are the worked anchor examples this course uses, each tied to a real line of work you can read this week:

How to choose well — three filters, in priority order:

1. Access. Can you get the data, the compute, and the baseline? A brilliant question you cannot run is a worse choice than a good question you can. Browser-only, no-GPU? Favor questions answerable with API calls, public benchmarks, and CPU-scale evaluation. (Appendix A’s free path — Colab, Codespaces, Overleaf — is built for exactly this.)
2. Genuine interest. You will read fifty papers in this domain. Pick the one whose papers you do not resent.
3. A live edge. A domain where AI is currently changing the answers has gaps that are fresh, not picked-over.

Coach’s Note — Do not pick “AI/ML methods” as your domain just because the whole course is about AI. That puts you in competition with frontier labs and ten thousand grad students. Pick a domain and bring AI to it. The networking student who knows congestion control has an edge an AI generalist does not.

2.3 — What a Gap Actually Is

A gap is not “nobody has done X.” That is almost never true, and you cannot prove a negative by feeling. A gap, operationalized, is Move 2 of Swales’ CARS model — establishing a niche — and it shows up as a cited limitation or absence in prior work. You establish your territory (Move 1: this area matters and here is what’s known), then you carve your niche (Move 2: but here is the specific thing that’s missing or wrong), then you occupy it (Move 3: so I do this).

In applied AI, gaps reliably appear in five shapes. Learn to spot them:

The single richest vein for a practicum student is the Future Work section. Authors of strong papers tell you what they did not do. Mine five recent papers’ Future Work, and you will have five candidate problems by Wednesday — each one backed by the very authors who’d be your reviewers.

The second-richest vein, and one most students never open: the OpenReview record. Top venues (ICLR 2026, NeurIPS 2026, ICML) run open peer review on OpenReview. You can read the reviewers arguing with the authors. Every “the authors did not evaluate…” / “this is only shown on…” / “I am concerned that…” is a gap that the field’s own gatekeepers have already certified as a gap. Read one full thread and tabulate every objection. That is gap analysis on hard mode, and it is the most honest signal you will find.

Why is OpenReview so much stronger than your own intuition about what’s missing? Because a reviewer who writes “this is only evaluated on one benchmark” is a domain expert staking their professional credibility on the claim that the gap is real and worth caring about. Your hunch that a gap exists is worth one vote — yours. A reviewer’s stated objection is worth the field’s. When the two agree, you can mark the gap confirmed and move on. When they disagree — you thought something was open and three reviewers treat it as solved — you just saved yourself a semester. Either way, the OpenReview thread is doing for free the thing your literature review will otherwise cost you weeks to do.

Coach’s Note — “Gap exists” is the easiest claim in research to assert and the hardest to verify. Back every gap with a cited absence, not a hunch. If your evidence for the gap is “I couldn’t find anything,” you have not yet done the literature search (that’s Chapter 3) — so flag it as unconfirmed and go prove it.

A worked example, because the abstract version slides off the brain. Suppose your domain is databases and you read a 2024–2026 text-to-SQL paper that reports strong execution accuracy on the Spider benchmark. The lazy take: “text-to-SQL is solved, no gap.” Look closer. Three of the five gap shapes are sitting right there. Narrow benchmark: it’s only Spider — does it hold on a messier, real schema? Missing baseline: it never compares query-plan latency against the database’s own cost-based optimizer, only output correctness. Untested transfer: it’s English-only — does it survive a non-English schema? Any one of those is a publishable practicum question, and the authors handed you all three by what they didn’t measure. The gap was never “nobody studies text-to-SQL.” The gap is the specific, unmeasured cell in the table.

2.4 — From Observation to Question

Where do questions come from? Two engines, and good researchers run both.

Academic-driven (top-down). You read the literature, find the cited gap, and ask the next question. This is the CARS engine. It produces work that reviewers recognize as a contribution because it sits in a conversation they’re already having.

Industry-driven (bottom-up). You hit a real, annoying problem — your team’s CI flags too many false-positive security alerts; your VR app makes a quarter of testers nauseous — and you ask whether it’s general. This is where a lot of the best applied-AI work starts, because the problem is demonstrably real before you prove it’s novel.

The conversion ritual — observation to question — is four steps:

1. State the observation plainly. “Our LLM code reviewer approves PRs that introduce SQL-injection bugs.”
2. Generalize it. “Do current LLM code reviewers systematically under-detect a specific class of security bug?”
3. Make it measurable. Name the thing you would count. “…measured as recall on OWASP-Top-10-labeled defects in SWE-bench-Verified patches.”
4. Make it comparative. Research is comparison. “…compared to a static analyzer baseline and to a human reviewer.” A question with no comparison has no result.

You will use AI for step 1→2 — it is genuinely good at generalizing an anecdote into a category. You will not trust it for the claim that the gap is open. That is step 2.5, and it lives in Chapter 3.

A note on which engine to run. Academic-driven questions are safer for a practicum — they sit in a conversation reviewers already recognize, so “why does this matter?” answers itself. Industry-driven questions are riskier and often better — the problem is provably real, but you have to do the extra work of showing the field cares about your specific instance of it, not just your team. The strongest practicum questions run both engines: a real, observed annoyance (bottom-up) that you can also tie to a cited gap in the literature (top-down). When the anecdote and the citation point at the same hole, you’ve found something.

Coach’s Note — Don’t fall for the “I’ll just pick whatever’s trending” trap. A question chosen because the topic is hot, with no observation and no cited gap behind it, has no spine — and you’ll feel its absence in Week 8 when you can’t answer a reviewer’s “why this?” The hottest topic with no real gap is a worse bet than a quiet corner with a clear one.

2.5 — Scoring a Question: FINER and Heilmeier

You will generate ten questions this week. You need a way to kill nine of them on purpose. Two classic instruments do the killing. Use both.

FINER — five properties of a good research question:

The Heilmeier Catechism — George Heilmeier’s questions for any research program, the sharpest “so what?” filter ever written. The first three are the ones that hurt:

1. What are you trying to do? (Say it in plain language — no jargon.)
2. How is it done today, and what are the limits of current practice?
3. What is new in your approach, and why do you think it will succeed?
4. Who cares? If you succeed, what difference does it make?
5. What are the risks? How much will it cost? How long will it take?
6. What are the mid-term and final “exams” to check for success?

If you cannot answer #1 without jargon, your question is foggy. If you cannot answer #4, your question is irrelevant — kill it. Run every candidate through the scoring template in code/question-scorer.csv and watch the field of ten collapse to two or three.

Two failure modes the scorer is built to catch:

• Too broad / not finishable. “How can AI improve cybersecurity?” fails F and is not a question — it’s a department. Narrow until it bleeds.
• Not falsifiable. “Is deep learning useful for networking?” cannot come back “no” in any meaningful way. Rewrite until a specific answer is at stake.

Coach’s Note — Heilmeier #2 — “what are the limits of current practice?” — is your gap, restated. If you can answer #2 with a citation, you have done real gap analysis. If you answer it with “I think people just…,” you haven’t read enough yet.

2.6 — Scoping: The Goldilocks Problem

Almost every first question is wrong-sized, and it’s almost always too big. “How can AI improve cybersecurity?” is a five-year research program for a lab, not a one-semester practicum for one person with a browser. The opposite error is rarer but real: a question so narrow that the answer is obvious or uninteresting (“Does GPT-4 return valid SQL for SELECT *?” — yes, who cares). You want the middle: small enough to finish, large enough to matter.

The practical test for “too big”: can you name the single experiment that would answer it, with one metric, one benchmark, and one baseline? If answering your question requires three experiments and a literature review of its own, it’s a thesis topic, not a practicum question — carve one experiment out of it. The narrowing moves, in order:

Five narrowing moves turned a department into a runnable experiment. The last two — comparison and metric — are the ones that convert a topic into a question, because they’re what makes a result possible.

Coach’s Note — When in doubt, narrow. A finished, honest answer to a small question is a contribution. An unfinished gesture at a big one is nothing. You can always widen in the Future Work section of the paper you actually publish — that’s literally what it’s for.

2.7 — Using AI to Find Problems (and Where It Lies to You)

AI is a legitimate and powerful brainstorming partner for this week’s work. Used well, it compresses days into hours. Used naively, it ends careers. Here is the honest map.

What it’s good for (use it):

• Generalizing your observation into a problem category (2.4, step 1→2).
• Listing candidate subtopics and candidate benchmarks in a domain you’re new to.
• Rephrasing a vague question into a sharper, comparative one.
• Stress-testing your question: “What’s the strongest reason this question is not novel?”

What it’s deadly at (verify everything):

• Inventing citations. This is the non-negotiable literacy of the semester. Walters & Wilder (2023, Scientific Reports) found 55% of GPT-3.5 and 18% of GPT-4 citations were entirely fabricated; among the real ones, 43% (GPT-3.5) / 24% (GPT-4) had substantive metadata errors. Bhattacharyya (2023, Cureus) found 87% of citations to real works had at least one metadata error. These are 2023-model numbers — treat them as illustrative, point-in-time; newer retrieval/“deep research” agents hallucinate less but still require per-citation verification.
• Asserting that a gap exists. A model will tell you “no one has studied X” with total confidence. It does not know. It is predicting plausible text.
• Confusing fluency for faithfulness. A 2026 study across 172 billion tokens of Q&A (arXiv:2603.08274) found fabrication rises with context length — the longer and smoother the answer, the more room for confident error.

The rule for the whole course, stated once: AI proposes, the literature disposes. Every gap, every citation, every “nobody has done this” gets checked against a real index — dblp, Semantic Scholar, the publisher — before it touches your report. The human is accountable for every claim. Per Appendix C and every major venue: an LLM can never be an author, and “the model did it” is never a defense.

A two-minute drill you will run this week: ask any chatbot for five references on a niche corner of your domain, then verify each one in dblp and Semantic Scholar. Count the fabrications. The number will not be zero, and it will change how you treat every AI output for the rest of your life as a researcher.

Coach’s Note — Disclose your AI use honestly, the way every venue now requires. Assistive use — grammar, rephrasing, brainstorming — generally needs no disclosure. Generative use — text, ideas, code you keep — gets disclosed in your Acknowledgements per the venue (ACM and IEEE both put it there, and IEEE wants you to name the system). Start the habit now, in your portfolio’s running AI-use note.

2.8 — Building the Candidate Set

You owe three domains and ten questions. Don’t generate them at random — generate them systematically, so the report writes itself.

The workflow, executed against the matrix in code/gap-analysis-template.txt:

1. Shortlist three domains from the menu (2.2). For each, write one sentence on why AI is changing it now.
2. For each domain, find one anchor paper (from the table in 2.2 or one you find), and read its Related Work and Future Work.
3. Mine three to four candidate questions per domain from the gap shapes (2.3): a Future-Work item, an untested-transfer idea, a missing-baseline you noticed. Aim for ten total across the three domains.
4. Score all ten in code/question-scorer.csv on FINER (1–5 each).
5. Write the gap-analysis paragraph for your top two: state the territory, the cited niche, and what you’d do.
6. Tag every cited gap confirmed or unconfirmed — honest about what you’ve actually verified versus what you still need to check in Chapter 3.

Don’t aim for ten good questions on the first pass — aim for ten real ones and let the scorer cull. The whole value of generating breadth is that it forces you to look at gap shapes and domains you’d otherwise skip, and the bad questions teach you something: when a question scores a 1 on Novel, ask why you couldn’t tell it was already solved, and you’ll learn where your blind spots in the literature are. The cull is not waste. It’s calibration.

This is portfolio work. Everything you produce this week — the domain notes, the candidate questions, the scoring sheet, the gap writeup — goes into your Git repository and your Zotero library that you stood up in Week 1. And the questions you sharpen here become the first real content of the ACM acmart / IEEE IEEEtran paper you started in Week 1: your introduction’s gap statement and your eventual research-question paragraph are being drafted right now, in rough.

One discipline to start this week and never drop: the honesty ledger. Every claim in your gap writeup gets a row — the claim, its citation, and a status of confirmed or unconfirmed. confirmed means you have personally read the source and it says what you claim. unconfirmed means it’s a hunch, an AI suggestion you haven’t checked, or a gap you believe is open but haven’t proven. This is not bureaucracy; it is the difference between a researcher and a bluffer. The unconfirmed tags become your Chapter 3 to-do list — the exact searches you’ll run to confirm or kill each hunch. A report with honest unconfirmed tags scores higher than one with confident fabrications, every time.

2.x — Interactive Lab: Research Question Workshop

Below this chapter on the site you’ll find the Research Question Workshop — your forge for this week. Open it.

Drop in a vague interest (“AI for databases”) and the Workshop walks you down the topic → problem → question staircase, prompting you for the measurable thing and the comparison at each step. Then it scores your candidate questions live on the five FINER dimensions — feasible, interesting, novel, ethical, relevant — and flags the two failure modes from 2.5: it catches the too-broad / not-finishable question (the one with no edge) and the not-falsifiable question (the one that can’t come back “no”). When it flags a question, rewrite it in place and watch the score move.

Use it to triage your ten candidates fast: feed each one in, keep the two or three that survive all five dimensions, and let the rest die early — that is the whole point. Then carry the survivors into code/question-scorer.csv for the written record your Research Opportunity Report needs. The Workshop is the gym; the report is the meet.

2.9 — “Nothing New Under the Sun”: The Apologetic

Now the hard question this chapter has been circling. Is there truly nothing new under the sun?

Ecclesiastes 1:9 says it flatly: “What has been is what will be, and what has been done is what will be done, and there is nothing new under the sun” (ESV). Open arXiv and you’ll feel the verse in your chest. The torrent never stops. Whatever you thought of, someone seems to have thought of first, and better-funded. If the Preacher is right, what is a researcher even doing?

Read the verse the way it’s meant. Qoheleth — the Preacher — is making a point about human striving under the sun: about the vanity of thinking our cleverness adds something permanent to a closed system, that we are the first to feel what we feel or build what we build. Generations rise and fall; the rivers run to the sea and the sea is not full; “there is nothing new” is a rebuke to pride, not a denial of discovery. It is the antidote to the grad student’s worst sins, which are despair (“it’s all been done”) and vanity (“but my idea is unprecedented”). Both are forms of the same disease: thinking the work is finally about us.

Here is the part that should put steel in your back. The same tradition that says “nothing new under the sun” also says, in the verse that names this whole course: “It is the glory of God to conceal things, but the glory of kings is to search things out” (Proverbs 25:2, ESV). The truth is already there. You are not manufacturing reality; you are uncovering it — searching out what was concealed. Your “novelty” is never ex nihilo, and it was never supposed to be. It is a specific, cited absence you fill in an order that already exists. That is exactly what gap analysis is. Far from killing research, “nothing new under the sun” is its proper humility: you stand on Deuteronomy’s “generations before us,” you add your honest paragraph, and you hand the conversation on.

So when arXiv makes you feel small, let it. The smallness is true and it is freeing. You are not required to be the origin of all things — that office is taken. You are required to search a matter out faithfully: to find the real gap, back it with real evidence, and tell the truth about what you find. That is novelty enough, and it is the only kind on offer to creatures. The student who internalizes this stops chasing the impossible (unprecedented genius) and starts doing the possible and good (honest discovery). It’s also, not incidentally, what reviewers reward.

There’s a sharper edge here too, one that bears directly on the AI thread. The reason a hallucinated citation is misconduct and not just a mistake is that it pretends to a novelty — or a support — that does not exist. It manufactures something out of nothing and presents it as found. That is precisely the sin Ecclesiastes warns against: the vanity of claiming we have made what we have only imagined. When you verify every citation against dblp before it enters your report, you are not just following a venue rule; you are refusing to bear false witness about what is and is not under the sun. Integrity with the literature and humility before it turn out to be the same discipline.

Coach’s Note — “There is nothing new under the sun” and “the glory of kings is to search things out” are not in tension; they are the two rails you ride between. Lose the first and you become the grandiose student who can’t see that their idea was published in 2019. Lose the second and you become the despairing one who never tries. Keep both and you become a researcher.

2.10 — Common Pitfalls

Pitfall: Confusing a topic for a research question. Example: Your “question” is “Transformers for time-series forecasting.” That’s a topic; it has no comparison and no measurable outcome and can’t come back “no.” Fix: Run it through the topic → problem → question staircase (2.1) until a specific, comparative, falsifiable question falls out. The Workshop widget forces this.

Pitfall: Asserting a gap you never verified. Example: “No one has applied RAG to cache-replacement policy” — based on a hunch and a quick ChatGPT answer. Fix: Tag it unconfirmed and treat it as a hypothesis about the literature, not a fact. The real search is Chapter 3. A gap you can’t cite isn’t a gap yet.

Pitfall: Trusting AI-generated citations. Example: Your gap paragraph cites “Chen et al., 2024, Adaptive Query Optimization with LLMs, VLDB” — which the model invented and which does not exist. Fix: Verify every reference in dblp, Semantic Scholar, or the publisher before it enters your report. Remember 55%/18% (Walters & Wilder 2023). ICCV rejects papers with non-existent citations without review; arXiv can ban for a year.

Pitfall: Picking an un-runnable question. Example: “Does fine-tuning a 70B model on proprietary network telemetry beat the baseline?” — but you have no GPU, no telemetry, and no baseline. Fix: Apply the Access filter first (2.2). Reshape toward what you can actually run: API-based, public-benchmark, CPU-scale. Feasible-and-good beats brilliant-and-impossible every time.

Pitfall: The not-falsifiable question. Example: “Can AI help with software engineering?” Yes. Obviously. There’s no stake. Fix: Demand a specific outcome that could fail. Name the metric, the benchmark, and the baseline you might lose to. If no honest result would surprise you, the question is dead.

Pitfall: Changing domains every week. Example: Networking in Week 2, databases in Week 4, robotics in Week 6 — and a literature review built on sand. Fix: Commit in 2.2 and stay. The whole practicum compounds within one domain. Switching resets your portfolio to zero.

Pitfall: Ten questions, all the same question. Example: All ten candidates are “LLMs for SQL optimization” with the verb changed. Fix: Force breadth: three different domains, and within each, different gap shapes (2.3) — one Future-Work item, one untested-transfer, one missing-baseline. Variety is what makes the choice a real choice.

2.11 — Reps

The reps for this week are in the exercises, and they are not busywork — every one moves your project forward toward Friday’s report. A preview of where they take you:

• Rep 1 locks your three candidate domains with a one-line “why AI is changing this now” each.
• Rep 3 mines five Future-Work sections into five candidate problems.
• Rep 5 is the AI-citation drill: ask for five references, verify each in dblp, count the fakes.
• Rep 7 reads one full OpenReview thread and tabulates every reviewer “the authors did not…” objection.
• Rep 9 scores all ten questions in the FINER sheet and kills the bottom seven.

Do them in order; they assemble into the deliverable. And take the on-page Check Your Reps quiz below the chapter before you start — it’s five questions, it’s fast, and it tells you whether the topic/problem/question distinction actually landed.

2.12 — This Week’s Deliverable

This week you produce the Research Opportunity Report — your first graded deliverable, worth 5% of the course grade. Full spec in Project 2.

In short: three candidate domains, ten candidate research questions, a FINER/Heilmeier scoring sheet, and a gap-analysis writeup that backs your top one or two with cited evidence and honest confirmed/unconfirmed tags. It is short, it is sharp, and it is the seed of everything: the proposal at the midterm and the paper at the symposium both grow from the question you sharpen here. Build it in your portfolio repo, draft the gap statement straight into your acmart/IEEEtran skeleton, and disclose any AI assistance honestly.

The grade is weighted hard toward two things: the quality of the gap analysis (a cited absence beats a confident hunch every time) and the honesty of your tags. The Hard tier asks for a one-page Domain Commitment Memo — a judgment call no AI can make for you, because it requires you to weigh your access, your interest, and your honest probability of finishing. That memo is where you stop hedging and commit. Don’t skip it lightly; the student who commits in Week 2 is the student who has a real paper in Week 16.

2.13 — Coach’s Final Word

The hardest part of research is not the experiment. It is having the nerve to point at one specific thing in a vast, intimidating field and say this — this is the matter I will search out. The avalanche on arXiv is designed, by sheer volume, to talk you out of that nerve. Don’t let it.

Remember the two verses that bracket this chapter. “Nothing new under the sun” keeps you humble — you are uncovering, not inventing, and that’s the only honest posture for a creature. “The glory of kings is to search things out” keeps you bold — the truth is already there, concealed, waiting to be found by someone disciplined enough to look. That someone is you, this week, with a scoring sheet and a verified citation.

Pick your domain. Sharpen ten questions until two of them could draw blood. Verify the gaps. Tell the truth about what you’ve actually checked. That’s the rep.

See you on Monday.

Up next: Do the the exercises to build your candidate set, then write the Project 2 Research Opportunity Report. Set up your tools in Appendix A and Appendix B; keep AI honest with Appendix C; look up terms in Appendix D. Previous: Chapter 1. Next: Chapter 3.