Project P8 — The Fact-Check Gauntlet (Midterm)

“Beloved, do not believe every spirit, but test the spirits to see whether they are from God, for many false prophets have gone out into the world.” — 1 John 4:1 (ESV)

Chapter: 8 — Understanding AI’s Limits Due: End of Week 8 (the midterm week) Submit: A link to a shared folder (Google Drive / OneDrive / Dropbox) or a single combined PDF. Non-technical — no GitHub, no code. See Appendix A. Two parts: (1) the auto-graded midterm quiz on Canvas, covering Weeks 1–8 — take it separately, closed-book, per your instructor’s instructions; and (2) this applied project, the Fact-Check Gauntlet. AI policy — MIDTERM (closed-AI for the audit): The heart of this project is your verification, done without asking an AI to do the checking for you. You may use a search engine and trusted primary sources to verify facts, but the catch-and-classify work must be your own eyes and judgment. You may use AI only in the clearly-marked Medium tier. When that tier does open the door to AI, use BoodleBox — sign in with your Concordia account at box.boodle.ai (any free assistant works off campus) — and treat its output with the same suspicion: a vetted, private, FERPA-compliant platform is the right place for your data, but no tool, BoodleBox included, is exempt from verification. Every deliverable ends with an honest note about what you used.

The Setup

Halfway through the course, here’s the truth: everything you’ve learned to do with AI is worthless if you can’t tell when it’s lying to you with a straight face. So the midterm isn’t “use AI faster.” It’s the opposite — it’s a gauntlet that proves you can catch AI when it’s confidently wrong.

You are the final reviewer at Ashford Consulting. Your colleague Dana drafted an AI-generated briefing for a slide she’s presenting to your leadership team on Thursday, and it looks fantastic — fluent, structured, authoritative. She’s asked you to check it before it goes up on the screen with the firm’s numbers behind it. Somewhere in that polished briefing are fabricated statistics, fake citations, an out-of-date “fact,” a confidently wrong claim, and a subtle bias. Your job is to find every one before Thursday embarrasses the room. This is what the whole first half of the course was building toward: test the spirits. Fluency is not truth, and confidence is not correctness.

Setup (the starter)

This chapter ships two starter files in code/:

• code/ai-answer-with-errors.txt — the polished, confident AI-generated memo with multiple seeded errors of different types. Your gauntlet.
• code/verification-checklist.txt — the verification method you’ll apply and then adapt into your own reusable playbook.

Copy both into your project folder.

Learning Targets

You will demonstrate that you can:

• Catch hallucinations — find fabricated facts, statistics, and citations in fluent AI output.
• Classify the failure — name each error’s type using the checklist’s six-part taxonomy (hallucinated fact / fabricated citation / fake quote / outdated–training-cutoff / plausible-wrong number / biased frame).
• Verify against sources — confirm or refute a claim using a trusted primary source, not another AI.
• Judge fitness for use — decide whether a piece of AI output is safe to send, needs revision, or must be thrown out.
• Build a reusable playbook — turn the checklist into your own verification routine for the rest of the course and your career.

Normal Tier

Goal: Run the full gauntlet — catch and classify every seeded error, verify against real sources, and produce your own verification playbook.

Required features

Produce a folder named p8-gauntlet containing:

1. error-log.xlsx (or a table) — the catch. Every error you find in code/ai-answer-with-errors.txt, one row each: the exact quoted text, the error type, the correct fact (or “unverifiable — so cut it”), and how you verified (which source you checked). Aim to find them all; the memo has several of different kinds.
2. fabricated-citation.docx — the deep cut. Pick one fabricated source in the memo — the “McKinsey Global Institute” study is the flagship, and it is not the only one — and show your work proving it’s fabricated (you searched for it and it doesn’t exist / the real source says something different). Fake citations are the most dangerous error because they look like verification.
3. The verdict. For the memo as a whole, write your reviewer’s decision: send / revise / reject, with a one-paragraph justification. If “revise,” list exactly what must change.
4. my-playbook.txt — your reusable verification routine. Adapt code/verification-checklist.txt into your own step-by-step playbook: the checks you’ll run on any important AI output, in order, plus your personal “would I stake my name on this?” test. You’ll use this the rest of the course.
5. When-not-to-use-AI. Name two tasks from your own work where, having done this gauntlet, you’d choose not to use AI at all — and say why.
6. Honest usage note — what you used to verify, and confirmation you did the catching yourself.

Normal-tier rubric (out of 100)

Medium Tier (+up to 25% extra credit)

Pick one or more:

1. Turn the tables (AI-allowed here). In BoodleBox, open a multi-bot chat and @-mention two different models (as of 2026), then ask both to fact-check the same memo. Compare their catches to yours and to each other: what did each miss, where do they disagree (your dig sites), and did either add a false “error”? Cross-checking one model against another is a genuine hallucination defense — but an AI checking an AI still has its own failure mode, so document it, and remember BoodleBox’s own output is not exempt. (Off campus, two free tools do the same job.)
2. Generate your own gauntlet. In BoodleBox, pick a model and prompt it to write a confident memo on a topic you know well, then catch its errors. Models hallucinate most where you can verify best — your own field.
3. The bias pass. Beyond factual errors, identify where the memo’s framing is skewed (loaded words, one-sided emphasis) and rewrite one passage neutrally.

Hard Tier (+up to 25% additional extra credit)

This is the judgment the machine cannot make for you — and on the midterm, it carries the most weight.

Write trust-standard.docx — a one-page standard, addressed to Ashford’s team, answering: what level of verification does a piece of AI output need before it can go to a client, and who decides? You must:

• Tier it. Not everything needs the same scrutiny. Define levels (e.g., internal brainstorm vs. client-facing claim vs. a number in a contract) and the verification each requires.
• Name the accountable human at each level — the person whose name is on it.
• Draw the never-line. What must never be sent on AI’s say-so alone?
• Ground it in the spine rule — you choose the tool; you own the verdict — and in this week’s lesson that AI is an assistant, not an authority.

A standard that says “always verify everything” is impractical; one that says “trust the AI, it’s usually right” is negligent. The grade is in drawing a usable line and defending it.

Submission

One shared folder or a combined PDF; submit the link per Appendix A:

p8-gauntlet/
  error-log.xlsx        (or a table)
  fabricated-citation.docx
  my-playbook.txt
  verdict.docx          — send/revise/reject + the two "when not to use AI" cases
  trust-standard.docx   (Hard tier)

Remember: the auto-graded midterm quiz on Canvas is a separate deliverable. Do both.

Hints (Read Before You Begin)

• The memo is designed to fool you. It’s fluent on purpose. Slow down — the errors hide behind good formatting.
• A fake citation looks exactly like a real one. The only way to catch it is to actually try to find the source. If you can’t find it, that’s your answer.
• Verify with a primary source, not another chatbot. An AI confirming an AI is not verification.
• “I couldn’t confirm it” means cut it. Unverifiable is not innocent-until-proven-guilty in a client memo.
• Your playbook is the real prize. The errors are practice; the routine you build is what you keep.

What Mastery Looks Like (Beyond the Rubric)

A mastery submission proves you can be handed something beautiful and confident and still find the rot — every seeded error, including the subtle confident-wrong one and the citation that doesn’t exist. Your verdict is decisive. Your playbook is something you’d actually use. And your Hard-tier trust standard draws a line a real firm could adopt: tiered, owned, with a clear never-line. Half the course was learning to use AI. This week proves you can judge it — which is the only thing that makes the using safe.

Coach’s Note — This is the hinge of the whole book. Everything in Phase 2 — images, voice, agents, delegation — gets more powerful and therefore more dangerous. The only thing standing between that power and a real mistake is a person who can look at fluent, confident output and calmly say “prove it.” That person is what this midterm certifies. Be that person.

When You’re Done (a short checklist)

• Every seeded error found, quoted, typed, and corrected with a real verification source.
• The fabricated citation proven fake with shown work.
• A clear send/revise/reject verdict with justification.
• A personal, reusable verification playbook written.
• Two honest “when not to use AI” cases.
• Usage note confirms you did the catching yourself.
• The Canvas midterm quiz taken separately.
• (Hard) trust-standard.docx draws a tiered, owned, defensible verification line.

A theological footnote. John does not tell his readers to believe nothing — he tells them not to believe every spirit, but to test them, “for many false prophets have gone out into the world.” The world was not short on confident voices; it was short on tested ones. An AI is not a false prophet — it has no intent — but it produces the same hazard: fluent, authoritative speech that may simply be untrue, and no error message will warn you. The discipline Scripture commands is exactly the one this midterm grades: not credulity, not cynicism, but testing — holding a confident claim up against the truth before you let it move you. You will carry this the rest of your career. Test the spirits. Then sign your name.

See you next week.