A July 2026 study by Capraro, Marcoccia and Quattrociocchi found access to AI advice cut people's willingness to say "I don't know" from 44% to 3%, dropped accuracy from 27% to 9%, and pushed confidence to 76%. Daniel Kahneman's Thinking, Fast and Slow explains why: confidence is a feeling built from story coherence, not evidence, and a fluent AI answer supplies exactly that. My fix is a four-step premortem run before trusting any AI call.

I run most consequential calls through Claude or ChatGPT before I commit to them — pricing changes, a hire, whether to kill a product line. Until three weeks ago I assumed the risk was the obvious one: the model might just be wrong. What a study published this month actually measured is scarier. The risk isn't that AI is wrong. It's that having AI in the room makes you stop noticing when you don't know something yourself.

The study is from Valerio Capraro (University of Milano-Bicocca), Chiara Marcoccia (École Normale Supérieure) and Walter Quattrociocchi (Sapienza University of Rome), posted to PsyArXiv in July 2026. They deliberately used questions AI models get wrong — visual trivia like the colour of a team's uniform in a specific film — and ran them through Step 3.5 Flash, a model that was usually incorrect on this set, precisely so nobody could call the effect "sensible delegation to a reliable tool." Without AI access, people said "I don't know" 44% of the time and were right 27% of the time they answered. With AI access, "I don't know" collapsed to 3%, accuracy fell to 9%, and stated confidence rose to 76%. Some participants who would have answered correctly alone asked the AI anyway and got it wrong. Money didn't fix it either — a cash incentive nudged "I don't know" back up to 8% and accuracy to 16%, both still far below the no-AI baseline. Wharton researchers gave the broader pattern a name earlier this year: cognitive surrender — accepting a wrong AI answer roughly 80% of the time while feeling more confident than people who never asked at all.

Daniel Kahneman diagnosed the mechanism a decade before any of this existed, and it's the single most useful idea in Thinking, Fast and Slow for anyone using AI to decide things: "the confidence people have in their beliefs depends mostly on the quality of the story they can tell about what they see, even if they see little." He called it WYSIATI — what you see is all there is. System 1 doesn't check whether an explanation is complete; it checks whether it's coherent. A clean, structured, three-paragraph answer from an AI model is maximally coherent by design — that's what these models are optimized to produce. So the same cognitive-ease effect Kahneman found with clear fonts and rhyming statements fires on a well-formatted Claude answer: it feels true because it reads easily, not because anyone checked it. System 2, which Kahneman calls "lazy," rubber-stamps the fluent story instead of interrogating it. Automation bias and WYSIATI are the same failure wearing different clothes.

What changed for me isn't "use AI less." It's that I stopped trusting my own sense of how carefully I'd evaluated an AI answer, because that sense is exactly what the research says goes first. I now run a short protocol on any AI-assisted decision above a real threshold — anything irreversible, or costing more than roughly a week to undo. I call it the Premortem Override, adapting Kahneman's own premortem technique to sit between the AI's answer and my decision instead of between my plan and its execution:

  1. Write your own answer first. Before reading the AI's output, write one sentence with your own view and a rough confidence percentage. This is the only step that protects the 44%-to-3% collapse — you can't lose a judgment you already committed to paper.
  2. Premortem the AI's answer, not your own. Imagine it's six months later and the AI's recommendation failed. Write down the two most likely reasons why, in your own words, before acting on it.
  3. Force a source, not a summary. Ask the model directly what it's drawing on. If it can't point to something checkable, treat the fluency as manufactured confidence, not evidence.
  4. Sit on anything irreversible for 24 hours. The incentive experiment in the July study barely moved the numbers — willpower and stakes don't fix this. Only a structural pause does.

I'll name the failure honestly: I don't run this on every decision, and I've skipped it under time pressure and paid for it — once on a vendor pricing call where I took a Claude-generated comparison at face value, signed, and only caught the error a week later reading the actual contract terms the model had summarized wrong. The protocol also doesn't help with high-frequency small decisions; you can't premortem fifty Slack replies a day, and I don't try to. It's for the handful of calls per month that are actually expensive to get wrong.

The uncomfortable finding in the July 2026 study isn't really about AI accuracy. Step 3.5 Flash being wrong on movie trivia was the point — the researchers wanted a model humans shouldn't trust, and people trusted it anyway. That's a human problem AI happens to trigger. Kahneman's answer, twelve years before generative AI existed, was to budget deliberate System 2 time for the decisions that matter, because your own sense of "I checked this carefully" is not reliable evidence that you did.

Sources: The Register, "Using AI makes people less likely to admit they don't know something," July 19, 2026; Capraro, Marcoccia & Quattrociocchi, PsyArXiv preprint, July 2026; TheNextWeb, "AI advice made people three times less accurate but twice as confident," July 2026; TheNextWeb on Wharton's "cognitive surrender" research; Daniel Kahneman, Thinking, Fast and Slow (2011).

Related: How to Find Your Passion · Best Self-Improvement Books · How to Make Better Decisions · What University Will Not Teach You