In partnership with
Are you looking to get in front of 340k+ AI readers? Sponsor AI Signal. |
Was this email forwarded to you? Subscribe here! |
Your AI makes things up
and it doesn't know it's doing it
The risk isn't the obvious wrong answer you'd catch anyway.
It's the invented fact sitting quietly inside a smart, mostly-correct paragraph, where everything around it is true so it reads as true too.
What's risk:
A reference librarian, Hana Lee Goldin, wrote up a client report that arrived with 47 sources at the bottom. It looked bulletproof. She started checking, and 31 of the 47 did not exist.
Real-sounding journals, real author names, titles that sounded right, and studies that were never written. This isn't a ChatGPT problem. It happens in Gemini, Claude, and Grok too, because of how all of them work.
My Take:
The risk isn't the obvious wrong answer you'd catch anyway. It's the invented fact sitting quietly inside a smart, mostly-correct paragraph, where everything around it is true so it reads as true too.
The reason is boring and important: the model isn't looking anything up, it's guessing the next word that sounds right, and when it doesn't know something it keeps guessing in the same confident tone. So a real fact and a made-up one come out looking identical.
The Test:
I ran the same niche question two ways. Cold, it invented a source. With one added line telling it to say "I don't know" if the answer wasn't in front of it, it told me straight that it wasn't sure.
Same model, same question, opposite honesty.
Watch the 1-min video 👇 for Before and After my own test. 2 Clicks = Big Difference!
is there a seahorse emoji?
There's a funny version of this too.
Ask most models "is there a seahorse emoji?" and they'll say yes with full confidence, then spiral into random fish emojis when you ask them to show it. There is no seahorse emoji. Harmless when it's an emoji. Less harmless when it's a citation in your client deck.
Your voice is the way you see the world, the opinions you actually hold, and the words you naturally reach for.
The format is what changes: the hook, structure, length, rhythm, and entry point.
Most people tell AI what to write.
The better move is to tell AI what must stay the same and what has to change.
The IT strategy every team needs for 2026
2026 will redefine IT as a strategic driver of global growth. Automation, AI-driven support, unified platforms, and zero-trust security are becoming standard, especially for distributed teams. This toolkit helps IT and HR leaders assess readiness, define goals, and build a scalable, audit-ready IT strategy for the year ahead. Learn what’s changing and how to prepare.
Deeper Cut
🔍 ChatGPT, make it grade its own confidence
Most people use ChatGPT and treat every sentence it gives back as equally solid. That's the floor, not the ceiling.
Ask it to rate itself. Add this to any prompt where facts matter:
“For each main claim, add a confidence label in parentheses, high, medium, or low. At the end, list anything you were unsure about or couldn't find.”
Now the answer comes back sorted for you. Instead of re-checking the whole thing, you check the two or three claims it tagged "low."
When I ran this on a competitor summary, it flagged a pricing number as low confidence. That number turned out to be a year old. I'd have pasted it otherwise.
Where it breaks: the labels are the model's own guess, so a confident hallucination can still get tagged "high." Treat the labels as a triage tool, not a guarantee.
That works for LinkedIn because it teaches the idea in public.
It gives the reader something they can nod at, save, or share.
Use This Tuesday
One line that removes most of the made-up confidence
in under five minutes.
By default the AI would rather invent an answer than admit it doesn't know. So take that option away.
Paste your source first, then ask your question with this attached:
Use only the text I gave you. If the answer isn't clearly in it, say "I don't know based on this" instead of guessing.
Context: [paste your doc, email thread, or pricing page]
Question: [your question]
One common mistake: people ask from memory and hope the model remembers. Hand it the actual document and it's reading, not guessing.
Big difference, and you barely did anything.
What done looks like: an answer you can trace back to a line in your own source, or a clean "that's not in here" instead of a confident invention you'd have to catch later.
Going deeper: if you're already grounding answers, run them through NotebookLM instead, so every claim comes back with an inline citation you can click before you trust it.
Power User Pattern
⚡ The verify-then-rebuild chain
The pattern: separate the writing from the checking. Let the model draft freely, then force it to fact-check its own claims in a fresh context, then rebuild the answer from only what survived.
Why it works: models are better at catching mistakes than at not making them.
A first pass written to sound good and a second pass built to poke holes are two different jobs, and doing them in the same breath is where buried errors slip through.
How to wire it
Get the first answer normally.
Ask it to extract every factual claim and turn each into a standalone yes/no question.
Open a brand new chat, paste those questions, and tell it to verify each one with search, not memory, with a citation where possible.
Back in the first chat, tell it to rewrite the answer using only the verified claims.
Stack tip: keep step 3 in a separate chat so the original context can't bias the fact-check.
For high-stakes calls, run the same question through a second model and compare where they disagree. The disagreement is the signal.
Forward this to the colleague who pastes AI answers straight into client decks.
The "say I don't know" line will save them one bad citation, which is one too many.
How was today's issue?
Dan Rice · AI Signal · Every Tuesday