AI FactScan1. Who Would You Trust in a Meeting?
Picture a Monday morning cross-functional meeting. The first colleague hesitates and stops to check their notes, but every number is correct. The second speaks smoothly and confidently, with polished charts. Look closely, though, and a few figures use the wrong formula.
When the meeting is moving fast and nobody can recalculate every number on the spot, who would you trust first?
Many people would lean toward the second colleague because they sound more reliable. We make a similar judgment when we read answers from ChatGPT, Claude, or Gemini.
2. A Complete Answer Has Not Necessarily Been Verified
Large language models are good at producing clear, natural writing. That skill can make them sound certain even when they are not.
A model can organize uncertain or incorrect information into polished sentences. Good writing and a confident tone do not show that reliable evidence sits behind the answer.
When fluency and accuracy pull apart, which one do people trust? A communication research team at Radboud University in the Netherlands tested that question with a Lord of the Rings experiment. The paper was published online on October 22, 2026.
3. How Researchers Used The Lord of the Rings to Test AI
The researchers recruited 395 participants. Each person read a scripted exchange between a user and a chatbot. The conversation centered on The Lord of the Rings, and participants had already demonstrated basic knowledge of the story.
The Lord of the Rings contains plenty of characters, places, and plot details with verifiable answers. The team separated fluency from factual accuracy and created four versions:
Natural, complete language with no inserted errors.
Natural, complete language containing incorrect information.
Brief, businesslike language with accurate information.
Brief, businesslike language containing incorrect information.
In this study, “low fluency” meant shorter, more businesslike language. It did not mean broken grammar or nonsense.
After reading the exchange, participants rated how intelligent and trustworthy the AI seemed, how satisfied they were with it, and whether they would use it again.
4. Fluent and Wrong Can Still Sound Smart
More fluent answers made the AI seem more intelligent. Hallucinations lowered those ratings.
Yet the fluent but wrong chatbot received a similar intelligence rating to the concise but correct chatbot. Equivalence and Bayesian tests found no meaningful difference between these two conditions.
Participants thought the two chatbots seemed equally intelligent. Fluency did not erase the effect of errors, but it clearly shaped how people judged the AI's ability.
About 80% did not notice the errors
These participants had already passed a basic Lord of the Rings knowledge screen.
Errors are easy to overlook when they arrive in natural, complete, confident sentences. The study also found that the more intelligent participants thought the AI was, the more likely they were to trust it, feel satisfied with it, and want to keep using it.
5. From Meetings to Chatbots, We Mistake Delivery for Ability
People have always made this kind of judgment. A polished presentation can look well prepared. A confident tone can sound well supported. An orderly explanation can make someone seem deeply knowledgeable.
AI makes that shortcut harder to notice. It rarely stumbles, and it does not look nervous when it is wrong. The user usually sees clean, finished prose.
Speaking well is one skill. Being right is another.
6. When AI Sounds Convincing, Check Three Things
Before using an AI answer in an important decision or report, make three quick checks.
- Which claims can you verify?Mark the names, numbers, research findings, and specific events in the answer.
- Do the cited sources exist?Check the title, author, date, and full URL. Make sure the citation was not invented.
- Does the source support the claim?An accessible link is only the first step. Read the original and see whether it reaches the conclusion AI gave you.
AI FactScan organizes citations from an AI answer and labels their source type and quality, helping you decide which links deserve a closer look. A source grade cannot replace verification. Important claims still need to be checked against the original.
7. Sounding Good and Being Right Are Different Skills
AI is useful, and fluent answers can help us understand unfamiliar topics quickly. Trouble starts when smooth writing feels like evidence and we skip the checks we would normally make.
The next time an AI answer sounds complete and certain, pause for a moment. Look at what it cites before you decide whether to trust it.