Hallucination

A hallucination is AI output that’s wrong and delivered with exactly the confidence of output that’s right — an invented citation, a policy that doesn’t exist, a total that was never in the document. It isn’t a bug awaiting a patch; it’s a property of how these systems work. They produce the most plausible answer, and plausible and true are different tests.

Why it matters: The danger isn’t that AI is sometimes wrong — everything is sometimes wrong. It’s that the wrongness carries no signal: no flag, no hedge, no drop in fluency. The same model that summarizes a fifty-page contract flawlessly will add up the numbers inside it and hand you a wrong total with complete serenity. The defense isn’t better prompting; it’s design — verification wherever output can be checked, a human wherever it can’t, and no unverifiable AI claim ever shipped into something that matters. One airline has already been to court over what its chatbot invented, and lost.