Hallucinations: measure them before you fix them

Every client asks how to eliminate hallucinations. The useful question is how many there are today, and on which specific questions. An assistant that hallucinates on five percent of queries might be acceptable; one that hallucinates on fifty percent is not. You need numbers before you fix anything.

Without a baseline, any improvement is an impression. With 200 labeled questions, it's a percentage you can defend in a steering committee. We built evaluation sets for each client, labeled good and bad answers, and tracked accuracy over time.

The pattern was consistent: accuracy plateaus quickly with prompt tweaks. After the first ten or fifteen iterations, further changes move the needle in single digits. There are limits to what the prompt can fix.

In systems with solid retrieval, most errors come from contradictory documents rather than pure invention. The company's manual says one thing about a process, and an email from last year says another. The model picks one and sounds confident about it. It's not hallucinating; it's resolving ambiguity the wrong way.

Fixing the source resolved more cases than any prompt tweak in three out of four projects. Consolidating contradictory documents, removing outdated guidance, clarifying edge cases — that kind of work paid off. The model was only as good as the documents it retrieved.

The lesson is hard to pitch: the AI work is forty percent retrieval, forty percent source data, and twenty percent the model itself.