Reranking: the cheapest improvement we've measured

Retrieving twenty candidates and reordering them with a small model raised precision eleven points. The cost was less than the price of a coffee per thousand queries. No part of that calculation makes sense until you build it and measure it.

Vector search alone returns things that are similar, not things that are correct. In technical documentation that difference matters. The document about brake systems and the document about electrical systems might both mention circuit. The similarity score ranks them equally. A human would know which one answers your question.

The reranker is a small model trained to compare a question and a document and score whether the document answers the question. It's slow compared to vector search, but cheap enough to run on a shortlist.

The reranker sees the question and the chunk together, which is exactly the comparison missing earlier in the pipeline. Vector search has only the question and the document separately. The reranker has context.

It's the first change we try when a retrieval system is underperforming. It works more often than switching to a bigger embedding model, and it costs less. A few lines of code plus a ranking call and the system usually improves noticeably.

In three out of four client retrieval audits this year, reranking was the first win. Measuring the baseline and running the experiment takes a day. The return justifies the investment immediately.