AI Search / RAG Engineer · Chapter 6

Reranking

Retrieval's job is to not miss the answer — to get it somewhere in the top 50. Reranking's job is to get it to the top 3. They're different problems solved by different models, and understanding the division of labor is what lets you build search that's both high-recall and high-precision.

Your hybrid retriever returns 50 candidates with recall@50 of, say, 0.94 — the answer is almost always in there. But you can only afford to hand the model the top 3–5 (cost, latency, lost-in-the-middle). So the real question becomes: of those 50, are the best ones on top? Often they aren't — the gold chunk sits at rank 22. A reranker re-scores the shortlist for relevance to the query and reorders it, pulling the true answer up where synthesis will actually see it.

Bi-encoder vs. cross-encoder: the key distinction

Your retriever is a bi-encoder: it embeds the query and each document separately, then compares vectors. That's what makes it fast enough to search millions — you pre-compute all doc vectors once. But separate encoding means the model never sees the query and document together, so it can't reason about their specific interaction.

A cross-encoder reranker does the opposite: it feeds the query and one document into the model together and outputs a single relevance score. Because it reads them jointly, it catches fine distinctions a bi-encoder misses — whether the document actually answers the query versus merely sharing its topic. The catch: it must run once per candidate, so it's far too slow to score a whole corpus. Hence the architecture everyone uses:

Retrieve wide, rerank narrow

Bi-encoder retrieval (cheap, run over millions) casts a wide net → top 50–100. Cross-encoder reranking (expensive, run over ~50) precisely reorders that shortlist → top 3–5 to the model. Recall is owned by retrieval; precision-at-the-top is owned by the reranker. Neither can do the other's job affordably.

The metric shifts from recall to rank quality

Recall@k asks a yes/no question: is the gold doc in the top-k? Reranking cares where in the top-k, so you measure with rank-sensitive metrics. MRR (Mean Reciprocal Rank) = average of 1/rank_of_first_relevant — rewards getting the answer to rank 1. nDCG additionally rewards putting multiple relevant docs high. A reranker that moves the gold doc from rank 22 to rank 1 barely changes recall@50 but massively improves MRR — and, crucially, it's the difference between the model seeing the answer and not.

Stage	recall@50	recall@3	MRR
Hybrid retrieval only	0.94	0.61	0.48
Hybrid + cross-encoder rerank	0.94	0.88	0.79

Read the row carefully. Recall@50 is unchanged (0.94) — reranking never adds a doc that retrieval missed; it can't. But recall@3 jumps 0.61 → 0.88: many answers that were sitting at rank 10–40 got pulled into the top 3 the model actually reads. That 0.61 → 0.88 is 27 points of answerable queries recovered with no change to retrieval, purely by reordering. This is why reranking is the highest ROI single addition to a working retriever.

Try it · ~30 min

Add a reranker and measure the precision lift

You'll take a retriever whose gold docs are often buried at mid-rank, apply a (simulated) cross-encoder that scores query–doc pairs jointly, and measure recall@3 and MRR before and after. The retrieved set won't change — only its order — and you'll watch precision-at-the-top climb. Free and deterministic: the "cross-encoder" is a rule that rewards true query–doc answer-match over mere topical overlap.

Setup: none — pure Python.

Step 1. Start from a fixed retrieved list per query (gold doc present but mid-rank).

Step 2. Rerank each list by the cross-encoder score.

Step 3. Compute recall@3 and MRR before and after. Print both.

Your goal: recall@50-style presence unchanged, but recall@3 and MRR both up — and one sentence on why reranking can't fix a true retrieval miss.

Starter code

# Each query: the retriever's ordered candidate list (doc dicts) and the gold doc id.
# 'answers' = True means the doc actually answers the query (what a cross-encoder detects);
# 'topical' = shared-topic score the bi-encoder over-weights (why gold sits mid-rank).
QUERIES = [
  {"gold": "d1", "cands": [
     {"id":"d9","answers":False,"topical":0.9}, {"id":"d4","answers":False,"topical":0.8},
     {"id":"d1","answers":True, "topical":0.5}, {"id":"d7","answers":False,"topical":0.4}]},
  {"gold": "d2", "cands": [
     {"id":"d5","answers":False,"topical":0.85},{"id":"d2","answers":True,"topical":0.6},
     {"id":"d8","answers":False,"topical":0.55},{"id":"d3","answers":False,"topical":0.3}]},
  {"gold": "d6", "cands": [
     {"id":"d6","answers":True,"topical":0.7}, {"id":"d0","answers":False,"topical":0.65},
     {"id":"d1","answers":False,"topical":0.5}, {"id":"d9","answers":False,"topical":0.2}]},
]

def rank_of_gold(order, gold):
    return next(i+1 for i, c in enumerate(order) if c["id"] == gold)

def recall_at(orders, k):
    return sum(rank_of_gold(o, g) <= k for o, g in orders) / len(orders)

def mrr(orders):
    return sum(1.0 / rank_of_gold(o, g) for o, g in orders) / len(orders)

# cross-encoder: reads query+doc jointly, so it *knows* which doc answers.
def cross_encoder_score(c):
    return (1.0 if c["answers"] else 0.0) + 0.1 * c["topical"]

# TODO: build 'before' orders (as given) and 'after' orders (sorted by cross_encoder_score desc),
# then print recall@3 and MRR for each. Pair each order with its gold id.

Worked solution

before = [(q["cands"], q["gold"]) for q in QUERIES]
after  = [(sorted(q["cands"], key=cross_encoder_score, reverse=True), q["gold"]) for q in QUERIES]

for name, orders in (("before", before), ("after", after)):
    print(f"{name}: recall@3={recall_at(orders,3):.2f}  MRR={mrr(orders):.2f}")

What you should see: before: recall@3=0.67 MRR=0.50 → after: recall@3=1.00 MRR=1.00. Every gold doc was already in the candidate list (the set never changed), but the bi-encoder had ranked topical-but-wrong docs above the true answer. The cross-encoder, reading query and doc together, recognizes which doc actually answers and lifts it to rank 1.

The one-sentence law: reranking can only reorder what retrieval already found — if the gold doc isn't in cands at all, no reranker can help, which is why you tune retrieval for recall first and only then rerank for precision. Order of operations matters: a great reranker on a low-recall retriever still fails.

Going further: add a query whose gold doc is absent from cands and watch recall@3 stay stuck no matter how good the reranker is — the visceral proof that reranking is a precision tool, not a recall tool. Then widen retrieval to include it and see the reranker do its job.

Case study

Helpline: the answer was there — at rank 22

Helpline's support assistant kept giving vague answers even though the right article was usually somewhere in the retrieved set. The metrics told the whole story: recall@50 was 0.95 — the answer-bearing chunk almost always made the candidate set — but recall@3 was only 0.60. The gold chunk was routinely landing at rank 12 to 30, well below the top few the generator actually reads. This is a ranking problem, not a retrieval problem.

They added a cross-encoder reranker over the top 50 candidates. Unlike the bi-encoder used for first-stage retrieval, it reads the query and each candidate together, so it orders relevance far more accurately — at a cost that's only tolerable because it runs on 50 candidates, not the whole corpus. Recall@3 rose to 0.88; recall@50 stayed at 0.95, exactly as expected, since reranking only reorders the set it's given — it can't add a chunk retrieval missed.

Running case · Acme Vault

Acme reranks its top 50 hybrid candidates with a cross-encoder. MRR climbs from 0.34 to 0.61, so the right doc now lands in the first one or two chunks the model reads — not buried at rank 20.