Query Understanding & Transformation

So far you've improved the index and the retriever. But half of retrieval failures are caused by the query — it's too short, too vague, or worded nothing like the document that answers it. Rewriting the query before you search is often the cheapest large recall win available.

Retrieval quietly assumes the user's words resemble the document's words (lexical) or its meaning (dense). Real queries break that assumption constantly: "it broke after the update" (which product? which update?), "cheapest plan" (versus a doc that says "pricing tiers"), or a multi-part question whose two halves live in different documents. The fix is a small LLM pass that transforms the query into something retrievable. Four techniques, in rising order of power and cost:

Why multi-query works: coverage through variation

A single query is a single point in retrieval space — it catches the docs near that phrasing and misses docs phrased differently. Generate three paraphrases of "how do I stop the app crashing on launch" — "resolve startup failures," "fix boot crash," "application won't start" — retrieve for each, and union the hits. Each phrasing lands near different documents, so the union recovers answers no single phrasing would. It's the same complementary-errors logic as hybrid (Ch. 5), applied to the query side instead of the retriever side.

HyDE: search with the answer, not the question

Hypothetical Document Embeddings exploits a subtle asymmetry: a question and its answer often share few words, but two answers to the same question look very alike. So instead of embedding the terse question, you ask the LLM to draft a plausible answer (it doesn't need to be correct — just shaped like the real document), embed that, and retrieve. The hypothetical answer sits much closer in embedding space to the real answer document than the question ever did. It costs one extra LLM call and can lift recall notably on corpora where questions and answers are worded very differently.

Starter code

from rank_bm25 import BM25Okapi
import re
def toks(s): return re.findall(r"[a-z0-9]+", s.lower())

CORPUS = [
  "Resolving application startup failures and boot crashes",   # 0
  "Reducing monthly subscription costs on the Basic plan",     # 1
  "How to reset a forgotten account password",                 # 2
  "Troubleshooting slow query performance in the database",    # 3
  "Enabling two-factor authentication for your login",         # 4
]
# (raw query, [paraphrases the LLM would generate], gold doc id)
CASES = [
  ("app won't launch", ["application startup failure","boot crash fix","program won't start"], 0),
  ("make it cheaper",  ["reduce subscription cost","lower monthly price","cheapest plan"],      1),
  ("can't log in, forgot my code", ["reset forgotten password","account recovery","login help"], 2),
]
bm25 = BM25Okapi([toks(d) for d in CORPUS])
def ranks(q):
    s = bm25.get_scores(toks(q)); return sorted(range(len(CORPUS)), key=lambda i:s[i], reverse=True)

# TODO single: recall@3 using ranks(raw)[:3]
# TODO multi:  union of ranks(p)[:3] for each paraphrase p; hit if gold in the union
#      (union recall@3 = fraction of CASES whose gold is in the unioned set)

def single_hit(raw, paras, gold): return gold in ranks(raw)[:3]

def multi_hit(raw, paras, gold):
    hits = set()
    for p in paras:
        hits.update(ranks(p)[:3])
    return gold in hits

sr = sum(single_hit(*c) for c in CASES) / len(CASES)
mr = sum(multi_hit(*c)  for c in CASES) / len(CASES)
print(f"single-query recall@3: {sr:.2f}")
print(f"multi-query  recall@3: {mr:.2f}")