Query Understanding & Transformation

You've improved the index and the retriever. Many remaining failures come from the query itself: it's too short, too vague, or worded nothing like the document that answers it. Rewriting the query before searching is often the cheapest large gain in recall.

Retrieval assumes the user's words resemble the document's words (lexical) or its meaning (dense). Real queries break that assumption all the time: "it broke after the update" (which product, which update?), "cheapest plan" when the document says "pricing tiers," or a two-part question whose halves live in different documents. A small language-model step can transform the query into something easier to retrieve. Here are four techniques, roughly from cheapest to most powerful:

Why multi-query helps

One phrasing finds the documents near that phrasing and misses the rest. Paraphrase "how do I stop the app crashing on launch" three ways ("resolve startup failures," "fix boot crash," "application won't start"), retrieve for each and merge the hits. Each phrasing lands near different documents, so together they recover answers no single one would. It's the same idea as hybrid search, applied to the query instead of the retriever.

HyDE: search with a draft answer

A question and its answer often share few words, but two answers to the same question tend to look alike. Hypothetical Document Embeddings (HyDE) takes advantage of that. Ask the model to draft a plausible answer (it doesn't need to be right, only shaped like the real document), embed the draft and search with it. The draft usually lands much closer to the real answer than the question did. It costs one extra model call.

Starter code

from rank_bm25 import BM25Okapi
import re
def toks(s): return re.findall(r"[a-z0-9]+", s.lower())

CORPUS = [
  "Resolving application startup failures and boot crashes",   # 0
  "Reducing monthly subscription costs on the Basic plan",     # 1
  "How to reset a forgotten account password",                 # 2
  "Troubleshooting slow query performance in the database",    # 3
  "Enabling two-factor authentication for your login",         # 4
]
# (raw query, [paraphrases the LLM would generate], gold doc id)
CASES = [
  ("app won't launch", ["application startup failure","boot crash fix","program won't start"], 0),
  ("make it cheaper",  ["reduce subscription cost","lower monthly price","cheapest plan"],      1),
  ("can't log in, forgot my code", ["reset forgotten password","account recovery","login help"], 2),
]
bm25 = BM25Okapi([toks(d) for d in CORPUS])
def ranks(q):
    s = bm25.get_scores(toks(q)); return sorted(range(len(CORPUS)), key=lambda i:s[i], reverse=True)

# TODO single: recall@3 using ranks(raw)[:3]
# TODO multi:  union of ranks(p)[:3] for each paraphrase p; hit if gold in the union
#      (union recall@3 = fraction of CASES whose gold is in the unioned set)

def single_hit(raw, paras, gold): return gold in ranks(raw)[:3]

def multi_hit(raw, paras, gold):
    hits = set()
    for p in paras:
        hits.update(ranks(p)[:3])
    return gold in hits

sr = sum(single_hit(*c) for c in CASES) / len(CASES)
mr = sum(multi_hit(*c)  for c in CASES) / len(CASES)
print(f"single-query recall@3: {sr:.2f}")
print(f"multi-query  recall@3: {mr:.2f}")