Advanced Retrieval

The vanilla "embed, retrieve top-k, generate" pipeline handles most queries. The ones it can't — filtered searches, multi-hop questions, queries that need a decision about whether to retrieve — are where senior RAG engineering lives. This chapter is the toolbox for the hard 20%.

Each advanced pattern exists because a specific class of query defeats vanilla RAG. Learn them as matched pairs — the failure and the fix — so you deploy the right one instead of cargo-culting all of them.

Metadata filtering: retrieve within a constraint

A user asks "what's the refund policy?" but they're an enterprise customer, and the enterprise policy differs from consumer. Pure semantic search happily returns the consumer chunk — it's textually similar and has no idea about the constraint. Metadata filtering attaches structured fields (customer_tier, product, date, language) to each chunk and filters before or during retrieval, so you only search the eligible subset. It's the single highest-leverage precision tool for corpora with natural partitions, and it prevents a whole class of confidently-wrong-but-relevant answers.

Multi-hop & agentic RAG: retrieve, reason, retrieve again

"Which of our SOC-2-certified vendors also support SSO?" needs two retrievals: first the SOC-2 vendors, then, for those, SSO support. A single retrieval can't do it — the answer isn't in any one chunk. Agentic RAG lets the model drive retrieval: it decides what to search, reads results, and searches again based on what it learned, looping until it can answer. It's more powerful and more expensive/slower — each hop is an LLM call plus a retrieval — so you reserve it for genuinely multi-step questions rather than making every query pay for the capability.

Adaptive retrieval: know when NOT to retrieve

Not every query needs RAG. "Rewrite this paragraph to be shorter" needs no corpus; retrieving for it just injects noise and cost. Mature systems put a cheap router in front: does this query need retrieval, and if so, which index? Retrieving for a no-retrieval query is a real failure mode — it can drag irrelevant chunks into context and lower answer quality.

Starter code

CHUNKS = [
  {"id":"c1","tier":"consumer",  "topical":0.95, "text":"Consumer refunds within 14 days."},
  {"id":"c2","tier":"enterprise","topical":0.80, "text":"Enterprise refunds are governed by contract terms."},
  {"id":"c3","tier":"consumer",  "topical":0.90, "text":"Consumer plan cancellation policy."},
  {"id":"c4","tier":"enterprise","topical":0.78, "text":"Enterprise cancellation requires 30-day notice."},
]
# (query, asker's tier, correct chunk id)
QUERIES = [
  ("refund policy",       "enterprise", "c2"),
  ("cancellation policy", "enterprise", "c4"),
]

def best(cands):  # highest topical score
    return max(cands, key=lambda c: c["topical"])

# TODO unfiltered: best(all chunks) — does it match the correct id?
# TODO filtered:   best(chunks whose tier == asker tier)
# Print accuracy (fraction correct) for each.

def accuracy(use_filter):
    correct = 0
    for q, tier, gold in QUERIES:
        cands = [c for c in CHUNKS if (c["tier"] == tier)] if use_filter else CHUNKS
        correct += best(cands)["id"] == gold
    return correct / len(QUERIES)

print(f"unfiltered accuracy: {accuracy(False):.2f}")
print(f"filtered   accuracy: {accuracy(True):.2f}")