Grounded Generation & Citations

You retrieved the right chunks and ranked them on top. The model can still ignore them, blend them with its own training memory, or state a fact no chunk supports. Generation is the last stage, and its failure mode — confident, well-cited-sounding, unsupported claims — is the one users actually see.

The whole promise of RAG is faithfulness: every claim in the answer is supported by the retrieved context, not by the model's parametric memory. A faithful answer can still be "wrong" if the context is wrong — but that's a retrieval problem you can fix. An unfaithful answer is worse: the model asserts something the context doesn't say, and no amount of good retrieval prevents it. This chapter is about closing that last gap.

Three levers that make generation faithful

1. The grounding instruction. Tell the model explicitly: answer only from the context; if the context doesn't contain the answer, say you don't know. This single instruction converts many hallucinations into honest refusals — the model's default is to be helpful and fill gaps; you must license it to abstain.
2. Inline citations. Require the model to tag each claim with the chunk id it came from ("[c12]"). Citations do double duty: they let users verify, and they make unfaithfulness measurable — a claim with no valid supporting chunk is a caught hallucination.
3. Context ordering. Because of lost-in-the-middle (Ch. 1), put the highest-ranked chunks at the start and end of the context, not buried in the middle. Order matters even when the right chunks are all present.

Measuring faithfulness

You measure it the way Chapter 8 measured retrieval: with a checkable signal. Decompose the answer into individual claims, and for each claim ask — is it supported by any retrieved chunk? The fraction supported is your faithfulness score. A cheap, scalable version uses an LLM-as-judge to check each claim against the context (this is exactly the boundary with the Eval Engineer's job). The point: hallucination stops being a vibe and becomes a number you can drive down.

Starter code

# Each item: the context the model was given, and the claims its answer made
# (already decomposed). 'key' is the checkable fact substring for each claim.
ITEMS = [
  {"context": "The /export endpoint is rate limited to 20 requests per minute. "
              "Keys rotate every 90 days.",
   "claims": [ {"text":"/export allows 20 req/min", "key":"20 requests per minute"},
               {"text":"keys rotate every 90 days", "key":"90 days"} ]},
  {"context": "The free tier allows 3 projects. Webhooks retry up to 5 times.",
   "claims": [ {"text":"free tier allows 3 projects", "key":"3 projects"},
               {"text":"there is a 30-day money-back guarantee", "key":"30-day money-back"} ]},  # unsupported!
]

def supported(claim, context):
    return claim["key"].lower() in context.lower()

# TODO: per item, compute faithfulness = supported/total; print unsupported claims;
#       print overall faithfulness across all claims.

total = supported_total = 0
for item in ITEMS:
    n = len(item["claims"])
    s = sum(supported(c, item["context"]) for c in item["claims"])
    total += n; supported_total += s
    print(f"faithfulness {s}/{n}")
    for c in item["claims"]:
        if not supported(c, item["context"]):
            print(f"   UNSUPPORTED: {c['text']}")

print(f"overall faithfulness: {supported_total/total:.2f}")