The Corpus Is the Asset
Teams obsess over which model to use. The thing that actually decides whether your assistant tells the truth is the pile of documents you point it at — and it's usually the part no one owns.
A RAG system can only be as good as the documents it retrieves from. If your corpus is full of outdated drafts, duplicates, and contradictions, a perfect retriever will faithfully hand the model a stale policy and the model will confidently recite it. "The AI is wrong" is often "the AI found exactly what you stored, and what you stored was wrong." Cleaning, de-duplicating, and assigning an owner to the corpus is unglamorous and decisive.
Two traps in particular cost leaders. Staleness: documents go out of date and nobody refreshes them, so the assistant's accuracy quietly decays even though no code changed. Contradiction: two versions of the same policy live in the corpus, and the system can't know which one you meant. The fix is ownership and hygiene, not a better model.
Garbage in
Your team wants two weeks to "clean up the docs" before wiring up the assistant. A director says just index everything now and fix it later. Who's right, and how do you decide?
The team is right, within reason. Indexing a messy corpus doesn't just produce messy answers — it produces confident, cited messy answers, which are worse because users trust them. You don't need perfection, but you do need the obvious wins: remove duplicates and superseded versions, tag documents with dates, and assign an owner. Frame it to the director as risk: every stale document is a potential "the AI told a customer the old refund policy" incident with your logo on it. Two weeks of hygiene is cheap insurance.
Vantage HR: the assistant that quoted the old policy
Vantage HR rolled out an internal assistant over its policy library. An employee asked about parental leave; the assistant confidently quoted twelve weeks — the previous year's policy, which had been updated to sixteen. Both versions were in the corpus. Retrieval worked perfectly; it just returned a document nobody had removed. The mistake reached dozens of employees before HR caught it.
The audit found roughly 40% of the indexed documents were stale drafts, duplicates, or superseded versions. The fix wasn't technical: they assigned a corpus owner, deleted superseded documents, stamped every policy with an effective date, and set a quarterly review. Accuracy on policy questions rose sharply — with no change to the model or the retriever, only to what they'd stored.
Brightline's audit of its help center turns up the same rot: nearly half is old drafts and near-duplicates. Rather than index the mess, they appoint an owner to prune and date the documents first. It's the least glamorous decision of the whole project and quietly the highest-leverage one.
Quiz · Chapter 3
- The biggest lever on RAG quality is usually:
- "The AI gave a wrong policy" most often means:
- The fix for a stale corpus is:
- Indexing a messy corpus is dangerous because: