Skip to content

Retrieval and RAG·Lesson 57 of 67·expert

A complete RAG pipeline

Ingest, chunk, embed, index, retrieve, re-rank, assemble, answer, cite — every stage from this module wired together, with the contract that makes an answer checkable.


In this lesson · 24 min

  • →Wire the ingest and query paths into one working system
  • →Write an answer contract the model can be held to
  • →Validate that every citation refers to a chunk that was actually supplied
  • →Refuse to answer when retrieval found nothing good enough
  • →Trace a query end to end so a bad answer can be diagnosed

Every piece exists now. This lesson assembles them and adds the part that makes the output trustworthy: a contract, and a check that the contract was kept.

The two paths

PythonPython not started

Ingest

PythonPython not started
PythonPython not started

Query

PythonPython not started

The third query returns nothing, which is exactly right — and it is the case most pipelines get wrong, because retrieval happily returns k results however irrelevant.

The answer contract

The prompt is where the system’s honesty is decided:

PythonPython not started

Rule 3 is the one that matters. Without an explicit refusal phrasing, a model asked a question its context cannot answer will produce a plausible answer from training — which is the failure everyone calls hallucination.

Validating the answer

Never trust the citations. Check them:

PythonPython not started

A citation pointing at c9 when only c1 and c2 were supplied is a fabrication you can detect mechanically, with no judge and no second model call.

The whole thing

PythonPython not started

Three questions, three different outcomes — answered, answered, refused — and the trace explains each one without guessing.

Failure modes

PythonPython not started

Try it yourself

Exercise 1

Write check_citations(answer, supplied_ids): return a dict with cited, unsupported, refused and ok. An answer is not ok if it cites nothing, or cites an id that was not supplied.

Your turnPython not started
Exercise 2

Write select_context(candidates, minimum, budget): drop anything below the score floor, fit the rest into the token budget best first, and return (passages, used, outcome) where outcome is answer or refuse.

Your turnPython not started

Check yourself

A user asks something your corpus does not cover. What should a well-built RAG system do?

Check yourself

Why check citations mechanically rather than relying on the prompt's instruction to cite?

Recap

  • Two paths: ingest offline (load, chunk, embed, index) and query online (embed, retrieve, re-rank, assemble, generate, validate).
  • The answer contract belongs in the system prompt: answer only from the context, cite every claim, and refuse in a fixed phrase when the context does not contain the answer.
  • Validate citations against the ids you actually supplied — a citation to an unsupplied chunk is mechanically detectable fabrication.
  • A similarity floor is what turns “always returns k things” into “found nothing, and said so”.
  • Trace every query: retrieved, selected, cited, unsupported, outcome. Most bad answers are retrieval problems in disguise.