Day 8 — Build Cited RAG

Today AskDocs joins retrieval to generation. This is where many demos stop too early: they stuff top-k text into a prompt and assume the answer is grounded.

Build the small prompt

The reference build_rag_prompt labels every chunk:

[1] (from rust.txt, score 0.81)
Rust gives memory safety without a garbage collector.

Then it asks the model to answer only from supplied context and admit when the answer is absent. Keep the labels stable so generated citations can be parsed.

Upgrade the prompt into a manifest

A production-shaped request records selection as data:

pub struct ContextItem {
    pub id: String,
    pub source: String,
    pub text: String,
    pub score: f32,
    pub content_sha256: String,
}

pub struct ContextManifest {
    pub query: String,
    pub items: Vec<ContextItem>,
    pub byte_budget: usize,
}

The rendered prompt is one view of the manifest. Store the manifest so a bad answer can be traced to retrieval, context selection, or generation.

Verify citations after generation

At minimum:

1. parse every citation ID;
2. reject IDs absent from the manifest;
3. require important claims to carry a citation;
4. check quoted text against the cited chunk;
5. abstain when no selected chunk supports the answer.

Retrieval score answers “which chunk looked close to the query?” It does not answer “does this chunk prove the generated sentence?”

Break it deliberately

Ask a question whose answer is not present:

cd rust
cargo run -p askr -- --mock --rag askr/sample-docs \
  "Who won the local election?"

The mock reveals the prompt rather than proving behavior. Add a scripted model that returns a confident, uncited answer. Your verifier should fail it even though it is fluent.

Then create a retrieval failure: put the correct answer in a chunk that lexical search ranks fourth while k=3. The generator never had the evidence. The fix belongs in retrieval or budget selection, not in a more forceful instruction.

Completion proof

Save the context manifest and a cited answer for three frozen questions: answerable, ambiguous, and unanswerable. Classify failures as acquisition, chunking, embedding, ranking, selection, generation, or verification.

Deep reference: Retrieval contracts, snapshots, and reranking.

Next: Day 9 — Stream and Evaluate →.