Retrieval-Augmented Generation (RAG)
Category: Grounding & context
RAG retrieves relevant documents from an external source and adds them to the prompt so the model answers from that evidence. It grounds responses in current, private, or authoritative data and cuts hallucination.
What it is
Retrieval-Augmented Generation (RAG)Retrieval-augmented generation combines a language model with a retrieval system. Given a query, it first fetches relevant passages from a knowledge source — a vector index, search engine, or database — then conditions the model on those passages to generate an answer grounded in the retrieved evidence. Lewis et al. introduced the approach as a way to combine parametric knowledge (in the weights) with non-parametric knowledge (in an external store).
RAG lets a model answer from information it was never trained on — recent events, proprietary documents, or authoritative sources — and, when the retrieved context is cited, makes answers verifiable. It is the standard pattern for question answering over private or fast-changing corpora.
When to use
- Answers must reflect private, proprietary, or frequently updated data.
- You need citations or traceable evidence for each answer.
- The knowledge base is too large or too current to bake into the model.
- Reducing hallucination on factual queries is a priority.
How it works
- Index the knowledge source (e.g. chunk documents and embed them for similarity search).
- At query time, retrieve the most relevant passages for the user's question.
- Insert the retrieved passages into the prompt as grounding context.
- Instruct the model to answer using only that context and to cite or defer when it is insufficient.
Illustrative structure
The structure grounds the answer in retrieved text: "Using only the context below, answer the question. If the context is insufficient, say so. Context: <retrieved passages> Question: <query>." The retrieval step selects the passages; the prompt confines the answer to them.
Pitfalls
- Retrieval quality caps answer quality — irrelevant chunks produce wrong answers.
- Chunking and embedding choices strongly affect what gets retrieved; they need tuning.
- Without a "say I don't know" instruction, the model may ignore weak context and hallucinate anyway.
- Stale or unmaintained indexes silently degrade over time.
Frequently asked questions
What is Retrieval-Augmented Generation (RAG)?
RAG retrieves relevant documents from an external source and adds them to the prompt so the model answers from that evidence. It grounds responses in current, private, or authoritative data and cuts hallucination.
When should you use Retrieval-Augmented Generation (RAG)?
Answers must reflect private, proprietary, or frequently updated data. You need citations or traceable evidence for each answer. The knowledge base is too large or too current to bake into the model. Reducing hallucination on factual queries is a priority.
What are common pitfalls of Retrieval-Augmented Generation (RAG)?
Retrieval quality caps answer quality — irrelevant chunks produce wrong answers. Chunking and embedding choices strongly affect what gets retrieved; they need tuning. Without a "say I don't know" instruction, the model may ignore weak context and hallucinate anyway. Stale or unmaintained indexes silently degrade over time.
Sources
Related techniques
← All prompt engineering techniques
Apply Retrieval-Augmented Generation (RAG) with Prompeteer, the Agentic Contextual AI Platform →