Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation (RAG) is a technique that enhances AI-generated responses by first retrieving relevant information before generating an answer.
What is retrieval-augmented generation?
Retrieval-Augmented Generation (RAG) is a technique that gives a large language model relevant source material before it generates an answer. Generation is the model writing the response. Retrieval is the search process that supplies supporting context from documents or data.
RAG can make an answer more relevant and easier to support with citations. It does not guarantee accuracy. The system can retrieve the wrong passage, miss an important document, or generate a claim that is not supported by the retrieved text.
How a RAG system works
A basic RAG workflow has four parts:
- Split source documents into smaller chunks.
- Convert each chunk into a vector embedding and store it with its source metadata.
- Search for chunks related to the user's question.
- Give the selected chunks to the language model as context for its answer.
The chunking strategy matters because it controls what information can be found together. Chunks that are too small can lose context. Chunks that are too large can mix unrelated ideas and weaken retrieval.
When to use RAG
RAG is useful when an AI application needs information that is private, frequently updated, too detailed for the model to remember, or expected to include sources. Common examples include internal knowledge assistants, document search, customer support, policy lookup, and research tools.
RAG is less useful when the task does not depend on external knowledge, when the source collection is small enough to provide directly, or when an exact database query would be more reliable than semantic search.
Vector RAG and graph RAG
Most RAG systems use vector embeddings to find passages with similar meaning. More advanced graph-based RAG systems retrieve entities and relationships rather than isolated passages. This can help when the answer depends on connections across several records or documents.
My mental model for RAG is a customer support call. Before answering your question, an agent asks for your name, address, or customer reference number to find the correct records. A RAG system performs a similar retrieval step before the model responds.