RAG (Retrieval-Augmented Generation)
RAG (Retrieval-Augmented Generation) is a technique that improves the accuracy and currency of responses by retrieving relevant information from external knowledge sources and appending the results to the input of an LLM.
LLMs only possess knowledge up to their training cutoff date. Moreover, even with the knowledge they do have, they can be confidently wrong (hallucination). RAG has established itself as a practical solution to these two weaknesses.
The mechanism is intuitive. Upon receiving a user's question, relevant documents are first retrieved from internal documents or a knowledge base. The retrieved results are then passed to the LLM along with the question. The LLM generates a response based not only on its own knowledge, but grounded in the provided documents. Since sources can be explicitly cited, verifying responses becomes straightforward.
Breaking down the components of RAG, they consist of document preprocessing (chunking), vector embedding, similarity search (semantic search), and prompt construction for the LLM. Each step involves choices, and something as simple as how chunks are split can significantly impact response quality.
The distinction between RAG and fine-tuning is frequently debated, but they serve different roles. RAG is a method for "having the model reference external knowledge," while fine-tuning is a method for "adjusting the model's behavior and tone." If the goal is to have the model accurately answer questions based on internal manuals, RAG is the reasonable starting point; if the goal is to standardize the format and style of responses, fine-tuning is. Many projects employ both in combination.
Articles covering this term
- Choosing Between Fine-Tuning and RAG: A Practical Guide Comparing Cost, Accuracy, and Use CasesFine-tuning vs RAG: which should you choose? We compare both across 4 axes—cost, accuracy, update frequency, and security—to guide your decision by use case.
- 10 RAG Implementation Failure Patterns and How to Avoid Them — Preventing Production Issues Before They Happen10 common RAG failures before & after production: chunk design errors, retrieval accuracy drops, hallucination fixes & more. Learn key avoidance strategies before you build.
- What is AI Grounding? An Implementation Guide to Fact Verification and Improving LLM Answer Accuracy with Web SearchAI grounding anchors LLM responses to authoritative sources. Learn how RAG and web search reduce hallucinations, plus implementation steps for enterprise deployment.
- What is Adaptive RAG? How to Balance Cost and Accuracy with Query-Driven Dynamic RetrievalLearn how Adaptive RAG eliminates inefficiencies of traditional RAG by selectively retrieving only when needed—covering mechanisms, implementation patterns, and B2B use cases.
Related Terms
RRF
RRF (Reciprocal Rank Fusion) is a scoring method that integrates ranking results returned by multipl
Agentic RAG
Agentic RAG is an architecture in which an LLM autonomously and iteratively generates search queries
Embedding
Embedding is a technique that transforms unstructured data such as text, images, and audio into fixe
GraphRAG
A next-generation RAG architecture that combines knowledge graphs and vector search, leveraging rela