Agentic RAG
Agentic RAG is an architecture in which an LLM autonomously and iteratively generates search queries, evaluates results, and decides whether to re-retrieve information as an agent, achieving answer accuracy that cannot be obtained with simple single-turn RAG.
Differences from Conventional RAG
A standard RAG pipeline operates in a linear flow: "user question → vector search → pass retrieved documents to LLM → generate answer." This is sufficient when the intent of the question is clear and the necessary information can be retrieved in a single search, but in practice, there are frequent cases where a single search does not yield all the required information.
In Agentic RAG, the LLM itself determines whether "the search results are insufficient" or "the query should be changed," rewriting the query or querying a different data source as needed. By incorporating multi-step reasoning, it can progressively collect and integrate multiple pieces of information to construct a final answer.
When Is It Effective?
Consider the example of querying an internal knowledge base. A question such as "Which proposal templates were used in the top 3 deals by sales last month?" requires multiple steps: searching sales data → identifying the deals → searching the proposal documents for each deal. By having the agent handle this decomposition and sequential search, the user can obtain an answer with a single question.
However, as the number of agent loop iterations increases, so do latency and token costs. Setting a loop limit and designing the system to return intermediate progress via streaming are essential for production use.
Articles covering this term
- Choosing Between Fine-Tuning and RAG: A Practical Guide Comparing Cost, Accuracy, and Use CasesFine-tuning vs RAG: which should you choose? We compare both across 4 axes—cost, accuracy, update frequency, and security—to guide your decision by use case.
- 10 RAG Implementation Failure Patterns and How to Avoid Them — Preventing Production Issues Before They Happen10 common RAG failures before & after production: chunk design errors, retrieval accuracy drops, hallucination fixes & more. Learn key avoidance strategies before you build.
- Latency Budget Design for AI Agents — How to Control the Trade-off Between Thinking Time and Response TimeExplore how "thinking time" causes response delays in multi-step reasoning agents, and compare latency budget allocation strategies and implementation patterns based on task complexity.
- What is Hybrid Search? How It Works and Implementation to Improve RAG Accuracy with Vector Search × Full-Text SearchExplains hybrid search using vector search, BM25, and RRF. Practical design patterns and implementation tips to improve RAG system retrieval accuracy.
Related Terms
RRF
RRF (Reciprocal Rank Fusion) is a scoring method that integrates ranking results returned by multipl
Embedding
Embedding is a technique that transforms unstructured data such as text, images, and audio into fixe
GraphRAG
A next-generation RAG architecture that combines knowledge graphs and vector search, leveraging rela
Gemini Embedding 2
Gemini Embedding 2 is a multimodal embedding model developed by Google, capable of converting text,