Multi-step reasoning
Multi-step reasoning is a reasoning approach in which an LLM arrives at a final answer not through a single response generation, but by going through multiple intermediate steps, such as generating sub-questions, verifying partial answers, and retrieving additional information.
Limitations of Simple Question Answering
A factual lookup such as "What is the revenue?" can be completed in a single step. However, a question like "What initiatives did the person in charge of the division with the highest year-over-year revenue growth introduce?" cannot be answered without going through multiple intermediate steps: comparing sales data → identifying the division → identifying the person in charge → searching for information on the initiatives.
Multi-step reasoning refers to an approach in which an LLM internally decomposes such complex questions and solves them incrementally. It is an extension of Chain-of-Thought (CoT) prompting, but differs in that, when combined with RAG, a search of external data sources is inserted at each step.
Relationship with Agentic RAG
Agentic RAG can be understood as an implementation of multi-step reasoning as an agent loop. The agent determines "what should be investigated next" and cycles through search → evaluation → re-search. Multi-step reasoning is the design pattern for that thought process, while Agentic RAG is the architecture that executes it.
Accuracy vs. Speed Trade-off
The more steps involved, the greater the comprehensiveness of the answer; however, the LLM inference cost and search latency accumulate at each step. In practice, many systems are designed with an upper limit on the number of steps (typically 3–5) and an early termination mechanism that triggers once sufficient information has been gathered.
Articles covering this term
- What is Adaptive RAG? How to Balance Cost and Accuracy with Query-Driven Dynamic RetrievalLearn how Adaptive RAG eliminates inefficiencies of traditional RAG by selectively retrieving only when needed—covering mechanisms, implementation patterns, and B2B use cases.
- Latency Budget Design for AI Agents — How to Control the Trade-off Between Thinking Time and Response TimeExplore how "thinking time" causes response delays in multi-step reasoning agents, and compare latency budget allocation strategies and implementation patterns based on task complexity.
- AI Agent Evaluation Design Guide — Tool Calling, Execution Traces, and Regression DetectionEvaluating AI agents requires validating tool calls and execution trajectories, not just final outputs. Learn golden set design, trajectory scoring, regression detection, and CI integration.
- 10 RAG Implementation Failure Patterns and How to Avoid Them — Preventing Production Issues Before They Happen10 common RAG failures before & after production: chunk design errors, retrieval accuracy drops, hallucination fixes & more. Learn key avoidance strategies before you build.
Related Terms
Context Engineering
Context Engineering is a technical discipline focused on systematically designing and optimizing the
System Prompt
A system prompt is an instruction given to an LLM before the start of a conversation with a user, de
AI Red Teaming (AI Red Teaming)
An evaluation method that systematically tests AI system vulnerabilities from an attacker's perspect
Grounding
A technique that cross-references LLM outputs with external data sources and search results to gener