Agentic RAG is a form of retrieval-augmented generation in which a language model acts as an autonomous agent that decides when, where and how to retrieve information, evaluates what it finds and repeats the process until it has enough evidence to answer. Within the Retrieval & knowledge domain of artificial intelligence, it extends classic RAG from a fixed retrieve-then-generate pipeline into a reasoning loop with tools, planning and self-verification.
The approach targets questions that a single search cannot answer: multi-hop questions, comparisons across sources, tasks that combine documents with databases or APIs, and queries whose wording does not match the vocabulary of the indexed content. The price is higher latency, higher token cost and more complex failure modes than a standard pipeline.
How agentic RAG works as a control loop
Agentic RAG replaces the single retrieval call with a plan, act, observe and reflect cycle driven by the model itself. The agent reads the user request, decomposes it into sub-questions, chooses a retrieval tool for each, inspects the returned passages and decides whether to search again, switch sources or stop.
The loop is usually implemented through tool calling: retrieval functions such as vector search, keyword search, SQL queries or web search are exposed to the model with a schema, and the model emits structured calls. Patterns such as ReAct-style reasoning, where thoughts and actions alternate, are common foundations. An orchestrator executes the calls, enforces limits and returns observations to the model’s context.
The following table lists the typical stages of one agentic retrieval cycle and what each stage contributes.
| Stage | What the agent does | Typical technique |
|---|---|---|
| Routing | Decides whether retrieval is needed and which source fits | Classifier prompt or tool selection |
| Decomposition | Splits a complex question into sub-queries | Planning prompt, query rewriting |
| Retrieval | Runs searches against one or more indexes | Hybrid search, BM25, vector similarity |
| Evaluation | Grades relevance and sufficiency of results | LLM-as-judge, reranking scores |
| Iteration | Reformulates queries or tries another source | Self-correction, fallback to web search |
| Synthesis | Writes the answer with citations to passages | Grounded generation with attribution |
Architectural variants of agentic RAG
Agentic RAG is a family of designs rather than one algorithm, and the variants differ mainly in how much autonomy the model receives. A single-agent router may only choose among indexes, while a multi-agent system can assign separate agents to planning, retrieval, verification and writing.
The table compares the most frequently deployed variants by autonomy and typical use.
| Variant | Degree of autonomy | Best suited for |
|---|---|---|
| Routing RAG | Low: one decision per query | Several separate knowledge bases |
| Corrective RAG | Medium: grades and retries retrieval | Noisy or incomplete indexes |
| Self-reflective RAG | Medium: critiques its own draft | High-accuracy question answering |
| Iterative multi-hop RAG | High: chains dependent searches | Questions needing linked facts |
| Multi-agent RAG | Very high: specialized cooperating agents | Research reports and complex analysis |
Variants are often combined. A research assistant might use routing for source selection, corrective grading for each result set and a reflection step before the final answer. Some systems also query a knowledge graph for entity relations, which overlaps with GraphRAG, while keeping unstructured passages in a vector index.
Agentic RAG compared with classic RAG
The essential difference is that classic RAG retrieves once with the original query, whereas agentic RAG treats retrieval as a decision the model can revisit. Classic pipelines are predictable and cheap; agentic ones are adaptive but harder to bound.
The comparison below summarizes the practical consequences of that difference for engineering teams.
| Property | Classic RAG | Agentic RAG |
|---|---|---|
| Retrieval calls per query | Usually one | Often 2 to 10 or more |
| Query formulation | User wording, sometimes rewritten once | Repeatedly rewritten by the agent |
| Sources | One index | Multiple indexes, tools and APIs |
| Latency | Roughly 1 to 3 seconds | Several seconds to minutes |
| Cost per answer | Low and predictable | Higher and variable |
| Handling of missing evidence | Answers anyway or refuses | Searches further or reports the gap |
| Debugging | Linear trace | Branching trace with many steps |
Neither design is universally superior. For frequently asked, single-fact questions, a well-tuned classic pipeline with good chunking and reranking often matches agentic quality at a fraction of the cost. Agentic RAG earns its overhead when questions are compositional or when the right source is not known in advance.
Designing and configuring an agentic RAG system
A robust implementation starts with strong retrieval primitives, because an agent cannot reason its way past a poor index. Clean chunking, sound embeddings, hybrid search and a reranker remain the foundation; the agentic layer only decides how to use them.
- Define narrow, well-described tools, each with a clear purpose, input schema and example queries.
- Set hard budgets: a maximum number of steps, tokens and wall-clock time per request.
- Add an explicit sufficiency check so the agent stops when evidence is adequate instead of searching indefinitely.
- Require passage-level citations in the final answer to support grounding and review.
- Log every tool call and observation to enable tracing, evaluation and cost analysis.
The decision table below helps match the design to the workload.
| Situation | Recommended design | Reason |
|---|---|---|
| Single index, factual lookups | Classic RAG with reranking | Lowest cost and latency |
| Several domain indexes | Routing RAG | One cheap decision improves precision |
| Questions needing chained facts | Iterative multi-hop RAG | Later queries depend on earlier answers |
| Low-quality or sparse corpus | Corrective RAG with fallback | Detects weak results and retries |
| Long analytical reports | Multi-agent RAG | Divides planning, research and writing |
Limitations, risks and evaluation of agentic RAG
The main risks of agentic RAG come from compounding errors across steps. A wrong sub-question early in the plan steers every later search, and an agent may loop on reformulations without progress or stop too early with confident but partial evidence.
Security is a specific concern. Retrieved documents can contain indirect prompt injection, text that tries to redirect the agent toward unintended tool calls. Mitigations include treating retrieved content strictly as data, restricting tool permissions, and requiring human-in-the-loop approval for actions with side effects.
Evaluation must therefore cover the trajectory, not only the final answer. Teams typically measure answer correctness and faithfulness, retrieval recall across all steps, average number of tool calls and cost per resolved query, and compare the agentic system against a classic baseline on the same question set.
In everyday language
A normal AI search assistant looks something up once and then answers with whatever it found. Agentic RAG works more like a careful researcher. It decides what it needs to know, looks in the right places, reads the results, and searches again if something is missing.
Think of the difference between glancing at one encyclopedia page and visiting a library where you check several books, ask a librarian and compare notes. The answer is usually better, but it takes longer and costs more. That trade-off is the heart of the idea.
Seen in the real world
A legal operations team at a mid-sized manufacturer receives questions such as which supplier contracts signed after 2022 allow termination without penalty and also lack a data protection clause. A classic single-pass pipeline retrieves a handful of passages that mention termination and misses most of the relevant agreements.
The team deploys an agentic RAG assistant with three tools: a SQL query over the contract registry, a hybrid search over clause text and a clause classifier. The agent first filters contracts by date in the registry, then searches each candidate for termination and data protection clauses, and finally checks borderline wording a second time before answering.
A typical query now takes about 40 seconds and eight tool calls instead of three seconds, but in the team’s internal review the list of matching contracts was complete, with every entry linked to the cited clause, replacing roughly a day of manual reading.