Agentic RAG — Evolution, Challenges, and Decision Criteria

Narrative

Practitioner coverage from late 2025 through May 2026 reveals a clear inflection. The Thoughtworks Technology Radar's Volume 33 (November 2025) is the most authoritative single signal: after RAG dominated Volume 32 in April 2025, Volume 33 shifted its central theme to agents and MCP, with Thoughtworks CTO Rachel Laycock framing it as a 'step change' toward context engineering. That editorial move compressed what had been a gradual architectural transition into a widely recognised industry event. InfoQ's April 2026 field report from three financial-services deployments (Q4 2025, ~1,500 multi-hop queries) supplied the sharpest empirical grounding: ~30% silent failure rate under static RAG, and roughly 60% of hallucinations tracing to unhandled execution errors rather than model reasoning — a finding with direct implications for where engineering effort should be directed.

On tooling, the picture is one of consolidation and functional specialisation. The benchmark evidence (AIMultiple, January 2026) shows a 53% token-count difference between the most and least efficient frameworks — Haystack at 1.57k tokens versus LangChain at 2.40k — which compounds materially at scale. LangGraph reached 1.0 stability in October 2025 and now functions as the de facto stateful agent orchestration layer, while LlamaIndex handles retrieval depth; MarsDevs and others report the dominant 2026 production pattern is the two-framework combination of LlamaIndex plus LangGraph. Evaluation tooling has matured in parallel: Ragas (400k+ monthly downloads), LangSmith, Arize Phoenix, and Langfuse form a three-layer observability stack, and MarsDevs quantifies production targets as faithfulness ≥0.9, answer relevancy ≥0.85, and context precision ≥0.8.

The cost-and-latency evidence is the most practically consequential finding for practitioners deciding when to switch. MarsDevs reports that agentic loops with three to four iterations take 8–12 seconds against the 1–2 second baseline for standard RAG, a 3–10x token cost multiplier, and a worse p95 latency profile. Redis quantifies semantic caching as recovering up to 73% of cost in high-repetition workloads. CSO Online reports 72–80% of enterprise RAG implementations underperformed or failed within their first year, with 51% of all enterprise AI failures in 2025 being RAG-related — a failure rate that drove both the agentic migration and the growth of specialist evaluation infrastructure. On accuracy, a 2025 MDPI study across 250 clinical vignettes showed a 55-percentage-point multi-hop accuracy gap: 34% for static RAG versus 89% for agentic RAG, described as a 'categorical capability gap' rather than a marginal improvement.

The information organisation dimension is where the least practitioner consensus exists but the most structurally interesting parallels emerge. The Towards Data Science piece from October 2025 documents the M&A evidence — ServiceNow's acquisition of data.world, Salesforce's $8 billion Informatica purchase — as market confirmation that knowledge graphs and metadata management are becoming the semantic backbone for AI. The RAGFlow year-end review introduces 'Context Engineering' as the successor to RAG optimisation, shifting focus from retrieval algorithms to the systematic design of the full retrieval–context assembly–reasoning pipeline. Thoughtworks independently surfaces the semantic layer as critical infrastructure for agentic text-to-SQL, noting that Snowflake Semantic Views, Databricks Metric Views, and dbt MetricFlow are converging around Open Semantic Interchange (OSI) v1.0 — a standardisation signal that partially mirrors dimensional modelling's role in the data warehouse era.