Handling Large Volatile Corpora with AI

Narrative

Frontier labs have deployed prompt caching as a standard feature across major APIs by 2026, with Anthropic, OpenAI, and Google all offering 50-90% input cost reductions and 13-85% latency improvements for cached prefixes. A 2026 study across 500+ agent sessions found that stable-prefix caching works best when system prompts, tool definitions, and policy rules are cached as immutable context, while dynamic user queries and session-specific data remain uncached—a pattern critical for volatile corpora where index churn is frequent. Both Anthropic and OpenAI now price cached reads at steep discounts (~90% for Anthropic, 50% for OpenAI) while batch processing APIs offer 50% off-peak discounts, creating a clear choice: interactive queries hit prefix caches, overnight re-indexing and summarisation use batch tiers. Release cadence across the five leading labs compressed from 170+ days between models in 2023 to 49 days by mid-2026, with Anthropic now shipping models every 71.5 days—reflecting a pivot toward agentic systems that must manage large, evolving contexts. METR's independent evaluations establish that frontier models' task-completion time horizons have doubled every seven months since 2019, with Claude Opus 4.6 reaching ~2 hours of autonomous work; measurements above 16 hours are deemed unreliable by METR's current task suite. DeepSeek models demonstrated parity with late-2024 frontier capabilities at dramatically lower cost ($0.27/M tokens vs $5/M for Anthropic Opus), reshaping economics for organisations processing large corpora. OpenAI's Frontier platform emphasises agent identity, permissions, and multi-system integration without requiring data replatforming—allowing agents to handle volatile contexts across distributed systems while caching stable definitions once and reusing across requests.