Token Cost of Ownership

Narrative

The central story from frontier lab coverage is a dramatic and continuing collapse in published API token prices — GPT-4 launched in March 2023 at roughly $30–$36 per million input tokens; by mid-2024 GPT-4o was available at ~$5/M, and by early 2026 flagship models cluster around $1.75–$5/M input — a 85–95% reduction in under three years. Anthropic cut Claude Opus pricing by 67% in a single announcement in 2025, while Google's Trillium (TPU v6e) infrastructure delivered a 4× improvement in inference performance-per-dollar. Yet financial disclosures and investigative reporting (The Register, Data Center Dynamics, Ed Zitron's newsletter) reveal that these user-facing prices are far below actual cost: OpenAI spent an estimated $8.4B on inference in 2025 alone against roughly $20B in revenue, and its cumulative losses through 2024 reached $5B. The 25× gap between flat subscription pricing and actual API cost for heavy users, documented by Centific, epitomises the subsidisation dynamic. METR's pre-deployment evaluations — covering GPT-4o (2024), GPT-4.5 (Feb 2025), Claude 3.7 (April 2025), GPT-5 (May 2025), and a January 2026 time-horizon update — add a critical safety and capability dimension: autonomous task completion horizons are doubling roughly every 7 months, meaning that as tokens become cheaper, the agentic compute per task is simultaneously expanding, driving total inference spending upward even as per-token unit costs fall.

On the infrastructure side, SemiAnalysis's InferenceMAX benchmarking (2025) and NVIDIA's Blackwell architecture results show hardware efficiency gains of up to 15× versus the prior H100 generation — the primary mechanism behind price declines on the supply side. Google's TPUv7 analysis underscores a structural cost-advantage dynamic: labs with proprietary silicon (Google with TPUs, potentially Anthropic with long-term AWS partnerships) face different cost floors than those dependent on GPU spot markets. However, two arXiv preprints from March 2026 formalise why these efficiency gains will not produce sustained price floors: the Structural Jevons Paradox — as token costs drop, enterprise architectures are redesigned to consume exponentially more compute via multi-agent loops, extended context, and deeper reasoning chains — and a token-futures paper modelling how lab subsidisation strategies will eventually give way to commodity market pricing. The consensus across these sources is that hyperscaler capex for 2026 (~$602B, roughly 75% AI-tied) and rising energy demand are price-floor candidates that will increasingly resist downward pressure, even as hardware efficiency continues to improve.