Token Cost Radar

1How much of the AI bill did the original business case miss?

ETCFO reports that KPMG advisers see enterprise AI moving from a technology purchase toward consumption economics in which prompts, model calls, agents, infrastructure, security, data pipelines, and oversight all contribute to cost. Citing KPMG's Q2 2026 AI Pulse survey, the article says 49% of organizations have delayed or scaled back AI deployment when benefits did not clearly outweigh costs, and estimates that initial investment cases can capture only 25% to 35% of three-year total cost of ownership.

2IDC says the tokenmaxxing correction did not fix the governance gap

IDC reports that engineering teams can often detect AI cost anomalies before finance has authority or visibility to respond. In one 2026 survey, 61% of organizations exceeded their 2025 cloud AI budgets, while 83% of those overruns remained below 15%. IDC also says difficulty budgeting token and inference-based pricing is now the largest pricing barrier reported by organizations evaluating AI vendors.

3Security teams start saving frontier models for frontier work

The New Stack reports that security workloads are adopting tiered model strategies as premium-model economics collide with routine, high-volume analysis. The recommended pattern is to use cheaper models and tighter prompts for common work, then escalate tasks requiring greater reasoning or security capability rather than applying frontier-model pricing across the entire workload.

4AI gateways turn model sprawl into an enforceable cost surface

GEC Newswire reports that F5 has expanded its AI Gateway with model-access optimization, agent-to-tool governance, and centralized controls intended to reduce token costs. F5 says organizations now manage an average of seven AI models and that 77% identify inference as their dominant AI activity, increasing the value of enforcing model and tool policy before requests reach expensive execution paths.

SchemaRouter: Field-Aware Tool Routing for Efficient Heterogeneous Agentic RAG

SchemaRouter selects not only which tools to call but which response fields to retrieve. On its 110-query benchmark, it used 227 retrieved-context tokens versus 2,066 for a fetch-everything baseline while matching that baseline's answer accuracy within overlapping confidence intervals. An aggressive minimum-field variant performed worse, showing that the economical target is relevant retrieval rather than minimum retrieval.

Why it matters: Tool routing can control payload bloat after tool selection as well as schema bloat before it. The finding also puts a useful boundary around tokenminimizing: removing evidence too aggressively can save almost nothing while damaging accuracy.

ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents

ReCache independently caches recurring tool and skill representations so they remain reusable even when resources appear in different combinations or orders. Across seven public tool and skill datasets, its resource-wise attention preserved invocation performance while producing a 3.655 times time-to-first-token speedup, and the complete system reduced allocated KV-tensor memory by 92.43%.

Why it matters: Large tool surfaces create reusable inference state as well as prompt text. Tool economics can therefore be improved below the schema layer by avoiding repeated reconstruction of capabilities the agent has already encountered.

Which Eviction Policy Should an LLM Cache Use?

This recent semantic-cache study finds LFU to be the strongest simple eviction default across its evaluated settings, but its more important result concerns safe reuse. At one tested similarity threshold, raw cache-hit rates of 51% to 60% became quality-adjusted hit rates of only 1.1% to 2.2% after checking whether cached answers were genuinely substitutable.

Why it matters: A cache hit is not token yield unless the reused answer is valid. Semantic-caching economics need to discount unsafe reuse before turning hit rate into claimed savings.

Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems

This recent arXiv paper defines token inflation as the gap between apparent single-call cost and true workflow cost after failures and retries. The authors measure inflation as high as 4.25 times in one evaluated setting. Their router reached 94.7% accuracy on GSM8K under a fixed budget while using 31% fewer tokens than the comparison router.

Why it matters: The cheapest first call can create the most expensive completed trajectory. Cost-aware routing needs expected retry and failure costs rather than a rate-card comparison alone.