Token Cost Radar

1What if the token price stays flat but the task gets 40% more expensive?

The Verge reports that Google's new Gemini 3.8 Flash keeps the same $0.75 per million input-token and $3.75 per million output-token pricing as 3.7 Flash, but Google warns that the model can spend more tokens and make more agentic evaluation steps when additional reasoning improves the result. Early measurements cited by The Verge put the increase around 40% per task in some workloads, illustrating why cost per token and cost per completed task can move in opposite directions.

2Anthropic cuts the quiet line item in the agent bill by 75%

Yahoo Finance reports that Anthropic cut Fable 5.1 cache-read pricing by 75% to $0.25 per million tokens while leaving the headline $10 input and $50 output rates unchanged. Anthropic says the change can reduce typical workload costs by about 25% and highly agentic workloads by as much as 45%, making prefix reuse rather than nominal input pricing the larger economic lever for repeated context.

3Is your AI bill a pricing problem or a design problem?

Forbes Business Council argues that falling token prices have not prevented enterprise AI costs from exceeding projections because application design determines how often models are called, how much context is resent, which models handle each step, and how long agent loops continue. The article frames routing, caching, context management, and workflow architecture as financial controls rather than merely engineering optimizations.

4Live inference markets put cache traffic above one trillion tokens

Surplus Intelligence's September 3 marketplace snapshot reports 1.30 trillion fresh input tokens, 1.15 trillion cache tokens, and 17.78 billion output tokens over the trailing 28 days. Its latest seven-day eligible traffic showed a 90.2% mean realized discount from direct-provider pricing. The dataset covers one inference marketplace rather than the whole industry, but it shows how large reuse and routing economics have become relative to fresh generation.

One Model, Many Minds: Replacing Multi-Agent Chatter with Mixture of Roles

This late-August arXiv paper replaces repeated text communication among specialized agents with latent roles combined inside one model invocation. Across its reported reasoning and personality evaluations, the approach matched multi-agent-system performance while reducing token cost by 20 times.

Why it matters: Agent-to-agent conversation can become a large communication tax. Useful specialization does not necessarily require serializing every intermediate thought into another model's context.

CacheRouter: Tool Discovery Without Sacrificing Prompt Caching

CacheRouter studies a conflict between progressive tool disclosure and prompt caching. Changing the visible tool catalog reduces schema tokens but can invalidate reusable prefixes. Its dual-path design keeps a stable core surface while discovering long-tail tools separately, reaching token-level cache-hit rates above 90% in prototype experiments.

Why it matters: Tokenminimizing can increase cost when removing schemas destroys a more valuable cache discount. Tool loading and cache geometry need to be optimized together.

SchemaRouter: Field-Aware Tool Routing for Agentic RAG

SchemaRouter chooses both which tools to invoke and which response fields to retrieve. On its 110-query benchmark, it used 227 retrieved-context tokens versus 2,066 for a fetch-everything baseline while maintaining comparable answer accuracy. A more aggressive minimum-field strategy lost accuracy.

Why it matters: The target is relevant context, not minimum context. Selective tool payloads can improve token yield dramatically until the system starts removing evidence required to solve the task.

ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents

ReCache independently caches recurring tool and skill representations so they remain reusable when resources appear in changing combinations or orders. Across seven public tool and skill datasets, its resource-wise attention produced a 3.655 times time-to-first-token speedup while the complete system reduced allocated KV-tensor memory by 92.43%.

Why it matters: Tool-surface economics extend below visible prompt tokens. Preserving computed state for recurring capabilities can make broad tool access cheaper without repeatedly reconstructing the same context.