Token Cost Radar

1Who owns the AI bill when every department is spending it?

Business Insider reports that EY is creating an AI Value Realization Office to centralize AI spending, monitor usage, decide which initiatives deserve additional funding, and measure business impact across departmental boundaries. EY says an internal AI router introduced in April has helped reduce token consumption by 60% alongside other governance measures, while 98% of senior decision-makers in a recent EY survey said token spending had caused them to reconsider their AI approach.

2The model price war is turning into a rate-card maze

The GPU Daily's August 15 roundup highlights widening price segmentation across the model market, including OpenAI's sharply cheaper Luna tier and DeepSeek's move toward peak and off-peak V4 pricing. The practical effect is that model identity alone no longer determines cost. Capability tier, caching, context length, and execution timing increasingly matter too.

3Writer says the harness can move the cost curve without changing the model

TechCrunch reports that Writer launched Palmyra X6 alongside agent-harness changes intended to reduce multi-step execution cost. Writer says harness improvements lowered costs by about 40% on average across tested models, reinforcing the argument that orchestration, reasoning policy, batching, and tool behavior can matter as much as model price.

ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step

This August arXiv study finds that increasing reasoning effort can improve tool-task completion by enabling more exhaustive search without reliably teaching agents the cheaper recovery strategy. In one comparison, Claude Sonnet 5 used 7,326 completion tokens per solved base task versus 1,584 for Gemini 3.1 Pro, and 11,652 versus 3,332 under tool-mapping drift.

Why it matters: More reasoning can buy reliability without buying efficiency. Tool-using agents need explicit strategies for stopping redundant search and reusing known structure rather than assuming additional reasoning will discover the economical path.

Cross-Model KV Cache Transfer in LLM Families

This arXiv paper studies a hidden penalty in model routing: switching models normally forces the receiving model to recompute the prompt prefill. Its cross-model KV mapping retained 73% to 98% of standalone-prefill accuracy on four of six evaluated model pairs and ran 2.7 to 25 times faster than re-prefilling.

Why it matters: Modelmaxxing can destroy part of its own savings when every model switch repurchases context. Cache transfer suggests routing economics should include the state lost during the handoff, not just the destination model's token rate.

MCP Tool Descriptions Are Smelly

This arXiv study examines 856 tools across 103 MCP servers and finds that 97.1% of evaluated tool descriptions contain at least one identified description smell. Adding fuller descriptions improved median task success by 5.85 percentage points but increased execution steps by 67.46%, while compact combinations often preserved reliability with lower token overhead.

Why it matters: Tool descriptions create a genuine token-quality frontier. Too little schema guidance harms selection and arguments, while indiscriminately adding detail consumes context and can increase execution work.

Continuous Semantic Caching for Low-Cost LLM Serving

This arXiv paper develops semantic-cache policies for an effectively continuous space of user queries rather than assuming a fixed catalog of prompts. Its framework learns query costs and demand while accounting for the overhead of changing cached responses.

Why it matters: Semantic caching is most valuable when systems can reuse expensive answers across meaningfully similar requests without pretending that production traffic consists of exact repeats. The research pushes caching from a lookup trick toward an adaptive inference-cost control.