Token Cost Radar

1What if the new model costs the same per token but works harder?

Google introduced Gemini 3.8 Flash at $0.75 per million input tokens and $3.75 per million output tokens, the same introductory rates as 3.7 Flash. Google also says 3.8 Flash may execute extra reasoning steps and call tools iteratively on complex tasks, sometimes consuming more tokens to maximize performance. Developers can lower the effort setting or stay on 3.7 Flash when compute efficiency matters more than the additional capability.

2Meta formally eases off tokenmaxxing

WIRED reports that Meta told employees this week that performance evaluations will no longer depend on how much they use AI tools. Meta spokesperson Tracy Clayton told WIRED that the company evaluates contributions rather than AI usage, while employees said adoption remains encouraged and the new Hatch agent can consume substantially more resources than conventional chatbots and coding assistants.

3METR's missing spend ceiling turned one stolen key into $600,000 of inference

METR disclosed that attackers stole an API key in March and used it for three weeks to consume credits worth approximately $600,000. METR says the usage blended into its normally heavy evaluation traffic, its dashboard lacked complete visibility into rate-limited requests, and there was no spending limit available on the key. The organization has since added spend alerts where possible and expanded abnormal-usage monitoring.

4Boomi puts model spending inside the agent control plane

Boomi's newly announced Agent Control Plane is designed to sit between enterprise agents and business systems while governing access, actions, security, and model spending. The launch reflects a broader shift toward controlling agent economics at the execution layer rather than reconstructing costs after autonomous work has already occurred.

Cost-Utility Alignment in LLM Agent Trajectories

This late-August arXiv paper argues that the agent trajectory, not the individual model call, is the appropriate unit for efficiency analysis. Its framework tracks tokens, latency, monetary fees, tool interactions, failures, and other costs alongside the utility each step contributes to the final task.

Why it matters: Token yield needs attribution. A costly reasoning step can be worthwhile when it changes the result, while a cheap repeated tool call can be pure waste. Measuring the trajectory makes that distinction visible.

CacheRouter: Tool Discovery Without Sacrificing Prompt Caching

CacheRouter studies the tension between progressive tool disclosure and prompt caching. Dynamically changing the visible tool list saves schema tokens but can invalidate reusable prefixes. Its dual-path prototype keeps a stable core tool surface and routes long-tail tools separately, reaching token-level cache-hit rates of 90.99% and 95.2% in its reported experiments.

Why it matters: Tool-surface tokenminimizing can backfire when it destroys a larger cache discount. The efficient design has to optimize resident schema size and prefix stability together.

Which Eviction Policy Should an LLM Cache Use?

This semantic-caching study finds LFU to be the strongest simple eviction default in its protocol, but its larger result concerns safe reuse. At one evaluated similarity threshold, raw cache-hit rates of 51% to 60% became quality-adjusted hit rates of only 1.1% to 2.2% after checking whether the cached answer was actually suitable for the new query.

Why it matters: A cache hit is not token yield when the reused answer is wrong. Semantic-cache savings need to be discounted for answer validity before they become meaningful economics.

Paritok-4B: Intent-Conditioned Context Compression for Coding Agents

Paritok-4B compresses coding-agent file reads and tool outputs by selecting exact spans rather than paraphrasing them. Across all 300 SWE-bench Lite instances, it reduced context to 25.7% of the original size while retaining 86.5% of uncompressed single-shot solve quality. The authors also found that using GPT-5 as the compressor was economically negative at list prices.

Why it matters: Context reduction has its own cost. A token-saving layer only improves economics when the compression work costs less than the downstream inference it removes while preserving the evidence required for success.