Token Cost Radar

1What comes after Uber's tokenmaxxing era?

Business Insider reports that Uber CTO Praveen Neppalli Naga believes the company's maximal AI-consumption phase is ending. Uber has quadrupled its use of frontier AI tools since the start of 2026, while lowering per-token costs through prompt caching, cheaper default models, greater developer cost visibility, and experiments with open-weight models.

2Uber says AI use is rising while spending stays stable

Business Insider reports that Uber CFO Balaji Krishnamurthy says the company is holding AI spending relatively stable despite broader adoption. Uber is using more economical models, practical defaults, and employee cost awareness to reduce cost per token while pursuing incremental productivity gains.

3Microsoft turns token discipline into internal policy

TechRadar reports that Microsoft is assigning token budget targets to divisions, tracking internal consumption, and making a less expensive GPT-5.6 model the default for many employee workflows. Executive vice president Jay Parikh told staff that Microsoft is optimizing for business impact rather than tokenmaxxing.

4Meta tests output tokens priced at twenty cents per million

The Wall Street Journal reports that Meta launched its Muse Code agent with a feedback-linked option priced at $0.20 per million output tokens. The unusually low rate adds another data point to the widening gap between commodity execution pricing and premium agent subscriptions.

LaCache: Robust Semantic Caching for LLM Serving

This new arXiv paper examines cache-collision attacks against semantic caching, where crafted queries can poison reusable responses for later users. It proposes a more robust design intended to preserve the latency and cost benefits of semantic reuse while reducing this attack surface.

Why it matters: Semantic caching can eliminate repeated inference, but an unsafe hit is negative token yield. Cost savings must be evaluated alongside response integrity and tenant isolation.

PrefixPlace: Provable Prefix Key-Value Placement

This new arXiv paper optimizes placement of reusable prefix KV-cache material across workers. In evaluated retrieval-augmented generation replays, it improved materialization-cost savings by 40.3% over vLLM Automatic Prefix Caching and by 6.3% over the strongest tested offline baseline.

Why it matters: Caching economics depend on where reusable state lives, not merely whether it exists. Better placement can reduce repeated prefix computation without changing the model or prompt.

TraceCompiler: Turning Repeated Agent Work into Deterministic Skills

This new arXiv paper mines clusters of noisy agent traces and compiles recurring procedures into mostly deterministic executable workflows. The approach targets repeated exploration, retries, accidental ordering, and lookups that agents otherwise purchase again on later runs.

Why it matters: The cheapest successful model call is sometimes the one converted into ordinary software after the procedure is understood. Trace compilation treats prior token spending as reusable process discovery.

Practical Online KV Cache Compaction for LLM Agents

This new arXiv paper compares token eviction and attention-matching methods for compacting agent context during execution. It finds that immediate compaction often harms performance, while delaying compaction until future agent queries reveal what matters recovers much of the loss.

Why it matters: Tokenminimizing has a timing problem. Removing context before its value becomes observable can save memory while destroying the evidence needed later in the workflow.

IR2Solve: Structured Intermediate Representations for Cost-Efficient Optimization

This arXiv paper uses one semantic model call followed by deterministic verification for optimization autoformulation. On a matched ten-instance panel, two comparison workflows used 3.3 and 22.9 times its token volume while making 8 and 39 semantic calls per instance, respectively.

Why it matters: Separating semantic interpretation from deterministic execution can prevent agents from repeatedly paying a model to perform work that conventional software can verify and complete.