Token Cost Radar

1Is tokenmaxxing dead, or just rational for founders?

Business Insider reports that Y Combinator CEO Garry Tan is explicitly defending tokenmaxxing for startup founders. Tan said aggressive agent usage can cost $50,000 to $100,000 annually, but argued that founders can justify the expense if it gives them capabilities earlier and successful agent procedures are then converted into reusable instructions. His position is a direct counterpoint to the recent enterprise push toward token restraint.

2AT&T wants open models handling most of its AI workload

The Wall Street Journal reports that AT&T ultimately wants open models to power 70% to 80% of its AI usage. Chief Data and AI Officer Andy Markus said moving applications from proprietary models to open alternatives has already produced savings of 80% to 90% in some cases. The company's approach treats model selection, customization, data control, and routing as a single cost strategy.

3Nvidia joins the cheaper-model and routing push

The Wall Street Journal reports that Nvidia released Nemotron 3.5 Lightning as a smaller open model aimed partly at reducing enterprise token costs. Nvidia also introduced Nemo Switchyard, an open-source routing library designed to select models according to task complexity, cost, latency, and performance.

LLMRouter: Evaluating Routing on Quality and Inference Cost

This recent arXiv paper provides a unified framework and benchmark for comparing more than 16 model-routing approaches across single-turn, multi-turn, personalized, memory, vision, and other workloads. Learned routers outperformed the strongest fixed-model baseline by 14.6% relatively, while lightweight routers became more competitive as cost constraints tightened.

Why it matters: Modelmaxxing is becoming an optimization problem with measurable tradeoffs. The paper also finds that extra multi-turn routing steps can add cost and redundant information without reliably improving results.

Marginal Value Estimation for Efficient Deep Research

This recent arXiv work studies when a research agent should expand additional search branches and when the expected improvement no longer justifies the tokens and runtime. Its strongest pure-efficiency configuration reduced average token cost to about 114,600 tokens while maintaining competitive report quality.

Why it matters: Long-running agents need a stopping rule as much as a token budget. Marginal-value estimation turns additional reasoning and retrieval into purchases that must justify themselves.

LaCache: Robust Semantic Caching for LLM Serving

This recent arXiv paper examines cache-collision attacks against semantic caching and proposes validating both query similarity and initial speculative output before reusing an answer. The method is designed to preserve semantic caching's latency and inference-cost benefits while making malicious or incorrect reuse substantially harder.

Why it matters: Caching can eliminate entire model calls, but savings are useful only if the reused answer remains trustworthy. Semantic-cache economics therefore need a reliability budget alongside the token budget.

How Do AI Agents Spend Your Money?

This arXiv study of eight frontier models on agentic coding tasks finds that agents can consume roughly 1,000 times more tokens than code reasoning or code chat, with input tokens dominating cost. Identical tasks varied by as much as 30 times in token consumption, greater usage did not reliably improve accuracy, and models systematically underestimated their own eventual token requirements.

Why it matters: The findings put hard numbers behind token discipline. Agent cost is stochastic, context-heavy, and difficult for the model itself to forecast, which strengthens the case for external budgets, retrieval controls, and runtime measurement.