Token Cost Radar

1What happens when token prices fall 100-fold but consumption rises 8,000-fold?

The Hindu reports September 19 on the widening gap between falling AI unit costs and rising enterprise spending. Coforge management said the unit cost of tokens needed to deliver comparable intelligence fell roughly 100-fold over seven months while token consumption grew about 8,000-fold. The company also said open-weight models represented about 35% of observed token consumption, up from roughly 11% to 12% in early 2025. The numbers illustrate the inference paradox directly: lower prices can accelerate consumption faster than they reduce the bill.

2StepFun enters the agentic price fight with a 1-million-token model

StepFun released Step 5 Preview on September 19 with a 1 million-token context window, vision support, and a Mixture-of-Experts architecture using 27 billion active parameters out of 600 billion total. Current reporting lists API pricing at $1 per million input tokens and $2.70 per million output tokens and says its measured capability matched Kimi K3 Max while costing about 65% less per task. StepFun says open weights are planned for October 15.

3Cached tokens are approaching fresh-input volume in an inference marketplace

Surplus Intelligence's latest 28-day snapshot, generated September 19, records 2.370 trillion fresh input tokens, 1.948 trillion cache tokens, and 37.03 billion output tokens across 42.4 million requests. Its latest seven full days of eligible traffic show a 90.9% mean realized discount versus direct-provider pricing. The marketplace is not representative of the entire industry, but cache volume equal to roughly 82% of fresh input makes reusable context impossible to treat as a minor billing detail.

4The inference index ticks higher while the price spread stays enormous

IFX closed September 19 at 83.46, up 0.37% from its previous reading. Its 29-model basket spans blended prices from $0.06 to $11.25 per million tokens, with an average of $2.54. Its capability-adjusted board currently identifies different price leaders for frontier, capable, and budget workloads, reinforcing the modelmaxxing case for routing by required capability rather than using one default model everywhere.

Task-to-Model Optimization for Enterprise LLM Coding Assistants

T2MO argues that naive token-cost minimization can increase end-to-end cost once retries, escalations, and developer waiting time are included. Its framework classifies tasks, grades difficulty, benchmarks candidate models, and routes each task to the cheapest model that satisfies quality and latency requirements, explicitly optimizing expected cost per completed task.

Why it matters: This is modelmaxxing with the denominator fixed. A cheap failed attempt followed by an expensive escalation can cost more than routing correctly the first time.

TokenPilot: Cache-Efficient Context Management for LLM Agents

TokenPilot tackles the conflict between pruning agent context and preserving prompt-cache continuity. Its framework stabilizes prefixes while removing environmental noise and evicts context only when its task relevance expires. Across PinchBench and Claw-Eval, the authors report cost reductions ranging from 56% to 87% depending on evaluation mode while maintaining competitive performance.

Why it matters: Tokenminimizing can backfire when prompt mutation destroys cheap cache reuse. The economical context is not simply the smallest context, but the smallest one that preserves useful information and reusable prefixes.

AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents

AgenticCache reuses frequent plan transitions instead of calling an LLM at every agent step, while an asynchronous updater validates and refreshes cached plans. Across four multi-agent embodied benchmarks and three models, the authors report average task-success gains of 22%, simulation-latency reductions of 65%, and token-usage reductions of 50%.

Why it matters: Caching can remove entire inference events rather than merely discounting repeated prompt tokens. That is a stronger form of token yield because the system gets useful work from computation it already paid for.

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving

This serving study routes short-context and long-context requests into separately configured inference pools using estimated total token budgets. On Azure and LMSYS traces serving Llama-3-70B on A100 GPUs, the authors report 31% to 42% fewer GPU-hours, 5.4 times fewer preemptions, and projected annual fleet savings of $2.86 million.

Why it matters: Token discipline extends below the API meter. Context size determines KV-cache allocation and concurrency, so the expected token footprint can determine which infrastructure pool should execute the request.