Token Cost Radar

1What if the cheapest token does not produce the cheapest task?

Business Insider reports September 18 on Bank of America analysis that China's AI price war is entering a new phase. Broad token-price cuts are giving way to competition around how efficiently models complete tasks, while advanced reasoning, long context, multimodality, and agentic capability can still command premiums. The report identifies cost per completed task, rather than price per token alone, as the more useful economic metric.

2A 69-task enterprise benchmark puts cost beside capability

AIMultiple's enterprise benchmark, updated September 17 and surfaced in current results, evaluates 16 models across 69 tasks in strategy, marketing, HR, sales, and operations. Gemini 3.8 Flash averaged a slightly higher score than Claude Sonnet 5 while costing $0.61 versus $1.23 per task, and GPT-5.6 Luna averaged $0.016 per task versus $1.73 for Claude Opus 5. The benchmark does not imply that cheaper models are universally better, but it demonstrates why model routing needs workload-level cost and quality data.

3The Token Price Index falls to $2.32 per million

The Token Price Index's September 18 weekly reading puts geometric-mean blended inference cost at $2.32 per million tokens across 24 models from 11 providers, down from $2.41 a week earlier. Its current components average $2.17 per million input tokens and $10.09 per million output tokens. The index also restored DeepSeek V4 Pro after DeepSeek reversed its planned retirement, illustrating how model availability itself can move a cost benchmark.

4Today's model-economics snapshot puts median input at $1 per million

Superpower Daily's September 19 Model Economics Index tracks 25 priced models in its verified launch dataset and reports median pricing of $1.00 per million input tokens and $3.75 per million output tokens. The snapshot also records context windows reaching 11 million disclosed tokens, underscoring why nominally cheap input can still become expensive when applications treat enormous context capacity as an invitation to fill it.

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving

This serving study routes short-context and long-context requests into separately configured inference pools using an estimated total token budget. On Azure and LMSYS traces serving Llama-3-70B on A100 GPUs, the authors report 31% to 42% fewer GPU-hours, 5.4 times fewer preemptions, and projected annual fleet savings of $2.86 million. A Qwen3-235B case study on AMD MI300X projects $15.4 million in annual savings at 10,000 requests per second.

Why it matters: Token budgeting can control physical infrastructure, not just API invoices. Routing by expected context footprint prevents the majority of short requests from paying the memory and concurrency penalty of worst-case provisioning.

AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents

AgenticCache reuses frequent plan transitions instead of making a fresh LLM planning call at every agent step, while a background process asynchronously validates and refreshes the cache. Across four multi-agent embodied benchmarks and three models, the authors report average task-success improvements of 22%, simulation-latency reductions of 65%, and token-usage reductions of 50%.

Why it matters: Semantic and plan caching can improve token yield without merely shortening prompts. Reusing successful intermediate reasoning removes entire inference calls rather than trimming a few tokens from each one.

Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference

This related serving study estimates total token demand before dispatch and separates short and long workloads into differently configured vLLM pools. On Azure and LMSYS traces, the authors report 17% to 39% fewer required GPU instances, corresponding to roughly $1.2 million to $2.0 million in annual savings at 1,000 requests per second.

Why it matters: The work connects token discipline to disaggregated serving. Context size is not just something the model reads, it determines memory allocation, concurrency, and ultimately how much hardware a fleet needs.

CachedAttention: Cost-Efficient Multi-Turn LLM Serving

CachedAttention preserves and reuses KV state from conversation history across multi-turn inference instead of repeatedly rebuilding it. Its evaluations report up to 87% lower time to first token, up to 7.8 times higher prompt-prefill throughput, and up to 70% lower end-to-end inference cost.

Why it matters: The economics of long context increasingly depend on whether the system recomputes history or treats previous computation as an asset. Token counts alone miss that distinction.