Token Cost Radar

1How do you keep falling token prices from producing a bigger AI bill?

Cyprus Mail reports September 13 that PwC sees corporate AI spending rising despite falling token prices because cheaper inference encourages companies to deploy AI across more processes. PwC says costs accumulate across planning, tool use, retrieval, reasoning, orchestration, safeguards, logging, and review, while agent delegation and retries can compound consumption further. Its recommended controls include spending limits, routing rules, workflow thresholds, and matching model tiers to task requirements.

2Chinese models turn compute scarcity into an efficiency argument

Fortune reports September 13 that Chinese AI labs are narrowing the capability gap partly by extracting more value from constrained compute. Futurum Group analyst Brendan Burke told Fortune that efficiency-focused attention techniques are reducing computational complexity, while Larridin cofounder Ameya Kanitkar said Chinese models in workflows his company tracks can handle roughly 75% of engineering tasks reasonably well at about one-fifth the cost of U.S. alternatives.

3DeepSeek's new Flash architecture attacks the prefill bill

Unite.AI reports that DeepSeek V4.1-Flash activates 8 billion parameters per token during prefill and 16 billion during decode, versus 13 billion for both stages in its predecessor. The architecture is aimed at workloads such as coding agents where repeated large prefixes can dominate inference. The report also notes that DeepSeek's new API pricing took effect September 10 and that V4-Pro requests are scheduled to route to V4.1-Flash beginning September 14.

4Cached context approaches fresh-input scale on an inference marketplace

Surplus Intelligence's latest 28-day snapshot, generated September 12, records 1.808 trillion fresh input tokens, 1.528 trillion cache tokens, and 28.37 billion output tokens across more than 30 million requests. Its latest seven-day eligible traffic shows an 86.5% mean realized discount versus direct-provider pricing. The data covers one marketplace, but it makes reuse economics difficult to treat as a rounding error.

Cache-Aware Prompt Compression: A Two-Tier Cost Model for LLM API Caching

This arXiv study examines the collision between prompt compression and prefix caching. Its cache-aware method was the cheapest strategy in all 16 LongBench-v2 configurations studied, with mean savings of 49% versus cache-only and 90% versus an unoptimized baseline while maintaining similar quality. In one enterprise tool-using workload, the starting schema prefix was 94,000 tokens.

Why it matters: Tokenminimizing can become negative-ROI when every query produces a different compressed prefix and destroys cache reuse. The economical prompt is not necessarily the shortest prompt.

What Does Context Compression Cost an Agent?

This controlled study finds that context compression can preserve task-completion rates while substantially increasing the work needed to reacquire discarded information. In one GPT-5.5 condition, retrieval calls increased from 21.0 to 63.9 while the change in completion rate was not statistically significant.

Why it matters: A smaller context can simply move the bill into additional tool calls. Token yield needs to include the cost of rediscovering information that compression removed.

WISERouter: LLM Routing with Workload Budget Constraint

WISERouter treats model routing as a workload-level budget problem rather than assigning the same spending limit to every query. Its offline and online policies allocate expensive models selectively across a workload while enforcing an aggregate constraint, with experiments on RouterBench and SWE-Bench reporting stronger budget adherence or comparable performance with less exploration data.

Why it matters: Modelmaxxing works better when cheap requests can leave budget available for difficult ones. A fixed per-query cap wastes that flexibility.

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving

This serving study separates short-context and long-context requests into differently configured infrastructure pools instead of provisioning everything for the worst case. On Azure and LMSYS traces using Llama-3-70B on A100 GPUs, the authors report 31% to 42% fewer GPU-hours, 5.4 times fewer preemptions, and projected annual fleet savings of $2.86 million.

Why it matters: Token budgeting reaches below the API layer. Request length can determine infrastructure placement, KV-cache efficiency, and therefore the real cost of inference.