Token Cost Radar

1What happens when the AI coding budget stops behaving like a seat license?

DevOps.com argues that AI coding spend increasingly behaves like cloud infrastructure: usage-metered, spiky, model-sensitive, and spread across vendors. It recommends tracking cost per developer, utilization, premium-model mix, and forecast versus budget rather than relying on seat count or raw token totals. The article also cautions that flat per-developer caps can throttle productive heavy users while leaving model-mix waste untouched.

2Wiz maps AI inference spend back to the feature that caused it

Wiz announced expanded cost attribution that maps cloud and AI spend to application services and traces AWS Bedrock consumption to IAM principals such as developer roles and service accounts. Wiz argues that token and inference costs become actionable only when engineering and FinOps teams can connect them to the service, team, or product feature generating the spend and ultimately calculate unit economics.

3The open-weight token discount widens to 82%

BenchLM's August 27 live pricing dataset tracks 148 models and reports median API prices of $1.00 per million input tokens and $4.00 per million output tokens. At its 3:1 input-output blend, open-weight models have a median price of $0.53 per million tokens versus $3.00 for proprietary models, an 82% discount. The tracked spread between the cheapest and most expensive models is roughly 4,773 times.

4CFO tokenomics moves from price per token to cost per business interaction

ETCFO's August 26 analysis says enterprises moving from pilots to production are discovering a broader AI cost base spanning model calls, agents, infrastructure, data, security, governance, and human oversight. It recommends dynamic model routing, cascade architectures, showback, and unit economics such as cost per business interaction. Citing KPMG's Q2 2026 AI Pulse survey, the authors say 49% of organizations have delayed or scaled back deployments when benefits did not clearly outweigh costs.

CacheRouter: Tool Discovery Without Sacrificing the Prompt Cache

CacheRouter identifies a tension between progressive tool disclosure and prompt caching: changing the visible tool list reduces schema tokens but can invalidate the reusable request prefix. Its dual-path design keeps a small fixed tool set in the main model's prompt and routes long-tail tools through a separate channel. In prototype tests, token-level cache-hit rates reached 90.99% and 95.2%, cutting input cost to about 12% and 8% of a no-cache baseline under the DeepSeek pricing used by the authors.

Why it matters: This complicates the simple rule that fewer resident tool schemas always means lower cost. Tokenminimizing has to preserve cache geometry as well as reduce prompt size, or one optimization can quietly erase another.

SchemaRouter: Field-Aware Tool Routing for Efficient Heterogeneous Agentic RAG

SchemaRouter selects both the tools and response fields required for a query. On its 110-query benchmark, it used 227 retrieved-context tokens versus 2,066 for a fetch-everything baseline while matching that baseline's answer accuracy within overlapping confidence intervals. A more aggressive minimum-field configuration lost accuracy.

Why it matters: The economical target is relevant context, not minimum context. Tool payloads can be narrowed substantially, but token yield falls when compression removes evidence the task actually needs.

ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents

ReCache independently caches recurring tool and skill representations so they remain reusable when resources appear in different combinations or orders. Across seven public tool and skill datasets, its resource-wise attention preserved invocation performance while producing a 3.655 times time-to-first-token speedup, and the complete system reduced allocated KV-tensor memory by 92.43%.

Why it matters: Tool-surface economics extend below visible token counts. If recurring capability descriptions can retain reusable inference state, the system can keep a broad capability surface without continually rebuilding its full computational footprint.

Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference

This serving study routes requests to infrastructure pools according to estimated total token budget rather than provisioning every instance for worst-case context. On Azure and LMSYS traces serving Llama-3-70B on A100 GPUs, the authors report 17% to 39% fewer required GPU instances, with modeled annual savings of $1.2 million to $2 million at 1,000 requests per second.

Why it matters: Token budgeting is becoming an infrastructure primitive as well as an application control. The expected size of a request can determine which model, which workflow, and now which serving pool should handle it.