Token Cost Radar

1What happens if token prices keep falling but usage does not rise fast enough?

Reuters reports October 7 that AI labs are betting lower token prices will stimulate enough additional usage to compensate for declining unit prices, a version of Jevons' paradox. Reuters cites recent data showing token prices down roughly 40%, while questioning whether demand is currently accelerating fast enough to offset that decline. The issue moves token economics beyond customer savings: falling inference prices also change the revenue model of the companies supplying those tokens.

2Mistral launches a 1T model at $1.36 input and $4.18 output

Mistral opened Mistral Large 4 to public API preview October 6. The mixture-of-experts model has 1.05 trillion total parameters, 49 billion active parameters per token, and a 1 million-token context window. Standard pricing is $1.36 per million input tokens, $0.14 cached input, and $4.18 output. Mistral's changelog says launch pricing is 50% off for two weeks, bringing those rates to $0.68, $0.07, and $2.09 respectively. The company says downloadable weights will follow later this month.

3Enterprise agent budgets move into the control plane

Cohere's North 2 adds granular cost controls, rate limits, user quotas, consumption tiers based on requests and token rates, and organization-wide caps. Administrators can monitor usage down to individual users and agents and control which models teams can use. The launch is a useful AI FinOps milestone because token governance is becoming an execution-time policy rather than a dashboard teams inspect after the invoice arrives.

4MCP schema overhead gets measured instead of merely complained about

ImportStatic published measurements October 6 across four real MCP servers and found that GitHub's 46 default tools consume 11,207 tokens of tool definitions under its test tokenizer. One issue-writing tool alone costs 807 tokens to describe. More importantly, the study argues that MCP overhead depends on the harness: clients that defer or selectively load tool definitions avoid paying the entire schema cost on every interaction. The protocol exposes capabilities, but the harness decides how much of that capability surface becomes resident context.

Harnessing LLMs as Agents: What Does It Cost?

This October 1 paper introduces the Language Model Agent Machine, a resource model that charges agent harnesses for bounded context, persistent memory, tool use, verification, communication, and repeated execution instead of treating the underlying model call as the whole computational cost. The authors derive bounds for context-memory traffic, recomputation, and verification and test several predictions on GPT-6 Astra.

Why it matters: Agent FinOps needs a cost model for the harness, not merely the model. Context movement, memory access, repeated calls, verification, and checkpointing can all consume resources even when the nominal model price is unchanged.

What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute

A panel of fifteen models found that a 0.5B model could reproduce 92% to 95% of reference tokens across three benchmarks, while the hardest 10% of tokens accounted for 64% to 80% of estimated FLOPs. Using the resulting difficulty map for routing reduced projected MATH-500 latency from 7.59 to 5.12 seconds while slightly improving accuracy, and reduced draft-token use by 32.6% at similar accuracy.

Why it matters: Modelmaxxing can become finer than routing whole requests. If token difficulty is highly uneven, sufficient compute suggests allocating expensive capacity only where individual pieces of generation actually require it.

Total Cost of Agency: Exact Attribution of Memory Injection Cost in Multi-Agent LLM Workflows

On a 200-task enterprise benchmark using real model APIs, the authors attribute roughly 12% of total billed cost to memory-injection tokens and find the share rising to 27.6% at workflow depth six. Reducing retrieval-window capacity from 32 entries to 2 lowered injected tokens by 28.7% with accuracy change within seed-level variation in their experiment.

Why it matters: Memory is not free merely because the model did not generate it. Retrieved state becomes billable input, so memory-window sizing belongs in token budgeting alongside model selection and output limits.

Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data

This work argues that repeatedly reopening unstructured documents forces agents to repurchase the same evidence in context. On FanOutQA, reasoning over an ideal pre-structured store was 28 times cheaper than repeatedly reasoning over source documents. Its adaptive data-cracking method structures useful information as a byproduct of earlier reasoning and cut cost by 53% in an extended evaluation while preserving accuracy.

Why it matters: Token yield improves when knowledge that inference already paid to discover becomes reusable structure. The cheapest future context may be information the agent never needs to reread as prose.