Token Cost Radar

Token Cost Radar

November 15, 2026

The latest token-economics story is a collision between falling model prices and rising infrastructure complexity. A November 14 pricing tracker records reductions for OpenAI's GPT-5.6 Sol, following cuts to several Chinese-developed models. New research shows that poorly managed agent context can create persistent inference slowdowns even after traffic spikes end, while another study demonstrates substantial latency improvements from adapting compute and cache allocation to agent workloads. Meanwhile, fresh cloud-cost analysis warns that infrastructure waste remains a serious problem. The emerging lesson is that cheaper tokens are only one ingredient in economical AI. Cache behavior, workload design, utilization and completed-task quality increasingly determine the actual cost.

Top Developments (Last 24 Hours)

1Are token prices falling faster than enterprise AI bills?

Tokenando's pricing-change tracker lists November 14 reductions of approximately 9.1% for OpenAI GPT-5.6 Sol, with input pricing moving from $4.40 to $4.00 per million tokens, output pricing from $22 to $20, and cached input from $0.44 to $0.40. Its November 13 records also show reductions for Moonshot AI's Kimi K2.5 and Z.ai's GLM batch offerings, including an approximately 55% reduction in GLM 5.2 batch output pricing. The same tracker records a doubling of one DeepSeek batch cached-input rate, demonstrating that not every price movement is downward. These are independently aggregated listed-price observations, not audited customer invoices or confirmation that every provider offers identical rates. The financial implication is that static model-price assumptions can become obsolete quickly, particularly when applications combine standard, batch and cached inference.

Tokenando ↗

2New cloud-cost analysis puts infrastructure waste at 29%

A Shattered analysis updated November 14 examines cloud waste as AI infrastructure spending expands. Its headline estimate of 29% cloud waste illustrates the continuing challenge of converting provisioned infrastructure into productive computation. The analysis discusses how AI workloads complicate conventional FinOps practices through expensive accelerators, variable demand and specialized resource requirements. The headline figure should not be interpreted as an independently verified measurement that 29% of all AI inference expenditure is wasted. General cloud inefficiency and inference-specific inefficiency are different quantities. For organizations self-hosting open-weight models, the practical lesson is that low software licensing costs do not guarantee inexpensive tokens when GPU utilization, capacity planning and operational overhead are poorly controlled.

Shattered ↗

3Can smarter cache management prevent an AI agent slowdown from becoming permanent?

CASRAI's November 14 coverage highlights Evolving Infra, a November 9 research preprint proposing agent-adaptive inference serving. The research addresses a limitation of conventional disaggregated inference systems: prefill and decode capacity may be allocated inefficiently while valuable key-value caches are evicted before agents reuse them. The proposed architecture separates compute allocation from cache management and adjusts both according to changing agent workloads. Its authors report reductions in mean end-to-end latency and P95 time to first token of up to 4.1 times and 14 times, respectively, against an evaluated NVIDIA Dynamo configuration. These are research benchmark comparisons, not independently verified reductions in commercial token prices. The broader development is growing attention to the infrastructure consequences of long-running agent sessions.

CASRAI ↗

Cheaper Tokens, More Expensive Workflows

Tokenmaxxing describes maximizing AI token consumption, sometimes treating volume as evidence of productivity. Tokenminimizing is the practice of reducing unnecessary token consumption while preserving useful results. Token yield measures useful completed work relative to tokens consumed. Today's pricing movements and inference research reinforce the progression from tokenmaxxing through tokenminimizing toward token yield. Lower prices improve unit economics, but longer agent sessions, cache misses and infrastructure inefficiency can increase total costs. No convincing fresh evidence established tokenminning, tokenmining or modelmaxxing as distinct newly trending expressions.

Fortune

IBM CEO Arvind Krishna argued in a November 5 Fortune essay that indiscriminate token rationing can undermine the productivity benefits organizations seek from AI. He distinguishes choosing a small number of valuable AI projects from restricting their ability to operate once funded. Krishna emphasizes inference infrastructure, model economics and the difference between provider pricing and underlying computation costs. The article is an executive opinion from a company selling AI infrastructure, not independent evidence that unlimited token spending produces better returns. Its relevance to current price reductions is that financial discipline should optimize useful outcomes rather than impose arbitrary consumption ceilings.

Fortune ↗

Flexera

Flexera's October analysis explicitly frames the transition from tokenmaxxing to valuemaxxing, bringing emerging AI consumption vocabulary into established technology-finance discussions. Valuemaxxing means prioritizing measurable business outcomes over maximum AI usage. Flexera recommends visibility into spending ownership, model routing, caching, budgets and the retirement of unproductive pilots. The article is vendor analysis rather than a new industry standard, but its terminology provides evidence that the vocabulary of AI cost governance is broadening beyond simple token minimization. Its emphasis on value aligns with current concerns about whether falling token prices translate into improved enterprise returns.

Flexera ↗

Menlo Ventures

Menlo Ventures' recent State of Enterprise AI report estimates that Anthropic, OpenAI and Google capture 69% of enterprise AI deployment spending, compared with 50% the previous year. It also estimates that Anthropic and OpenAI together account for 86% of enterprise language-model API expenditure. The figures come from a survey of nearly 400 US enterprise decision-makers and Menlo's market modeling, not audited global provider revenue. The findings provide an important counterweight to the latest price cuts: customers may continue paying premiums for stronger capabilities when those capabilities improve task completion. Price competition and capability-driven spending concentration can occur simultaneously.

Menlo Ventures ↗

Microsoft

Microsoft's earlier research on tool search remains important as agent platforms expose increasingly large MCP and API catalogs. Loading every tool definition upfront consumes context even when most tools are irrelevant to the task. Retrieval-based tool discovery instead exposes relevant definitions when needed. Microsoft's published experiments report substantial reductions in tool-schema token overhead as catalog sizes increase. These measurements do not automatically establish equivalent savings across complete workflows because discovery, incorrect selections and retries also consume resources. The architectural lesson is that efficient tool-surface management should be evaluated alongside model routing and cache performance.

Microsoft Command Line ↗

Research Watch

Evolving Infra: Agent-Adaptive LLM Serving at Scale

Submitted November 9, this paper proposes dynamically adapting prefill and decode resources while coordinating distributed key-value cache management with agent context behavior. The system allows serving nodes to change roles as workload composition shifts. Across coding, mathematics and computer-use tasks using GLM-5.2 and Kimi K2.6 on configurations involving up to 256 NVIDIA GB300 GPUs, the authors report improvements of up to 4.1 times in mean end-to-end latency and 14 times in P95 time to first token relative to their evaluated Dynamo baseline. These are workload-specific experimental results.

Why it matters: Agent inference efficiency depends on matching infrastructure behavior to changing sessions, not merely selecting cheaper models. Better cache retention and resource allocation can reduce wasted computation and improve serving capacity.

arXiv ↗

Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving

Submitted November 10, this research shows how a brief traffic surge can leave a prefix-cached inference server operating inefficiently long after arrivals return to normal. In nine of twelve matched experimental pairs, a temporary burst produced approximately 10% cache-hit rates, roughly half the throughput and a growing backlog for the remaining observation period. The researchers attribute the behavior to longer-lived sessions increasing the working set beyond available cache capacity, causing repeated recomputation. A cache-aware admission budget prevented collapse in all tested configurations. The results are specific to the evaluated systems and workloads.

Why it matters: A cache miss is not merely a small latency penalty when an agent repeatedly processes long histories. Poor cache management can create persistent resource waste, making workload admission and memory budgeting part of inference cost governance.

arXiv ↗

When Cache Pressure Becomes a Trap: Metastability in Agent Serving

Submitted November 2, this independent investigation identifies a related failure mode in disaggregated inference serving. In an SGLang experiment, a ten-minute traffic surge reduced goodput by approximately 88% during the following hour. The researchers explain how slower requests prolong sessions, expand their combined context requirements and trigger additional cache misses. Their proposed admission and cache-management approach achieved 1.20 to 1.70 times the goodput of one comparison system and 2.35 to 3.59 times that of another in evaluated hardware experiments. These are experimental performance comparisons, not direct customer-billing measurements.

Why it matters: Two recent research efforts independently highlight how agent context growth can destabilize serving economics. Controlling active-session memory requirements may be as important as reducing the number of tokens in individual prompts.

arXiv ↗

Phrase of the Day

“Valuemaxxing”

Valuemaxxing means optimizing AI investment for measurable useful outcomes rather than maximizing token consumption or minimizing expenditure indiscriminately. Flexera uses the expression explicitly in its discussion of the transition from tokenmaxxing to outcome-focused AI governance.

  1. AI adoption
  2. Tokenmaxxing
  3. Token shock
  4. Tokenminimizing
  5. Token discipline
  6. Token yield
  7. Cost per successful task
  8. Valuemaxxing
  9. Verified business value

Today's pricing and infrastructure findings explain why the transition from tokenmaxxing to valuemaxxing is more than a vocabulary exercise. Tokenando's latest records show that model prices can change substantially within days, making static cost assumptions unreliable. Recent research demonstrates that serving systems can waste enormous amounts of computation when cache management fails, even when the number of incoming requests returns to normal. At the same time, enterprise research suggests that buyers continue paying premium prices for models they believe deliver stronger results. These developments expose three different economic questions. What does each token cost to purchase? How efficiently does the infrastructure produce and reuse the necessary computation? What useful work does the completed workflow deliver? Tokenminimizing addresses avoidable consumption, while token yield connects consumption with successful results. Valuemaxxing extends the measurement to the business value of those results. Practical governance therefore requires more than fixed spending caps. Teams need representative workload benchmarks, cache and infrastructure utilization measurements, complete execution accounting, quality thresholds, and evidence that the resulting work is worth its total cost. The best optimization may involve a cheaper model, a better cache policy, more selective context loading or even additional reasoning when it prevents expensive failures.

The cheapest token is not always the most valuable one. The expensive token that prevents a failed job may earn its keep.

Flexera ↗

The jCodeMunch read

Today's research reinforces a recurring principle in agent economics: unnecessary context creates costs beyond its visible token count. It can increase prefill work, pressure caches and complicate long-running sessions. jCodeMunch addresses one part of that problem by reducing code-reading tokens through tree-sitter symbol retrieval and byte-precise context, allowing coding agents to request relevant code elements instead of broad source files. Its contribution complements inference-side improvements in caching and resource allocation. The goal is not simply fewer tokens, but reliable completed work with less unnecessary processing.

See how the 96.3% per-lookup figure is measured →

← All editions