Token Cost Radar

Token Cost Radar

November 20, 2026

The latest AI cost developments put unnecessary context and measurable returns under the microscope. November 19 reporting reveals that Uber eliminated 50,000 to 70,000 tokens of potential MCP tool-schema overhead from coding-agent startup sessions. Google introduced on-demand tool retrieval for Gemini agents, while Microsoft published its internal approach to FinOps for AI and return on token spend. The emerging theme is that organizations can reduce inference expenditure before changing models or negotiating lower token prices. The challenge is distinguishing genuine efficiency improvements from savings that disappear when retrieval overhead, retries, cache behavior, and task quality are included.

Top Developments (Last 24 Hours)

1How did Uber stop its coding agents from loading 70,000 unnecessary tokens?

VentureBeat reported November 19 that Uber standardized four cost controls for interactive AI coding sessions: automatic context compaction at 400,000 tokens, medium reasoning effort, differentiated prompt-cache retention, and elimination of automatic MCP tool-schema loading. Uber found that directly exposing more than 100 tools could consume 50,000 to 70,000 tokens before an engineer submitted a prompt. Its alternative routes tool calls through command-line interfaces connected to a centralized MCP gateway, with tool search available for discovery. Uber reports that weekly agentic requests increased 9.4 times between February and August while overall AI spending remained approximately flat after April. Its broader optimization program reduced cost per 1,000 requests by nearly 34% from its peak and cost per session by 52% from its June peak. These are company-reported operational results, and the savings cannot be attributed exclusively to tool-schema reduction.

VentureBeat ↗

2Google introduces on-demand tool retrieval for Gemini agents

Google's November 19 Tool Retrieval announcement introduces deferred loading of tool definitions through the Gemini Interactions API. Instead of placing every complete tool schema into the initial prompt, developers can expose compact descriptions and allow agents to retrieve detailed parameters when needed. The preview supports compatible Gemini models and remote MCP servers. Google says this approach can reduce unnecessary input-token consumption, latency, and tool-selection complexity. There is no separate Tool Retrieval fee, but retrieval operations and loaded schemas still contribute to billed model tokens. Google has not published a universal percentage reduction in complete workflow costs. The economic benefit depends on catalog size, the proportion of tools actually used, retrieval frequency, and caching behavior.

PPC Land ↗

3Microsoft shifts AI FinOps from tokenmaxxing toward measurable value

Microsoft's November 19 Inside Track article describes how its internal IT organization is approaching consumption-based AI spending. The company explicitly identifies three stages of enterprise adoption: tokenmaxxing, spending controls, and valuemaxxing. Microsoft recommends measuring usage before imposing caps, attributing consumption to teams and projects, budgeting for unusually expensive periods, and tracking cost per outcome rather than merely cost per token. The company reports that individual AI consumption can vary by factors of 10 to 100 between light users and intensive agent users. Microsoft also began showing employees and vendors their token consumption and spending during November 2026. These are company-reported operating practices, not independent proof of financial returns. The significance is that AI spending governance is becoming a formal enterprise management discipline.

Microsoft Inside Track ↗

The Hidden Cost of Context

Tokenmaxxing means maximizing AI token consumption, sometimes treating high usage as evidence of productivity. Tokenminimizing means reducing unnecessary token consumption while preserving useful results. Token yield measures useful completed work relative to tokens consumed. The progression from tokenmaxxing through tokenminimizing toward token yield has a concrete engineering example in Uber's removal of unnecessary tool schemas and Microsoft's emphasis on outcome-based spending. The latest coverage also explicitly uses valuemaxxing, while fresh evidence of widespread adoption of tokenminning, tokenmining, or modelmaxxing remains limited.

Uber Engineering

Uber's engineering account explains the mechanisms behind its agent cost controls. Interactive coding sessions repeatedly send accumulated conversation history and tool results, making unnecessary context expensive across multiple turns. Uber compacts context at 400,000 tokens, defaults reasoning to medium, and uses longer cache retention for primary agents than for short-lived subagents. It also exposes MCP capabilities through command-line resolution and searchable catalogs rather than automatically inserting every schema. These settings are company-specific optimizations, not universal recommendations. Their significance is that context management, caching, and tool discovery can be treated as measurable infrastructure policies rather than individual developer preferences.

Uber Engineering ↗

Microsoft Foundry

Microsoft's November 18 Foundry FinOps guidance connects agent-level spending visibility with quality-aware optimization. The company describes workflows for identifying expensive agents, inspecting traces, evaluating model alternatives, and applying spending limits. It also documents cost reporting by project, agent, and model, along with programmatic controls and exports to Azure Cost Management. Some capabilities remain in preview. The practical distinction is that reducing expenditure should be evaluated against task-quality requirements rather than treated as an isolated objective. This makes observability and evaluation essential companions to token budgeting.

Microsoft Community Hub ↗

AI Core Updates

November 20 analysis of Gemini Tool Retrieval illustrates the potential economics of deferred tool loading. Its hypothetical example compares an agent carrying 50 complete tool schemas against one retaining compact descriptions and retrieving a single detailed schema. Under the article's stated assumptions, initial schema overhead falls from 20,000 to approximately 2,400 tokens per request before additional search overhead. These figures are illustrative calculations, not measured customer savings. The analysis also notes that output tokens and reasoning charges remain unaffected by schema deferral. The takeaway is that tool-surface optimization should be evaluated using realistic catalog sizes, retrieval patterns, and complete billing records.

AI Core Updates ↗

Microsoft Foundry Model Router

Microsoft's model-routing documentation describes policies for selecting models according to task requirements, cost, and performance. Routing can help organizations avoid assigning every request to an expensive frontier model, but the routing layer may introduce additional charges and operational overhead. Organizations should compare complete execution costs, including routing, retries, latency, and task quality. This remains relevant to Microsoft's latest FinOps guidance because model selection and spending controls are increasingly managed together rather than independently by application teams.

Microsoft Learn ↗

Research Watch

Think or Remember: The Exchange Rate of Agentic Inference

This November research investigates whether long-running agents should retain previous reasoning history or remove it to reduce context size. Its findings show that the economic result depends on whether the inference system can reuse previously computed states. Removing history can reduce visible prompt length while increasing repeated computation when valuable cached state is lost. The results depend on the evaluated models, serving configurations, and workloads.

Why it matters: Context reduction is not automatically cost reduction. Cache-aware accounting is necessary to determine whether removing information genuinely improves inference efficiency.

arXiv ↗

The SIMPLE Law for Energy Cost of LLM Inference

This November paper develops an analytical framework for estimating inference energy consumption across language-model architectures and serving configurations. It examines how model structure, sequence length, reasoning budgets, and prefix caching affect computational requirements. Its findings indicate that two workloads consuming similar numbers of tokens can impose different physical computation costs because prefill, decoding, cache management, and hardware utilization contribute differently.

Why it matters: Token prices are useful billing units but imperfect measures of underlying computation. Understanding physical inference costs helps explain why caching and serving architecture matter to AI economics.

arXiv ↗

Smart Model Routing for Order-of-Magnitude LLM Cost Reduction

Submitted November 11, this paper presents a framework for assigning requests to the least expensive model expected to satisfy a specified quality threshold. It examines task difficulty, model-price differences, routing calibration, and the overhead of model selection. A numerical example demonstrates substantial potential reductions in blended inference expenditure under selected assumptions. The research is primarily analytical and does not establish universal production savings.

Why it matters: Routing must account for the quality of completed work and the cost of making routing decisions. Selecting the cheapest model is not necessarily the cheapest way to complete a task successfully.

arXiv ↗

Phrase of the Day

“Valuemaxxing”

Valuemaxxing means optimizing AI consumption for the greatest measurable business impact rather than maximizing token usage. Microsoft's November 19 FinOps article explicitly uses the term to describe the next stage of enterprise AI governance.

  1. AI adoption
  2. Tokenmaxxing
  3. Token budgeting
  4. Tokenminimizing
  5. Token discipline
  6. Token yield
  7. Cost per successful task
  8. Valuemaxxing
  9. Verified business value

Microsoft's latest FinOps discussion offers a clear framework for understanding the shift in AI economics. The first stage emphasizes adoption and experimentation. The second introduces visibility, budgets, alerts, and spending limits. The third evaluates whether AI consumption produces sufficient business value. Uber's newly reported engineering practices provide a practical example of how organizations can support that progression. Eliminating unnecessary tool schemas, managing context size, tuning reasoning effort, and preserving reusable cached information can reduce avoidable computation. Google's Tool Retrieval preview extends the same principle to broader agent ecosystems by allowing capabilities to be loaded when needed. These developments illustrate why token yield is a useful intermediate measurement between raw consumption and business value. Organizations need to understand what their agents consume, how much of that consumption contributes to completed work, and whether the resulting work justifies its cost. The strongest measurement systems combine token accounting with task-quality evaluation, cache utilization, routing decisions, retries, and accepted outcomes. They also distinguish token counts from actual dollars because input, output, reasoning, and cached tokens may have different prices. The objective is not to minimize AI activity indiscriminately. It is to eliminate unnecessary processing while preserving the work that creates measurable value.

The goal is not to win the token-consumption contest. It is to win something worth paying for.

Microsoft Inside Track ↗

The jCodeMunch read

Uber's reported 50,000 to 70,000 tokens of potential MCP startup overhead and Google's introduction of deferred tool loading demonstrate the economic importance of avoiding unnecessary context. jCodeMunch reduces code-reading tokens through tree-sitter symbol retrieval and byte-precise context, allowing coding agents to obtain relevant code elements instead of unnecessarily broad source files. That complements selective tool loading, model routing, and caching. The larger objective is reliable completed work without repeatedly paying to process information an agent does not need.

See how the 96.3% per-lookup figure is measured →

← All editions