August 22, 2026
Today's token-cost story is about the overhead surrounding the model call. Fresh analysis finds that a browser tool can add thousands of input tokens before an agent reads the user's prompt, CoreWeave is pushing prefix-aware routing to stop agents from recomputing repeated context, and new pricing data shows an enormous gap between open-weight and proprietary inference costs. The vocabulary arc from tokenmaxxing to tokenminimizing and token yield is getting more precise: the useful denominator is increasingly the successful task, including the context, tools, caches, retries, and routing required to finish it.
TokenCost's August 21 analysis of Anthropic's published tool overhead says declaring Claude's browser toolset consumes about 6,610 input tokens on several current models and about 6,670 on Sonnet 5 before the user's prompt and browsing results are counted. The analysis puts a concrete price on tool-surface bloat: capability declarations themselves can become a recurring inference charge.
TokenCost ↗CoreWeave's August 21 engineering analysis argues that agent workloads repeatedly resend largely unchanged system prompts, tool definitions, and conversation history. It recommends prefix-aware routing so requests can reach infrastructure already holding reusable KV-cache state, reducing repeated prompt processing, time to first token, and the infrastructure cost of long-running agents.
CoreWeave ↗BenchLM's August 21 pricing dataset tracks 146 models and reports median API prices of $1.00 per million input tokens and $3.80 per million output tokens. At a 3:1 input-output blend, the open-weight models in its dataset have a median API price 81% below proprietary models, while the spread between its cheapest and most expensive tracked models reaches roughly 4,773 times.
BenchLM ↗Tokenmaxxing is the practice of maximizing AI token consumption as a proxy for productivity. Tokenminimizing is the practice of removing avoidable consumption while preserving the required outcome. Modelmaxxing means selecting the best-value model capable of completing each task. Token yield measures useful output relative to the resources consumed. Cost per successful task carries that arc to its practical endpoint by counting what the complete execution actually cost when it worked.
ITO's August 21 infrastructure analysis argues that public benchmarks and headline token prices are poor predictors of production cost. It recommends evaluating models against real workloads and measuring cost per successful task, including retries, caching behavior, reliability, and the operational consequences of model switching.
ITO ↗LLM Gateway added client-managed prompt caching on August 21, allowing applications to control their own cache markers without the gateway adding cache writes. The change reflects a maturing cost-control layer in which cache-write premiums, cache reuse, routing, and provider behavior are managed explicitly rather than left invisible inside the request path.
LLM Gateway ↗BenchLM's August 21 price index puts frontier token prices 88% below its March 2023 baseline, while noting that frontier prices rose 36.4% year over year even as mid-tier prices fell 35.8%. The split reinforces modelmaxxing: the cheapest part of the market and the frontier are no longer moving in lockstep.
BenchLM ↗BCG's enterprise token-cost framework recommends stopping avoidable spending, routing work to the appropriate model, caching repeated content, applying controls, and measuring return on AI at the workflow level. Its framing treats token economics as a governance problem rather than simply a procurement problem.
Boston Consulting Group ↗Anthropic's retrieval-based tool discovery remains the clearest reference point for on-demand tool loading. Instead of placing every tool definition in context upfront, the agent retrieves relevant schemas as needed. Anthropic reports an 85% token reduction in its evaluation while preserving access to the complete tool library.
Anthropic ↗DeepSeek's current API pricing separates model tier, cache-hit input, cache-miss input, generated output, and peak versus off-peak execution. That makes caching and time of day explicit cost-routing variables alongside model choice.
DeepSeek API Docs ↗Submitted August 20, ReCache independently caches recurring tool and skill representations so they can be reused even when resources appear in different combinations or orders. On seven public tool and skill datasets, resource-wise attention preserved invocation performance while delivering a 3.655 times time-to-first-token speedup. The complete system reduced allocated KV-tensor memory by 92.43%.
Why it matters: Tool schemas are not merely prompt tokens. They also create reusable inference state. Separating their representation from the surrounding prompt can attack tool-surface cost below the text layer.
arXiv ↗Submitted August 20, this systematic semantic-cache study compares seven eviction strategies across multiple workloads, capacities, and embedding models. LFU was the strongest simple default, but the larger finding was sobering: at one commonly derived similarity threshold, only 2.1% to 3.9% of sampled cache hits were judged answer-substitutable, turning raw hit rates of 51% to 60% into quality-adjusted rates of only 1.1% to 2.2%.
Why it matters: Semantic caching can look spectacular on a hit-rate dashboard and produce very little safe reuse. Token savings should be measured after answer-validity checks, not before them.
arXiv ↗This recent arXiv paper defines token inflation as the gap between apparent single-call cost and true workflow cost after failures and retries. The authors measure inflation as high as 4.25 times in one evaluated setting, while their router reached 94.7% accuracy on GSM8K under a fixed budget using 31% fewer tokens than the comparison router.
Why it matters: The cheapest call can produce the expensive trajectory. Model routing needs expected retry and failure cost rather than a rate-card comparison alone.
arXiv ↗This arXiv paper gives agents retrieval-based access to more than 3,000 MCP tools without mounting the complete catalog in every context. Its hybrid retrieval system reports 98% Top-15 recall, 8.9 times faster tool selection, and 23.8 times lower token usage.
Why it matters: A large tool library does not require a large permanent prompt. Retrieval converts tool-surface bloat from a fixed tax into an on-demand expense.
arXiv ↗“Cost per successful task”
Cost per successful task is the complete AI expense required to produce one accepted result, including model calls, retries, context, tool overhead, caching behavior, and routing rather than just the advertised token rate.
The likely winners are teams that optimize the complete execution path and can distinguish cheap tokens from cheap successful work.
The rate card prices the ingredients. Cost per successful task prices dinner.
ITO ↗Today's strongest findings concern repeated schemas, selective tools, and context that keeps getting repurchased. jCodeMunch applies the same principle to code reading through tree-sitter symbol retrieval and byte-precise context, reducing the tokens spent presenting code that has no bearing on the task.
See how the 95%+ cut is measured →