The token-cost debate has acquired a practical answer from one of enterprise software's biggest names. SAP is replacing expensive AI models with cheaper alternatives when they can perform the same work, a strategy the company expects to contribute to more than €2 billion in efficiency gains. GitHub has introduced Copilot Fusion, which automatically routes coding tasks among models and, according to its published benchmark comparison, delivers comparable performance at approximately half the inference cost. GitHub is also shifting enterprise customers toward centrally managed AI consumption budgets and experimenting with routing work to locally hosted models. Meanwhile, Cloudera's latest research finds that 89% of surveyed enterprises have limited or redesigned AI deployments because of token costs, and Axios reports that actual corporate token prices have fallen approximately 35% since June. New research supplies the missing economic perspective: four of five AI agents managing simulated vending-machine businesses performed worse than a conventional rules-based system, and their rankings changed when inference costs were included. The industry's emerging challenge is no longer simply making intelligence cheaper. It is determining whether that intelligence produces enough useful work to justify its consumption.
Top Developments (Last 24 Hours)
1How does SAP cut AI costs without sacrificing the quality of its products?
Reuters reported October 28 that SAP is replacing expensive AI models with cheaper alternatives wherever they meet the same performance requirements. CFO Dominik Asam said the company has already removed some of the most expensive models from products including its internal consulting assistant. SAP uses more than 70 AI models from providers including OpenAI, Anthropic, and Google, and increasingly selects the least expensive model capable of completing each task to the required standard. The company expects this strategy to contribute to more than €2 billion in AI-driven efficiency gains. Importantly, SAP does not expect its total AI expenditure to decline. Spending will continue increasing as adoption expands, but lower execution costs should help preserve margins. SAP charges customers for completed tasks rather than the underlying computation, making inference efficiency directly relevant to profitability. Reuters also reports that customers in regulated industries increasingly restrict models based on their country of origin, adding compliance requirements to model-selection decisions. The development is a concrete example of quality-constrained model routing becoming an enterprise financial strategy.
Reuters ↗2GitHub Copilot Fusion launches with claims of 50% lower coding-agent costs
GitHub announced general availability of Copilot Fusion on October 28. Rather than requiring developers to select individual models, Fusion automatically routes and orchestrates models according to the task and a user-selected effort level: Standard, High, or Max. Its execution strategies include single-model completion, cascading from inexpensive models to stronger models when necessary, and using one model to critique another. GitHub reports that Fusion Max scored 60 on the Artificial Analysis Coding Agent Index, within two points of Claude Code using Opus 5.5 Max, while consuming approximately half as many tokens, costing approximately half as much per task, and completing tasks about 40% faster. These are GitHub-reported benchmark comparisons, not guarantees across arbitrary coding workloads. Administrators can restrict which models Fusion uses. Billing remains based on the individual models actually invoked, with a temporary 30% discount for paid Copilot users. The launch demonstrates how cost-aware orchestration is moving from experimental infrastructure into mainstream developer tools.
GitHub ↗3GitHub Enterprise moves toward centrally budgeted AI consumption
GitHub announced October 28 that organizations working with GitHub or Microsoft account teams can now obtain Copilot access as part of GitHub Enterprise Cloud licensing. Under the new arrangement, Copilot access is included for licensed users, but metered AI consumption is charged against the enterprise's central budget beginning with the first billable request. Unlike conventional Copilot subscriptions, this arrangement does not include monthly AI credits per seat. Administrators manage consumption through organizational policies, spending alerts, and attribution controls. When the budget is exhausted, code completions and next-edit suggestions remain available, but Copilot Chat, code review, and cloud-agent capabilities require additional budget. Users without GitHub Enterprise licenses can obtain equivalent Copilot access through a $10 monthly standalone offering, also without included AI credits. The arrangement is currently available through account teams, with self-service availability planned in stages. Existing self-service subscriptions are unaffected. The significance is a further separation of software access from variable inference expenditure, allowing enterprises to manage AI as a shared computational resource rather than an unlimited per-seat entitlement.
GitHub ↗4GitHub adds experimental routing between local and cloud AI models
GitHub introduced an experimental local-model routing capability for Copilot Fusion on October 28. The system can select between supported models running on a developer's computer and models hosted in the cloud. On compatible Windows devices, GitHub provides managed installation of MAI Code 1.1 Flash through Windows ML. Developers using Windows, macOS, or Linux can also configure supported local models through compatible inference servers, including MAI Code 1.1 Flash and Qwen3.8 Flash Next. The feature currently requires opting into experimental functionality and selecting Fusion Standard. GitHub emphasizes that local execution does not make the overall session offline because routing decisions still involve cloud services. The company has not published a controlled cost comparison demonstrating that local execution is cheaper than hosted inference. Actual economics depend on hardware availability, electricity, throughput, and model quality. Nevertheless, the capability extends cost-aware routing beyond choosing among commercial APIs. It introduces the possibility of assigning suitable work to already-owned computing resources.
GitHub ↗5Alphabet's $9.1 billion quarterly cash burn puts AI infrastructure economics under scrutiny
Reuters reported October 28 that Alphabet recorded approximately $9.1 billion in negative free cash flow during its third quarter as spending on AI and cloud infrastructure accelerated. The company reported revenue of approximately $127.6 billion, slightly below analyst expectations, while Google Cloud revenue more than doubled to $30.71 billion. Alphabet expects full-year capital expenditure of approximately $195 billion to $205 billion and indicated that investment could increase further in 2027. Investors reacted negatively to the combination of enormous infrastructure spending, weaker-than-expected advertising results, and pressure on cash generation. Google Cloud's strong growth demonstrates substantial demand for AI-related computing, but does not independently establish the profitability of every new infrastructure investment. The report highlights the financial tension underlying declining inference prices. Customers increasingly expect cheaper computation, while infrastructure providers must finance equipment, facilities, networking, and energy capacity before realizing the revenue expected from future demand.
Reuters ↗Enterprise AI Moves From Consumption Targets to Economic Accountability
Tokenmaxxing means treating high AI token consumption as a desirable outcome in itself. Tokenminimizing means eliminating unnecessary token consumption while preserving useful results. Token yield measures useful completed work relative to the tokens consumed. The progression from tokenmaxxing through tokenminimizing toward token yield is becoming increasingly visible in enterprise purchasing decisions, developer-tool pricing, and research into agent productivity. Today's coverage provides evidence that organizations are beginning to evaluate AI through successful outcomes, spending accountability, and computational efficiency rather than adoption statistics alone. Searches did not establish meaningful new momentum for the spellings tokenminning or tokenmining, or for modelmaxxing.
ITPro
ITPro's October 28 examination of Cloudera's State of Enterprise AI 2026 report finds that 89% of surveyed organizations have limited or redesigned AI deployments because of rising token consumption. The underlying research surveyed 1,500 global IT leaders and decision-makers in September. Another 45% reported moving workloads in response to higher or more volatile costs. Despite these pressures, 55% increased AI investment. Only 23% said their AI initiatives generated more value than they cost, while 58% reported approximately equal value and expenditure. Another 27% reported little or no measurable business impact. Cost observability remains incomplete: 90% said they could measure end-to-end AI expenditure at least somewhat accurately, but only 27% expressed high confidence in that measurement. These are self-reported survey findings, not audited financial outcomes. The results suggest that enterprises are increasingly concerned with the economics of AI execution, even as the advertised price of individual tokens declines.
ITPro ↗Axios
Axios reported October 28 that OpenAI's introduction of GPT-6.1 Terra has intensified competition over inference pricing. The publication cites an index maintained by corporate expense-management company Ramp, which estimates token prices at approximately $0.59 per million as of October 23. That represents a decline of roughly 35% since the end of June. Unlike indices based exclusively on published API prices, Ramp's measurement incorporates actual corporate spending on OpenAI and Anthropic. Its scope is therefore narrower than the entire inference market, and the figure should not be interpreted as a universal model price. Axios also highlights a potential problem for infrastructure investors: growing token consumption does not necessarily produce proportionate revenue growth when prices decline. The relevant financial measure is total spending, which combines the quantity of tokens consumed with their effective price. Lower unit costs can encourage adoption while simultaneously pressuring provider revenue per unit of computation.
Axios ↗The Indian Express
In an interview published October 29 in India, Microsoft CoreAI executive Jay Parikh described how enterprises can manage increasingly expensive developer-agent workloads. Parikh said organizations can restrict available models, inspect consumption reports, attribute expenditure, and establish hard spending limits. He illustrated the controls with a hypothetical allowance of $100 per developer per week. That figure is an example, not a newly announced universal Microsoft spending cap. Parikh also discussed GitHub Copilot Fusion's ability to select affordable models automatically rather than relying on developers to switch manually after completing demanding tasks. The interview emphasizes that model diversity serves financial and operational purposes, including business continuity and avoiding dependence on a single provider. These controls are particularly relevant to organizations transitioning from predictable software subscriptions to variable inference charges. As agents execute longer workflows, spending governance increasingly requires visibility into the models selected and the computational resources consumed.
The Indian Express ↗GitHub Engineering
GitHub's technical explanation of context handling and model routing, published in June, gains renewed relevance following the October 28 Fusion launch. The company identifies recurring instructions, conversation history, repository information, and tool definitions as major contributors to coding-agent input consumption. Its approach combines prompt caching with deferred tool loading. Instead of placing every available MCP or application tool schema into the model's context, tool search allows relevant definitions to be retrieved when needed. Stable prompt prefixes can also be reused through provider caching mechanisms. These techniques address different costs: deferred loading reduces unnecessary context, while caching discounts or avoids repeated processing of context that remains necessary. The article does not establish a universal percentage reduction applicable to every coding session. Its importance is architectural. A broad collection of available tools need not require an equally broad collection of resident tool definitions. The October 28 introduction of automated model orchestration makes these earlier context-efficiency improvements part of a larger cost-control strategy.
GitHub Blog ↗Tidelines
Tidelines' October 27 inference-spending snapshot illustrates why model-market share should be measured in both tokens and dollars. Using OpenRouter rankings and catalog prices, the service estimates approximately 25 trillion tracked tokens per day and $30.4 million in daily expenditure at published rates. Its usage-weighted blended price is approximately $0.465 per million tokens under its specified input-output weighting. The service reports that average token prices declined 46.6% over the preceding 90 days while estimated daily spending increased 103.8% over 30 days. These are modeled expenditure estimates, not verified customer payments. The calculations exclude discounts, cached-token pricing, and free traffic, and the underlying observations represent OpenRouter rather than the entire inference market. Nevertheless, the divergence between falling unit prices and rising estimated expenditure demonstrates the importance of measuring consumption and spending separately. Organizations can achieve lower prices per token while still facing rapidly increasing bills if workloads expand sufficiently.
Tidelines ↗Research Watch
AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations
Submitted October 27, AutoVend evaluates whether language-model agents can operate economically sustainable businesses over extended periods. The benchmark assigns an agent responsibility for 20 vending machines across five types of locations over 180 simulated days. Agents manage inventory, purchasing, replenishment, expiration, and recurring operating expenses. Five LLM agents were compared with a conventional rules-based inventory-management system. Four of the five performed worse than the rules-based baseline. Particularly significant is the finding that model rankings changed when returns were measured relative to inference expenditure rather than business returns alone. Improvements in individual operating metrics did not consistently translate into greater profitability because inventory commitments and execution costs could erase those gains. These are simulated business outcomes, not evidence from actual vending-machine operations.
Why it matters: The paper directly tests whether more sophisticated AI execution produces enough economic value to justify its computational cost. It demonstrates that a model producing better individual decisions can still deliver inferior financial results. For autonomous agents, the meaningful comparison is not simply task success or token consumption, but the economic value produced after accounting for inference and operational expenses.
arXiv ↗Does Your Generalist Skill Optimizer Really Incentivize Agentic Tasks? Rethinking Skill Optimization via Bipartite Attribution
Submitted October 27, this paper examines whether automatically rewriting an agent's procedural instructions actually improves its performance. The researchers evaluated three skill optimizers across three agentic benchmarks and two models. Although optimized instructions appeared to improve results on small development sets, those gains largely disappeared on held-out tests. Across the evaluated configurations, optimized skills improved success by only 0.1 percentage points on average while increasing input-token consumption by approximately 16%. The researchers found that optimization frequently added instructions requiring agents to recheck work or perform operations one call at a time. Those additional steps increased conversation length and caused accumulated context to be transmitted repeatedly. Their BiPA methodology traces token consumption to the instructions and execution behaviors responsible for it.
Why it matters: Longer or more elaborate agent instructions are not automatically better. Adding procedural rules can increase model calls and repeated context consumption without producing measurable improvements in successful task completion. The research provides a concrete example of why agent optimization should evaluate quality and computational cost together.
arXiv ↗Metacognitive Decoding: Efficient Inference for Diffusion Language Agents
Submitted October 27, this paper introduces a lightweight controller that adjusts how diffusion language models allocate computation during agent interactions. Diffusion language models can generate tokens through parallel refinement, but conventional decoding approaches may spend unnecessary computation refining responses or generating more text than a task requires. The proposed controller learns when tokens are sufficiently complete and how far generation should continue, using agent trajectories to balance response quality against computational expenditure. Across five evaluated benchmarks, the researchers report an average 20.6% reduction in inference floating-point operations compared with their most efficient baseline while maintaining competitive task accuracy. The results are experimental and apply to the evaluated diffusion architectures rather than conventional autoregressive models generally.
Why it matters: Inference efficiency can improve through decisions made inside the generation process, not merely through shorter prompts or cheaper model selection. Learning when additional computation is unnecessary could reduce serving costs while preserving useful results. The paper illustrates how computational budgeting may increasingly become an intrinsic model capability.
arXiv ↗Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents
Submitted October 26, this study investigates implementation errors that prevent AI applications from obtaining expected prompt-caching benefits. Researchers examined 120 caching-related pull requests across ten popular open-source agent systems and identified recurring problems involving incorrect cache configuration and inaccurate usage accounting. Many errors produced no obvious runtime failures, allowing unnecessary inference expenditure to continue unnoticed. The researchers developed CACHELINT, which combines static analysis with checks against provider-specific caching requirements. On the labeled dataset, CACHELINT achieved 0.71 precision and 0.78 recall. A broader evaluation across 571 agent projects identified 32 caching mis-integrations at 0.86 precision. The researchers report 21 developer confirmations and 18 upstream fixes, including a correction in Anthropic's official SDK.
Why it matters: Prompt caching is frequently treated as an automatic source of savings, but its benefits depend on correct integration and accurate accounting. Silent configuration errors can increase costs without disrupting application functionality. The study demonstrates why production AI cost governance needs verification of actual cache behavior rather than relying exclusively on configuration settings or advertised discount rates.
arXiv ↗Phrase of the Day
“Return on Token”
Return on Token is the useful business value generated relative to the AI token expenditure required to produce it. The concept receives a particularly strong practical illustration today. SAP is replacing expensive models with cheaper alternatives while preserving task quality, Cloudera's enterprise survey finds that only 23% of respondents believe their AI initiatives generate more value than they cost, and new research demonstrates that agent rankings can change when inference expenditure is included in the calculation.
- AI adoption
- Tokenmaxxing
- Token shock
- Tokenminimizing
- Token discipline
- Quality-constrained model routing
- Token yield
- Return on Token
- Cost per successful business outcome
The vocabulary progression reflects a change in what organizations are trying to optimize. Early AI adoption programs encouraged employees to experiment, use more tools, and consume more tokens. That made sense while organizations were discovering useful applications, but token consumption eventually became an unreliable substitute for productivity. The next response was to impose spending limits and reduce unnecessary usage. Those measures can control expenditure, but they do not independently establish whether the remaining computation is valuable. Today's developments illustrate the next stage. SAP selects the least expensive model capable of meeting a task's requirements because the company earns revenue from completed work rather than from the amount of inference consumed. GitHub's new orchestration system similarly attempts to preserve coding performance while reducing the computation required to achieve it. Cloudera's survey shows why this distinction matters to enterprise buyers. Although AI investment continues increasing, relatively few respondents believe their initiatives generate more value than they cost. AutoVend adds experimental evidence from autonomous business operations: four of five evaluated agents performed worse than a conventional rules-based system, and accounting for inference expenditure changed their relative economic rankings. These findings do not establish a universal method for calculating business value per token. Outcomes differ across applications, and measuring their value can be difficult. Nevertheless, the direction is clear. An organization should evaluate whether additional computation produces additional useful work, whether that work meets quality requirements, and whether its value exceeds the complete cost of execution. A smaller invoice is useful. A better relationship between expenditure and results is the actual objective.
- verified business outcomes
- quality-adjusted task completion
- cost per successful workflow
- task-specific model routing
- automated reasoning-effort selection
- selective context retrieval
- on-demand tool-schema loading
- effective prompt-cache utilization
- reduced unnecessary agent turns
- reusable intermediate results
- local and cloud inference optimization
- project-level spending accountability
- outcome-based AI pricing
- measured financial returns
- quality-preserving token efficiency
Tokenmaxxing asks how much AI you used. Tokenminimizing asks how little you can get away with. Return on Token asks whether the work was worth paying for.
ITPro ↗