Token Cost Radar

1How much can Chinese AI models lower adoption costs?

The Financial Times reports that Singapore sovereign wealth fund GIC expects competitive open-weight models from Chinese developers including DeepSeek and Moonshot AI to reduce global AI adoption costs and accelerate deployment.

Why it matters: Lower-cost models widen the routing menu and reduce dependence on premium APIs. The likely result is greater adoption, but also more total consumption as previously uneconomic workloads become affordable.

2Does every enterprise task really need the strongest model?

Reuters reports that SAP CFO Dominik Asam expects companies to use the cheapest reliable tool capable of delivering a safe outcome, whether that is conventional software, an open-source model, or a frontier system. He also warned that messy data and multi-step processes can create extremely high token costs.

Why it matters: Modelmaxxing is the practice of matching each task to the best-value model that can meet its reliability, security, and quality requirements. SAP's framing places that discipline directly inside core business processes.

3Amazon attacks Alexa's projected AI bill from every angle

Business Insider reports that Amazon is reducing Alexa+'s reliance on costly Anthropic models while expanding internal-model routing, caching predictable responses, avoiding redundant inference, using deterministic software for simple requests, and improving GPU utilization.

Why it matters: The case shows that production savings rarely come from one lever. Model choice, cache design, request classification, hardware efficiency, and orchestration all contribute to cost per completed interaction.

4AI memory demand is raising the infrastructure floor

Reuters reports that booming AI data-center demand has strengthened the pricing power of Chinese memory-chip makers, with scarce DRAM and flash capacity becoming increasingly valuable.

Why it matters: Token prices rest on physical inputs. Rising memory costs can offset efficiency gains because long contexts, KV caches, and model serving consume enormous amounts of data-center memory and bandwidth.

Delivery, Not Storage: Cue-Anchored Working Memory as a Harness Property for Coding Agents

This newly posted arXiv paper argues that operational memory should be delivered automatically by the agent harness when relevant cues appear, rather than depending on an agent to remember to write and retrieve notes.

• Reports zero voluntary memory operations across 114 evaluated turns.
• Finds 39% of intra-session rereads retrieved content already purchased before compaction.
• Uses path, symbol, semantic, event, and temporal triggers.
• Reports deterministic delivery through repeated context compactions.

Why it matters: Repeated rediscovery is repeated token spend. Cue-triggered memory can preserve useful operational knowledge without permanently loading it into every turn.

Solar Open 2 Technical Report

This arXiv report presents a Korean open-weight mixture-of-experts model designed around long-context agent work, efficient tokenization, hybrid attention, and data selected for value per training token.

• Supports a one-million-token context window.
• Activates 15 billion of 250 billion parameters per token.
• Reports roughly 24% better Korean token efficiency than the strongest global tokenizer tested.
• Uses one softmax-attention layer for every three linear-attention layers.

Why it matters: Token economics begin at model design. Better tokenization, sparse activation, and cheaper attention can reduce both training and inference resources for the same workload.

Scalable LLM Agent Tool Access in the Cloud

This arXiv paper proposes a gateway for discovering and accessing thousands of MCP tools without mounting the full catalog in every agent context.

• Scales access beyond 3,000 tools.
• Uses hybrid retrieval for tool recommendation.
• Reports 8.9 times faster tool selection.
• Reports 23.8 times lower token usage.

Why it matters: Large tool catalogs create both token overhead and selection noise. Retrieval-based access preserves broad capability without buying the entire schema warehouse on every turn.

A Two-Tier Cost Model for LLM API Caching

This arXiv paper models when stable prefix caching is more economical than repeatedly compressing or rebuilding large prompts.

• Tests an enterprise assistant with a 94,000-token tool-schema prefix.
• Reports 51.7% savings after three reuses in that workload.
• Evaluates production-scale RAG and tool-use cases.
• Finds query-aware compression can cost more than ordinary cached prompting.

Why it matters: The shortest prompt is not automatically the cheapest. Rewriting stable context can destroy cache discounts and force the provider to process the same material again.

The Harness Effect: How Orchestration Design Sets Token Economics

This arXiv study holds tasks and models constant while changing how the agent harness assembles context, exposes tools, caches prompts, delegates work, and handles failures.

• Evaluates 22 fixed tasks across six foundation models.
• Reports 38% fewer tokens per task.
• Reports 41% lower blended cost per task.
• Finds task quality remained approximately level in the evaluated sample.

Why it matters: The model is only one component of the bill. Harness design determines how often context, tools, retries, and failed work are purchased again.