Token Cost Radar

1Why are companies turning to Chinese AI models to cut costs?

The Financial Times reports that companies including DoorDash, Siemens, Airbnb, and Lindy are using or evaluating models from Chinese developers such as DeepSeek, Z.ai, and Moonshot AI. The report points to lower operating costs, stronger coding performance, open weights, and self-hosting flexibility as key attractions.

Why it matters: Model routing is becoming a global procurement strategy. Lower-cost alternatives can reduce the blended price of AI workloads while giving enterprises more control over deployment.

2How](https://www.ft.com/content/9c8ff45b-7c20-4c2e-93c9-c52339ffdcee%22}},{%22title%22:%22How) much patience is a free frontier-class model worth?

Business Insider tested Z.ai's free, open-source GLM-5.2 and found capable results on writing, research, shopping advice, and trip planning, but also encountered slow responses, capacity limits, and uneven reliability.

Why it matters: The cost comparison is no longer simply cheap versus capable. Free and inexpensive models can deliver useful work, but latency, availability, integrations, and retries belong in the total cost calculation.

3AI](https://www.businessinsider.com/chinese-glm-52-ai-model-test-2026-7%22}},{%22title%22:%22AI) pricing may need to fall 90% for broad enterprise adoption

TechRadar reports that Palo Alto Networks CEO Nikesh Arora said AI remains too expensive for widespread deployment and argued that pricing may need to decline by as much as 90% to become broadly affordable.

Why it matters: Falling unit prices have not solved the budget problem because demand, agent loops, and consumption-based billing keep expanding. Enterprise adoption increasingly depends on lowering cost per useful outcome.

4Can](https://www.techradar.com/pro/we-need-to-see-the-pricing-for-ai-come-down-palo-alto-ceo-arora-says-ai-is-too-expensive-and-needs-to-fall-90-percent-to-become-affordable%22}},{%22title%22:%22Can) enterprises govern AI spend before the invoice arrives?

Airia argues that consumption pricing, redundant context, agent loops, and MCP tool calls can make a small group of power users disproportionately expensive. It recommends centralized visibility, budget controls, routing, and context management.

Why it matters: AI FinOps is shifting from retrospective reporting to preventive control. Per-agent, per-tool, and per-workflow limits are more useful when they can intervene before spending escapes its budget.

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

This arXiv paper tests how changing the orchestration layer affects agent cost while holding models and tasks constant.

• Evaluates 22 fixed tasks across six foundation models.
• Reports 38% fewer tokens per task with the tested harness.
• Reports 41% lower blended cost per task.
• Finds task-completion quality remained approximately level in the evaluated sample.

Why it matters: Context assembly, tool exposure, retries, caching, delegation, and failure handling can move cost more than switching between nearby model tiers.

Frequency-Guided](https://arxiv.org/abs/2607.06906%22}},{%22title%22:%22Frequency-Guided) Depth Sharing for Robust KV Cache Compression

This arXiv paper introduces FreqDepthKV, an inference-time method for reducing long-context KV-cache memory and bandwidth requirements without retraining the model.

• Shares low-frequency cache components across adjacent layers.
• Retains sparse high-frequency residuals for important evidence.
• Reports a 3.9 times effective compression ratio.
• Reports higher decoding throughput and lower time to first token.

Why it matters: Long context has a hardware bill beyond its API token charge. Better cache compression can improve inference yield without changing the model.

AgentGym2](https://arxiv.org/abs/2607.06519%22}},{%22title%22:%22AgentGym2): Benchmarking LLM Agents in De-Idealized Real-World Environments

This arXiv benchmark evaluates agents across realistic environments and analyzes performance against model size, interaction rounds, and total cost.

• Finds current proprietary and open-source agents still struggle on realistic tasks.
• Shows agentic post-training improved evaluated open-model performance.
• Finds stronger agents often used a moderate number of interaction rounds.
• Highlights a persistent trade-off between performance and budget.

Why it matters: Agent quality does not rise automatically with more turns or more spending. Interaction strategy is becoming a first-class token-budget decision.

Token-Budget-Aware](https://arxiv.org/abs/2607.05174%22}},{%22title%22:%22Token-Budget-Aware) Pool Routing for Cost-Efficient LLM Inference

This arXiv paper estimates each request's token budget and routes it to either a short-context or long-context serving pool.

• Targets 4 to 8 times wasted concurrency from worst-case provisioning.
• Learns token estimates online without requiring a tokenizer.
• Separates high-throughput short requests from high-capacity long requests.
• Reports 17% to 39% fewer GPU instances on evaluated traces.

Why it matters: Cost-routing can operate below model selection. Matching request shape to serving capacity reduces infrastructure waste before generation begins.

Not](https://arxiv.org/abs/2604.09613%22}},{%22title%22:%22Not) All Tokens Are Worth Caching

This arXiv paper introduces a semantic-adaptive eviction policy for LLM prefix caches, learning which types of cached tokens are most likely to be reused.

• Finds token categories can vary by up to 756 times in reuse rate.
• Separates session reuse from structural prompt reuse.
• Learns cache priorities online from eviction feedback.
• Reports 1.4 to 2.7 times faster time to first token than evaluated baselines.

Why it matters: Caching every token equally wastes scarce GPU memory. Semantic cache policy can raise inference yield by retaining the prefixes most likely to avoid future computation.