Token Cost Radar

1Are Chinese AI models becoming good enough to make premium token pricing harder to justify?

Reuters reports October 12 that Intel CEO Lip-Bu Tan described adoption of Chinese frontier AI models as very fast, citing comparatively low token costs and capabilities he considers good enough for many applications. Speaking at a DBS-organized panel in Singapore, Tan specifically mentioned DeepSeek, Kimi, and GLM. He also expressed support for open-source development as a driver of efficiency. These are Tan's observations, not a new quantitative market-share study. Nevertheless, the comments reinforce an increasingly important procurement question: when a less expensive model meets the workload's quality requirements, what additional value justifies paying frontier-model rates?

2Why pay a generative model to answer a yes-or-no question? Microsoft introduces Decision-1

Microsoft's October 9 announcement, receiving fresh coverage October 12, introduces Microsoft-Decision-1, a specialized model for classification, routing, ranking, verification, and workflow control. Built by post-training Alibaba's Qwen3.5-9B, the model returns probability scores for predefined options rather than generating open-ended responses. Microsoft prices input at $0.042 per million tokens, with no output-token charge. In its own evaluation spanning 36 benchmarks and nearly 150,000 questions, Microsoft reports the highest accuracy among tested models and approximately 35 times lower median latency than GPT-6 Sol. Internal Xbox Research testing reportedly achieved comparable classification quality at more than 14 times the speed and 200 times lower cost. These are vendor-reported results requiring independent validation. The architectural significance is substantial: high-volume decisions can potentially be separated from expensive general-purpose generation.

3A new GPU benchmark asks what useful AI output actually costs

Tensor Machines announced TensorBench on October 8, an open-source benchmark connecting GPU performance, power consumption, and workload requirements to the cost of useful inference output. In the company's preliminary measurements, the fastest configuration of one GPU model delivered nearly 15% more tokens per second than the slowest configuration on the same inference workload. Another comparison found that a higher power cap consumed approximately 36% more power while producing only 21% more FP16 compute throughput. These results are specific to the tested hardware and workloads. The benchmark aims to measure energy consumed per accepted output token and the proportion of requests meeting service requirements. Its significance is that infrastructure cost per useful token can vary even when the GPU model and nominal hourly rental price remain unchanged.

Speedbumps: Rejection Attacks on Speculative Decoding

Submitted October 7, this paper identifies a new way adversarial inputs can increase inference costs. Speculative decoding normally accelerates generation by allowing a smaller model to draft tokens that a larger model verifies. The researchers demonstrate attacks that deliberately increase disagreement between the draft and target models, reducing the number of accepted tokens and forcing additional expensive verification passes. In some experimental configurations, the attacks made speculative decoding slower than ordinary autoregressive generation. The researchers also found that some attack effects transferred across compatible model combinations.

Why it matters: Inference efficiency is not only a performance optimization. It can become an operational security concern when untrusted inputs influence how much computation a system performs. Providers evaluating speculative decoding need to consider adversarially induced cost increases alongside ordinary throughput improvements.

Secure Speculative Decoding for Large Language Models

Submitted October 6, this paper examines security tradeoffs in lossy speculative decoding, where efficiency improvements can alter the target model's original output behavior. The authors find that some methods increase susceptibility to jailbreak and prompt-injection attacks disproportionately to their measured quality degradation. They propose SecureSD, which applies stricter verification to draft tokens early in generation, where the researchers identify much of the security deterioration. Experiments across security and utility benchmarks show improvements over evaluated speculative-decoding methods while preserving much of their efficiency advantage.

Why it matters: The cheapest inference path is not necessarily the safest one. Optimizations that reduce computation must be evaluated against security and reliability requirements, especially when the resulting model output controls external tools or business processes.

ProMCP: Profiling Token Flows and Latency Costs in Model Context Protocol-Based LLM Agents

This July ACL 2026 paper remains an important measurement baseline for the current discussion of agent tool-surface costs. Researchers profiled MCP execution across 20 servers and 169 tools, decomposing workflows into six communication stages. In configurations using customized clients, planning and schema injection accounted for 56% to 72% of total tokens and 60% to 67% of latency. In commercial off-the-shelf client configurations, final answer synthesis accounted for more than 85% of latency. Actual tool execution represented a relatively small portion of total measured cost. The results demonstrate that MCP overhead varies substantially with client architecture and deployment configuration.

Why it matters: The cost of a tool-enabled agent cannot be inferred from tool execution time alone. Loading schemas, planning actions, and repeatedly processing context can dominate the token budget. The research supports measuring the complete tool interaction and selectively exposing relevant capabilities rather than assuming every available schema must remain resident.