Token Cost Radar

1What happens to token economics when a model API becomes free?

November 9 reporting says LG AI Research has opened its K-EXAONE 2.5 API without usage charges through the EXAONE API Platform. The company reportedly announced the initiative at NVIDIA AI Day Seoul 2026. According to the report, LG describes K-EXAONE 2.5 as a technical-validation model and claims performance improvements exceeding 70% across 24 evaluation metrics compared with an earlier version. The report also compares company-supplied benchmark results with Chinese models including DeepSeek V4.1 Flash and GLM 5.3. These are vendor-reported measurements, not independently replicated evaluations. The reporting does not establish permanent free access, production service-level guarantees, rate limits, or the availability of downloadable model weights. The announcement nevertheless illustrates a significant competitive possibility: model providers can use free inference access to encourage adoption, experimentation, and ecosystem development. For developers, a zero-dollar API rate removes one expense but does not eliminate the costs of integration, evaluation, reliability, migration, or failed tasks.

2How do you calculate the real cost per million tokens on your own GPUs?

A November 8 Tech Insider tutorial describes a 13-step monitoring implementation for self-hosted vLLM inference using Prometheus and Grafana. The guide covers request metrics, queueing, time to first token, token-generation latency, GPU utilization, and benchmark traffic. It then connects measured throughput with infrastructure expenditure to calculate an effective cost per million tokens. This distinction is important because self-hosted inference has no conventional external API bill, but it still consumes GPU capacity, electricity, engineering effort, and operational resources. Low utilization can make an apparently inexpensive open-weight model costly to serve. Conversely, sustained demand and efficient batching can improve the economics of dedicated infrastructure. The article is an implementation tutorial rather than a controlled comparison of commercial providers. Its practical contribution is a repeatable measurement approach that can support comparisons between hosted APIs and internally operated models.

3Revolut's 90% AI coding cost reduction receives renewed attention

A November 8 OpenTools article revisits Revolut's reported migration of AI-assisted coding workloads from commercial services to self-hosted GLM models developed by China's Z.ai. The report says approximately 2,000 engineers use the infrastructure and cites a reduction of roughly 90% in AI coding inference expenditure. It also describes engineering work that reportedly reduced initial response latency from approximately 14 seconds to 1.6 seconds. The underlying migration was reported earlier in November, so this is renewed coverage rather than a new deployment announcement. Revolut's figures are company-reported results, not an independently controlled comparison. The economics depend on substantial GPU capacity, engineering expertise, workload volume, and utilization. The broader significance is that cheaper inference can change employee behavior: organizations may relax restrictive usage budgets when the cost of serving productive requests falls sufficiently. Whether that increases total business value still requires measurement.

What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute

Published October 1, this paper investigates how much computation individual generated tokens actually require. The researchers evaluate fifteen language models of increasing capacity across three model families. Their method identifies the smallest model capable of reproducing each reference token when supplied with the correct preceding context. Across three core benchmarks, a 0.5-billion-parameter model reproduced 92% to 95% of reference tokens. The most computationally expensive 10% of tokens accounted for approximately 64% to 80% of estimated floating-point operations. On MATH-500, the resulting compute-allocation map helped reduce projected routing latency from 7.59 seconds to 5.12 seconds compared with a confidence-routing baseline while slightly improving accuracy. These are experimental estimates of sufficient computation, not direct measurements of commercial API costs.

Why it matters: The study suggests that uniform computation across every generated token can be inefficient. More selective allocation of model capacity could reduce inference expenditure while preserving difficult reasoning where it matters.

TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents

Published October 15, TokenEconomizer investigates recurring token expenditure across sequences of realistic coding tasks. Its external controller identifies useful operational instructions, removes persistently irrelevant tools, and intervenes when agents enter unproductive failure loops. Across evaluated configurations using Claude Code, Codex, and GitHub Copilot CLI, the researchers report average token-cost reductions of 18.4% to 30.9% while maintaining comparable task success. Additional experiments involving web development and security auditing report reductions of 6.4% to 21.2%. These are experimental findings rather than guaranteed production savings.

Why it matters: The paper demonstrates that unnecessary tool exposure and repeated agent failures can create substantial expenditure independently of model pricing. Its results support measuring efficiency across sequences of tasks rather than isolated requests.

Beyond Speedup: Utilizing KV Cache for Sampling and Reasoning

Published January 28, this research examines using key-value caches as reusable representations for reasoning and sampling rather than treating them exclusively as decoding accelerators. The authors evaluate cache-derived representations on Llama and Qwen model families and investigate switching between faster and more deliberate reasoning modes. In selected experiments involving Qwen3 and DeepSeek-R1-distilled models, the approach reduced generated tokens by up to 5.7 times with minimal accuracy loss. The findings concern specific model architectures and experimental tasks, not a universal reduction in inference charges.

Why it matters: The research illustrates that previously computed model state can support additional decisions without requiring every intermediate representation to be regenerated. This is relevant to long-running agents, where repeated context processing and reasoning can dominate total consumption.