Token Cost Radar

1What does a cheap token mean when the AI factory costs $60 million per megawatt?

NVIDIA's October 1 infrastructure analysis says an AI factory costs roughly $60 million per megawatt and argues that tokens per second per megawatt is the metric governing earning capacity. Citing SemiAnalysis AgentX data, NVIDIA says Vera Rubin NVL72 delivers more than 30 times higher throughput per megawatt than GB300 NVL72 and up to 45 times lower cost per million tokens on DeepSeek V4 Pro. The figures come from NVIDIA and cited third-party analysis, but the framing matters: inference economics is moving from API rate cards toward useful token production per constrained physical resource.

2CFO-oriented AI economics expands the bill beyond tokens

Yarken announced October 1 an AI Economics platform that normalizes tokens, seats, credits, GPU hours, agent runs, licenses, cloud infrastructure, labor, and other AI costs and links them to projects and business outcomes. The company argues that token-focused accounting can materially underestimate total AI cost as deployments scale. It also explicitly treats changing token prices and token-budget reallocation as finance controls rather than purely engineering concerns.

3A new enterprise gateway routes every model call by sensitivity and cost

C1.ai announced October 1 an enterprise LLM gateway designed to route model calls using policy and cost while attributing each call and its expense to the user, agent, or application that generated it. The launch reflects the growing role of AI gateways as an enforcement surface for model routing, spend attribution, and governance rather than merely a common API endpoint.

4Decision-only inference puts a 4.2-cent price on a billion input tokens

October 1 coverage of TypeSafe AI's Jev lists pricing at $0.042 per million input tokens with free output and reports a 32,000-token context window. Jev returns typed probabilistic decisions instead of generated text for operations such as classification and routing. The coverage cites early-user reports of substantial speed and cost savings over LLM classifiers, while also noting weaknesses in arithmetic, dates, and noisy state. The broader economic hook is stronger than any launch benchmark: bounded decisions do not necessarily require paying a generative model to produce prose.

JEV-as-a-Judge: Accept When Confident, Escalate When Unsure

This recently updated study evaluates a decision-only judge that returns verdict probabilities instead of generated reasoning. On ordinary preference and evidence-grounded factuality, the authors report performance within three points of GPT-6 at 0.36% of its estimated fee and 0.15-second median latency. A confidence-gated cascade that escalates uncertain cases was 0.9 points more accurate than GPT-6 on 1,610 held-out pairs at 41% of its fee.

Why it matters: This is model routing pushed below model selection. If an inexpensive decision model can resolve routine cases and expose uncertainty, expensive generative inference can be reserved for cases where it is actually needed.

Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents

Growing Harness moves recurring agent control decisions out of repeatedly reconstructed model context and into reusable executable code learned from task feedback. Across BrowseComp-Plus and WebArena-Verified with deployment models from 4B to 120B parameters, the authors report 76.0% to 91.8% fewer LLM calls and 74.4% to 98.6% lower deployed-agent inference cost relative to a tool-calling agent.

Why it matters: Tokenminimizing can mean eliminating repeated reasoning rather than compressing it. Once recurring control becomes reliable executable logic, the system stops purchasing the same reasoning tokens on every run.

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows

AgentRouter assigns individual steps inside an agent trajectory to one of four model tiers rather than sending the whole workflow to a frontier model. Trained on 50,000 annotated trajectory steps, the authors report 72% lower cost than frontier-only execution while retaining 97.3% of frontier-only quality, with less than 5 milliseconds of routing overhead per step on an A100.

Why it matters: Modelmaxxing becomes more precise when the routing unit is the step. Planning may deserve frontier inference while extraction, formatting, and routine operations may not.

Toollery: Scaling LLM Agents to Thousands of Skills and Tools

Toollery treats large capability libraries as a retrieval problem rather than placing every skill and tool specification into model context. Its evaluations include a roughly 79,000-capability benchmark, BFCL-V4 with more than 440 tools, and 3,396 proprietary requests over 220 tools. The framework retrieves a compact candidate set before final LLM selection and reports improved recall over ordinary specification retrieval in its evaluated settings.

Why it matters: Tool-surface bloat has the same answer as document bloat: retrieve candidates before asking the expensive model to reason over them. Capability can scale without resident schema context scaling alongside it.