Cheaper Models, Bigger Bills. Welcome to the Token Paradox.

The economics of AI look promising on paper. Models are getting cheaper, GPUs are getting faster, and the cost per token keeps falling. Then why are so many organizations still struggling to manage their AI costs?

At HumanX, WEKA joined a panel of AI experts to dig into the real math behind AI spending and define a fresh take on ROI: an organization’s “return on intelligence.”

Why AI costs keep rising even as token prices fall

When it comes to what AI costs, a crucial paradox is emerging. WEKA’s Val Bercovici explained it this way during the panel: “The volume of tokens is exploding at the same time as some reductions in unit costs are happening.”

The culprit is the shift from simple chat sessions to full agentic workflows. Where a basic prompt-response interaction might consume a few hundred tokens, a coding agent running overnight could burn through millions. Add multimodal inputs, PDFs and multi-turn agent loops, and the math completely transforms.

The message is clear: Supercharged token demand is outrunning AI’s recent efficiency gains.

How to tie token spend to business outcomes

Panel moderator Hope King, founder of Macro Talk, identified the central challenge: You can’t pull back on demand because you need to move faster than your competitors, but you also need to manage costs. The solution? Focus your spending on business outcomes.

The most important shift companies can make right now is moving from measuring AI success by usage to measuring it by outcomes. Jonah Surkes, Director, Growth Equity at Generation Investment Management, put it directly: “There’s a human psychological desire to ‘tokenmaxx’ right now. When the measure of return is usage, and the measure of how well you’re doing your job is usage, you’re going to use as much as you can.”

The fix isn’t to pull back. It’s to get precise. Jonah recommends two concrete moves. First, tie every token spent to a specific business outcome, whether it’s a customer support resolution, a line of shipped code, or an analyst report delivered. Second, make architectural decisions that give you more control, whether this means vertically integrating your AI infrastructure or working with storage partners who can reduce the overhead of how your tokens are actually processed.

For builders, Val offered a specific, implementable tactic: Build an AI FinOps skill into your agent stack that monitors model pricing on platforms like Open Router in real time, optimizes for the narrowest possible gap between input token cost and cache-read pricing, and routes workloads accordingly. He noted, “As caching technologies mature, there almost shouldn’t need to be a difference between those two prices. They should very much converge.”

What the AI memory wall means for your inference costs

One of the most underappreciated constraints on AI’s ROI is at the infrastructure level, and it’s getting harder to ignore. As Val noted, the AI memory wall describes a physical bottleneck: GPU memory capacity can no longer keep pace with what agentic and multimodal workloads actually require.

The practical consequence is waste. Without sufficient memory to sustain key-value cache (KV cache) at scale, inference systems are forced to recompute tokens repeatedly, which burns energy, adds latency, and degrades output quality. For builders, the symptom is familiar as they hit rate limits, unpredictable overnight compute bills, and throughput that doesn’t scale linearly with spend. The result is an unsustainable tokenomics model.

The implication for infrastructure strategy is direct. As Val explained, “It’s really important to squeeze out all the efficiency from your inference so you do have the budget for lots of quality iterations, lots of guardrail and safety iterations.”

Who should own your organization’s AI strategy?

Governance remains one of the messiest unsolved problems in enterprise AI. Not every company has a chief AI officer, and even those that do face real questions about where decision-making authority sits.

Val described the chief AI officer or CAIO role — pronounced ciao, as he noted with a smile — less as a command position and more as an adviser. “I work very closely with our [executive team],” he said. “The job last year was to evangelize the usage of agents. The job this year is very much tokenmaxxing.” The model mirrors how security leaders operate: deeply embedded across functions, but without direct ownership of budgets.

Jonah added a less obvious but increasingly validated perspective. The chief people officer deserves a seat at the table. As agents begin replacing discrete tasks previously handled by humans, decisions about resource allocation transform into fundamental HR questions.

Dan Lawrence, general manager, Americas at Nebius, also noted that the companies poised to succeed are those where engineering and product management are converging.

“The fastest-working companies with AI are ones where the engineers are actually going out and talking to customers, helping define product,” he said, “because things happen so rapidly with model changes and automated code that it’s a really powerful phenomenon.”

How AI leaders are thinking about energy consumption

The ROI conversation doesn’t stop at the balance sheet. As AI workloads scale, so does their energy footprint, and the panelists were clear that this is a shared responsibility, not one that falls solely on infrastructure providers.

Jonah noted that while near-term demand for gas plants to power data centers in the United States has risen roughly 5x, the longer arc is more encouraging: “There is an enormous increase in demand for renewable power and battery-stored power to power data centers because it’s fundamentally cheaper.”

For Val, the energy and efficiency stories are intertwined. Performance efficiency at the infrastructure layer — the kind WEKA is built around — directly reduces energy consumption while also improving output quality. More tokens within a fixed energy and latency budget means more iterations, better answers, and safer agent responses. Val’s take on tokenmaxxing: generating maximum, high-quality tokens from your infrastructure, without waste.

Dan brought it back to a principle anyone can act on: “Sustainability is a shared responsibility between providers and consumers of services. Just like at home, you pay for your electric bill and they have to give you the electricity, but you also have to turn the lights off.” (Read more from Nebius on power efficiency.)

The bottom line

This panel highlighted that the companies winning on AI ROI aren’t necessarily spending the most. They’re spending the most deliberately. Measure outcomes, not usage. Optimize your infrastructure before you scale your spend. And don’t wait for perfect governance structures to emerge before you start experimenting.

The opportunity cost of moving slowly is, right now, too high.

👉 Watch the full HumanX panel to go deeper on AI spending, the memory wall, and what return on intelligence really looks like in practice.