# The New Math of Inference

**Published:** November 16, 2026

!["THE NEW MATH OF INFERENCE" text centered against a dark background with wavy purple digital lines.](https://cdn.sanity.io/images/ult5g8gw/production/fa3845c291fedcfed500e3a63024b56681ff7b50-960x540.jpg)

**Watch:** https://fast.wistia.net/embed/iframe/3vuh0bicyt

Your dashboard says your GPUs are busy. It doesn’t tell you what they’re getting done. Agents reuse their earlier context at every step. When that context gets pushed out of GPU memory, the GPU has to compute it again. That repeated work looks like a healthy fleet on a utilization chart, and it shows up as cost on the invoice.

## Transcript

Pop quiz. Your dashboard says your GPUs are 95% busy, but what are they actually doing?  
  
Here’s the thing. For years, we’ve graded AI on one question. How busy are the GPUs? And for training, that made sense.  
  
Training is a project and projects end. Inference doesn’t. It just keeps going. An agent doesn’t answer a question and walk away.  
  
It works a task, step after step, and at every step, it goes back over everything it already built.  
  
That context is the DNA of inference. Push that context out of memory, and the GPU has to rebuild it before it can take the next step. The same work twice. Your dashboard sees a busy GPU and calls that healthy. Your invoice doesn’t, because a busy GPU isn’t necessarily a productive GPU. It’s time to stop asking how busy your GPUs are and start asking how much useful work each one actually gives you. Per watt? Per rack? Per dollar?  
  
That’s the new math of inference.  
  
Now meet the test, the inference economic scorecard. Written by independent analysts, shaped by the people running at scale, and nobody grades on a curve. Nine measures, four rules, one test for everyone.  
  
And if a request fails or times out, it doesn’t count. Now you’ve got a number you can stand behind. Not just GPUs looking busy. A measurement for what GPUs actually get done and what that costs.  
  
So run the test, bring your numbers, then ask every vendor for theirs - including us.

## Related Videos

- [Why Inference Economics Are Reshaping Neoclouds](/video/why-inference-economics-are-reshaping-neoclouds)
- [Power, Tokens, Talent: Inside the $7 Trillion AI Compute Race](/video/compute-as-capital-redefining-ai-s-new-currency)
- [Why AI Inference Margins Are Suddenly Getting Better](/video/why-ai-inference-margins-are-suddenly-getting-better)
