# Why Inference Economics Are Reshaping Neoclouds **Published:** November 6, 2026 ![Two men seated in modern armchairs having a conversation in a studio lounge with decorated industrial shelving in the background.](https://cdn.sanity.io/images/ult5g8gw/production/704def0535492cc3f69115888c64c6d37e98f809-960x540.jpg) **Watch:** https://fast.wistia.net/embed/iframe/mws9p0o9cs WEKA’s Liran Zvibel and Silicon Data’s Steve Hou on the shift to inference, how neoclouds differentiate, and getting more tokens from the same GPUs. ## Speakers - **Liran Zvibel** — Co-Founder & CEO, WEKA - **Steven Hou** — Silicon Data ## Transcript **Steve Hou:** All right. Welcome back to Neocloud Summit in San Francisco. My name is Steve Hou. I'm the head of research at Silicon Data, and I have the great pleasure of interviewing Liran Zvibel from WEKA. For those in the audience who are not familiar, can you say a few words about what WEKA is and does? **Liran Zvibel:** At WEKA, we deliver the best storage and memory platform for AI and GPU compute. We deliver the most performant, scalable, and cost-performant very large storage system. We used to be very successful with a lot of these companies running very large training, and now we're seeing quite a strong shift toward inference. By the way, WEKA is pronounced like a Greek unit. **Steve:** Okay. **Liran:** It's a very big number: 10 to the power of 30. It's like a trillion exa, or a million yotta, so it's a huge, huge amount of capacity. **Steve:** Not quite a googol, but WEKA. **Liran:** Yeah, but a googol is also not the right word. It's almost. **Steve:** Enormous. And can I hazard a guess that the official color of WEKA is purple? **Liran:** Yes. We really like purple. **Steve:** Love it. My favorite color, too. ### [00:01:25] Founding WEKA and the Evolution of AI Infrastructure **Steve:** You co-founded WEKA more than a decade ago, long before the word “neocloud” became fashionable, which I don’t know how you feel about. How did you see AI data infrastructure come about, and how has this market evolved since the day you founded the company? **Liran:** We started WEKA in early 2014; you could call it late 2013. We couldn’t convince anyone that we actually imagined what’s going on today, but we saw three things coming. First, containers arrived with a very big push for microservices, so you could fundamentally think about new ways of building software. Second, NVMe started happening, so you could connect flash to servers, on premises and in the cloud, in a common way. And third, the network finally caught up to the server. Fast networking is also what allows us to run the very large clusters we run today. We said, “With these big technology shifts, we can build a fundamentally different storage product.” We did that. It took us a few years to make it to market. And even though we didn't plan for it, our first customers were using GPUs back in 2017 and 2018. They didn’t call it AI yet; they called it machine learning and deep learning. They were solving autonomous driving, dictation, and other autonomous problems. We kept saying, “There’s great money in that. That’s not what we planned for, but let’s take this detour and work with these crazy GPU guys, as long as the money’s good.” We kept doing it for a few years, and we kept getting better and better at supporting them. Then in 2023, the GPT moment happened. Everyone started calling it AI, and now we don’t say it's a detour. This is what we do. ### [00:03:45] From AI Training to AI Inference **Steve:** A big theme today, and something you alluded to earlier, is that AI has shifted very rapidly away from training toward much more inference. How does that impact the way AI infrastructure has to be built? The crazy GPU guys, what are they asking you for now? **Liran:** With training, it’s about what’s the biggest, craziest thing you can build to help you win, and there is almost no question of economics or even reason. Depending on how you count it, there are between dozens and hundreds of different AI labs building models. But looking at the horizon, there won’t be so many. No matter what you’re trying to win at, only one, two, or three players will win it, and you have to be the best at the end. Spending is not a problem, because if you don’t make it to the end, it all didn’t matter. So with training, it’s like Formula One. You need to build the best car, the biggest, making it from A to B in the fastest way, and allow them to train. Inference is a fundamentally different problem. Inference is how everyone is actually making money out of AI. How do you make it work in production? It needs to be scalable, and you have to make ends meet. So inference is about economics. How much am I paying for power? How much am I paying for capital? What is the overall OPEX cost? And then, how valuable are the tokens I’m delivering, and how many tokens can I deliver? It’s a fundamentally different question, and the people who do it well solve it in a fundamentally different way. ### [00:05:55] How Neoclouds Differentiate **Steve:** To follow on that, I’m generally curious. WEKA runs in many neoclouds. What differentiates the neoclouds that succeed and the ones in your estimation may not be able to make it in the end? **Liran:** On the one hand, you could say all neoclouds should be the same, but on the other hand, they’re not. Going around here, there are hundreds of people. I’m actually blown away at how many came to this show, and it’s not even everyone. The different neoclouds often have a different angle, a different thing they optimize on, and they come from different backgrounds. Some deliver the best security. Some deliver different kinds of accelerators. Some deliver unique networks. Some deliver better multi-tenancy. Some deliver renewable energy, or better access to energy. A lot of them have a knack for what they think they do best. At this point, there is such a big shortage that everyone’s winning. But in the long term, once we get through the current hump, the ones that pick the right angle will come out ahead. Not every product is one the market actually likes. You go to the supermarket, and there are many brands that deliver the same thing. Not all of them are as popular, and it’s not always easy to forecast which will be. But the neoclouds that have the right differentiator that customers want, and that are also able to execute and deliver, are the ones that will end up winning. ### [00:08:09] Single Offtaker vs. Independent Cloud **Steve:** You’ve observed there is a difference between a neocloud that serves a single offtaker as a tenant and one that operates an independent cloud, and that they behave quite differently. Can you tell us a little about that? **Liran:** To begin with, they’re funded in an entirely different way. The ones with a single offtaker get debt differently. They first find the offtaker, and then they use that offtaker for the underwriting. Usually they end up being much more capital efficient and capital effective. But the end price is also way lower. If you’re a single offtaker and you think you can take everything a neocloud can give you, you’re going to get a much better deal. I’m sure this is something you see at Silicon Data. **Steve:** Yes. **Liran:** The neoclouds that go to market to the more general public, the long tail, pay significantly more for capital. From that moment on, everything they do has to look fundamentally different for their ends to meet. It also means they need to use network and storage technology implemented in a way that delivers very good separation across customers. So not only is access to funds more expensive, running their whole stack is more expensive. If you’re a neocloud, you need to choose what kind of spending structure and operational structure you want. And if you’re an offtaker, you need to ask: Do I think I’m big enough and stable enough to commit for three, four, or five years to a large deployment I can enjoy? Or, unfortunately, I can’t do that, and I'm going to pay more on a per-usage basis. **Steve:** That’s really interesting. ### [00:10:24] WEKA Augmented Memory Grid **Steve:** One last question, and this is something I looked up before we came on. You have something called WEKA Augmented Memory Grid, and I was very curious about it. I didn’t have the chance to look up exactly what it is. Would you tell us a little about it? **Liran:** We started as a storage company. We started with a product that delivered the best parallel file system. We then augmented it with more protocols, and then we added object storage, but that still stays within the storage realm. Then we looked at inference, and we wanted to help our customers optimize it. You can optimize inference through a lot of different parameters. You can use different decoding techniques, you can use quantization, you can deliver a different model. There are many ways of optimizing your compute. But one of the biggest questions is: How much of the work do you actually have to perform, and how much can you simply reuse from something you’ve already computed? With WEKA Augmented Memory Grid, we’re taking our incredibly low-latency file system and our GPUDirect Storage technology, and we’re showing that we can push remote flash to GPU HBM as fast as NVIDIA does with Dynamo from DRAM. This is practically alchemy, modern alchemy: Getting flash to work like DRAM. What we’re doing is helping neoclouds and other inference service providers increase their cache efficiency. And the better the cache efficiency, the more tokens you deliver. The easiest example: Say we take you from a 90% cache hit rate to a 98% cache hit rate. If you’re caching at 90%, your GPUs are performing work on 10% of the queries. If you’re caching at 98%, your GPUs only have to work on 2%. That’s a 5x difference, and we’re showing some of our customers that we can take them way higher than 98%. In that example, we increased the number of tokens the same token factory delivers by 5x. So with WEKA Augmented Memory Grid, we’re helping customers leverage the improvements in compute, but really focus on getting as much as they can out of cache and getting as little work done on the GPU as possible, so they can deliver as many output tokens as possible. **Steve:** That's fascinating. Thank you very much for that explanation. I really learned a lot, and thank you so much for stopping by. Unfortunately, that’s all the time we have for today, but it was a real pleasure to speak with you, Liran, and we should chat again soon. **Liran:** Thank you very much, Steve. I really enjoyed it. ## Related Videos - [Power, Tokens, Talent: Inside the $7 Trillion AI Compute Race](/video/compute-as-capital-redefining-ai-s-new-currency) - [Why AI Inference Margins Are Suddenly Getting Better](/video/why-ai-inference-margins-are-suddenly-getting-better) - [Inference Is Eating Memory, and Tokens Now Run AI Economics](/video/inference-is-eating-memory-and-tokens-now-run-ai-economics)