Why the AI Memory Wall Depends on Storage, Not DRAM

00:00 The “DeepSeek” shock and frontier lab distillation

Val Bercovici: DeepSeek kind of went away for 484 days — more than a year. A lot of us were thinking, yeah, they’ve got this great reputation, but are they ever going to come back? Are they just gone? Then boom, version 4 came out, and people are still trying to catch up to it.

Michael Smith Jr.: The Elon Musk / OpenAI trial was very revealing, because under oath Elon admitted that even the frontier labs distill from each other. Were you shocked?

Val: Not shocked, but pleasantly surprised it was out in the open. It’s pretty well known that everyone already distills from everyone else — even the big models, even the frontier labs.

Michael: The accusation that the Chinese labs are doing it seems a little self-serving.

Val: It’s a half-truth at best. They’re all doing it.

01:10 Is AI a bubble? Unit economics vs. token demand

Michael: It kind of feels like AI is everywhere. Some people are saying it’s a bubble. Let’s separate the stock market from what’s truly going on. Are we in a bubble?

Val: No. But I understand why people think we are. We’ve honestly never seen these amounts of capital flowing into any industry this fast. On the other side, the unit economics are not great for a lot of the big providers. Some have publicly said they’re at negative gross margin right now. So the unit economics have to get better.

The reason I’m confident we’re not in a bubble is that the comparisons to the dot-com boom — the Cisco moment, the fiberoptic overprovisioning — are wrong. The proof point is a site called OpenRouter. If you watch it regularly like I do, you’ll see token consumption spiking up and to the right continuously — almost a geometric, super-linear hockey stick. That is real-world, paid token traffic. It may not represent 100% of all token traffic, but it’s some meaningful portion — 3%, 5%, 15%, some people say — and that is the clearest indicator that we literally have more demand for intelligence than there is supply on the globe right now.

Raz Kotler: It’s a great point. What happened with Uber recently kind of feeds the bubble narrative. Their CTO was on stage at a conference in San Francisco saying they blew their entire annual token budget by April.

Val: Exactly. The bills came in.

03:00 Beyond chat: The era of agentic harnesses

Raz: Most people know AI is everywhere but don’t understand what’s underneath. Can you help simplify the full stack for a business audience?

Val: The first thing is to avoid what I call AI psychosis. Executives are especially guilty of this. They all had their ChatGPT session two or three years ago, and maybe they still chat with a good model today. But chat is not what AI is today, and certainly not what production AI is.

Chat is a short context window: just a few sentences, maybe a PDF attachment, one prompt, one response. Almost all of the really popular, high-revenue AI you’re seeing from Anthropic, OpenAI, Codex, and others is an agent harness. It’s a very engineered application with a set of rules: a system prompt not just for the model, but for the agent; a user prompt with a goal like “write me this app”; the goal is split into subtasks; those subtasks are spawned and invoked in parallel — 10, 20, 100 subtasks — and iterated in a loop. Nowadays, the leading long-horizon agents run for days or weeks.

You’ve got multiple dimensions: long context, multiple turns, and high concurrency with all these parallel subtasks. To process that requires about 1,000 times more infrastructure than a chat session.

04:45 The memory wall: Why infrastructure is the bottleneck

Val: You hear about the “memory wall” very often, and that’s literally the working memory we have to create. When you take a goal for an agent and all the subtasks that are spawned, there’s an enormous amplification. 100,000 tokens — considered small today, though it was very large last year — still requires about 5 to 10 GB of memory, even with the new compressed KV cache approaches. That’s a lot of working memory.

We’ve got the compute, but there’s an imbalance. We have enormous processing capability with Blackwell and the upcoming Vera Rubin processors, but not enough memory. We’re always evicting memory. By the time we build up these KV caches across 100 parallel subtasks, we’ve run out of memory and we’re recomputing the same KV cache wastefully, over and over again.

Michael: How does that bottleneck get fixed? Is it “just make more memory,” or make the agentic harness more efficient?

Val: Both — but let me first define KV cache, because I think most people are like, “What the hell is KV cache?”

06:23 What is KV cache? The “RAM of AI” explained

Val: Jensen (Huang) did a really good job explaining this at CES earlier this year. KV cache is the working memory of AI. Just like we spec our phones by RAM and storage, KV cache is the RAM of AI. It’s QKV: query, key-value matrices.

Models have weights — very intelligent weights, given how models are trained today. When you send a prompt to a model, you’re literally bending the weights. So you take the built-in weights for the model, and your prompt bends them to give you the specific output you want.

Take that 100,000-token prompt: You’re adding up to 20,000 dimensions of what each word means in what context. Is “light” luminescence? Is it weight? Is it humor? That’s what attention is: You’re putting relationships between all the words, paragraphs, sentences, attachments, diagrams, videos, and audio streams, whatever you include. KV cache is the attempt to store all those interconnections — up to 20,000 dimensions of meaning — to these multimodal inputs and create actual meaning out of them. From that you get reasoning tokens and output tokens.

08:10 DeepSeek’s innovation: Sophisticated layers and efficiency

Michael: There’s this battle of closed source versus open source, and even a theory that China is dumping open source on the market to snuff the revenue oxygen from closed source. As a technologist, what’s your take on DeepSeek?

Val: DeepSeek is highly respected for two reasons. The obvious one is they publish more than any other open or closed lab. They write extremely well, extremely elegantly, and they publish on genuinely novel algorithms. Double credit there.

For me, the key thing is that they did something rare. After DeepSeek shocked the West with V3 and then R1, every other lab — Kimi K2, Moonshot, MiniMax, Qwen, Zu AI, GLM — pretty much published new models every month. DeepSeek went away for 484 days. A lot of us thought they were just gone. Then boom, V4 came out, and it is so intricate, so sophisticated, particularly in that it has five layers of attention algorithms intertwined. No one has publicly done that. It’s rumored that Anthropic and OpenAI have that level of sophistication, but they don’t talk about it, they don’t publish.

DeepSeek published, and they’ve done so much innovation that the entire inference server ecosystem — vLLM, SGLang, TRT-LLM communities — is struggling just to keep up and serve it optimally. And then you get into what we do with KV cache offloading: There’s already a struggle to do the simple offload from HBM to DRAM. What we’re doing goes from DRAM to storage, or in the most advanced case, we optimize directly from the GPU’s HBM to storage, bypassing the CPU and DRAM entirely. That is bleeding-edge right now, and it creates real efficiencies at runtime from a cost perspective because tokens are expensive.

11:47 Software-defined memory: WEKA’s Augmented Memory Grid

Val: To answer your earlier question about how the bottleneck gets fixed, WEKA is literally making more memory. There’s an older term in the industry called software-defined storage, and technically we’re in that category. The product we have, Augmented Memory Grid, is software-defined memory.

We take the built-in hardware in a GPU server — the GPUs, the high-speed networks, and the embedded scratch-space NVMe drives — and install WEKA software called Axon, which is a hyperconverged piece of software, on those servers. When Linux boots up, it sees a file system. When you load a GPU kernel into the GPU, it sees limitless memory — memory at storage capacity, easily petabytes, potentially an exabyte of memory. That’s how we get around the memory wall and help scale it, because now there’s no reason to evict memory anymore. You’ve got abundant memory for GPUs for the first time.

The way you can recognize when people are running this next level of sophistication is when they talk about garbage collection of KV cache, which was almost an oxymoron until recently. That’s where the state of the art is right now.

13:34 Global competition: Is China vs. U.S. competition good for AI?

Michael: From a technical perspective, DeepSeek is truly innovating, right? And the tension in these models is actually net good for all of us?

Val: Very much so. There’s a prolific researcher named Nathan Lambert who did a deep dive into China recently, and he was noticing what’s obvious if you go there. Yes, there’s some state influence — like with DeepSeek now — but MiniMax is just a raw startup, Zu AI is just a startup, Alibaba is very commercial with Qwen. I think it’s super healthy what’s going on right now. You’ve clearly got what I would say personally is an over-capitalization of the frontier labs in the U.S. right now, but you’re getting amazing results — truly frontier models and frontier tokens from the Opus class, the GPT high-end class. And at the same time, the biggest trend is model routing.

14:48 Model routing: Tokenmaxxing vs. signalmaxxing

Val: It’s funny how tokenmaxxing kind of rose and died within about two months once the bills came in, which was pretty obvious. It’s just like the cloud days 15 years ago, where the month-two bill reset everyone’s expectations. So now tokenmaxxing is out and token capping is in — we call it signalmaxxing at WEKA.

Once this is in place, you look at the middle-tier model. Anthropic’s branding is brilliant: Opus at the top, Sonnet as the middle tier, Haiku at the low tier. OpenAI, worst naming company in the world: You’ve got GPT 5.5 Medium, GPT 5.5 Low, Extra Low, and all these weird variations.

Simplistically, you’ve got three tiers. For orchestration, deep planning, or really heavy analysis, you pay the premium token tax. But agent harnesses now use the Sonnet class and the Haiku class for the bulk of the work. And there’s no dispute that DeepSeek V4 Pro, Kimi 2.6, MiniMax M3, and so on are at least at the Sonnet class, with DeepSeek Flash at the Haiku class. With routing — either inside the agent harness or by offloading to OpenRouter — you get automatic optimization of token usage. As soon as you go below the frontier class, prices plunge 90%.

16:13 Pricing wars: DeepSeek’s 87x cost advantage

Val: DeepSeek continues to shock the world. Not necessarily with the highest model quality, though they’re up there. They shock with pricing efficiency, and particularly with cache repricing, which is directly related to KV cache.

In China, their cache repricing is 87x lower than even DeepSeek in the U.S. or Singapore. The hosted model of DeepSeek in China is around $0.365 per million cache-read tokens. Something is fundamentally different going on in China. You can only speculate: Are they getting free power from the government? Free processors from Huawei? But this is not an arbitrary price. It’s the most important price because more than 80% of all tokens for agents are cache-read tokens. So the cache-read price is the forefront of pricing across all models right now.

Michael: Why wouldn’t China want to give them whatever they need to remain competitive and get this on the market at a lower price?

Val: They would, and I’d argue they’re doing much more than just being competitive. They’re completely leading the market, but doing it in a very domestic way. They’re almost subsidizing it for Chinese usage. Then when they sell overseas, they price more normally. And for any kind of public organization overseas, it’s not even allowed to use a model inference out of China anyway.

18:00 Coding with agents: From Cursor AI to Claude Code

Michael: Which model would you code with? And how does WEKA feel about AI writing code?

Val: My main role a year and a half ago was convincing our very skeptical, brilliant engineers to actually use Cursor AI and coding agents. There was a lot of resistance early on, which has completely flipped now. They’re way ahead of me in terms of how aggressively they’re adopting it.

It’s still very subjective. Anthropic did a brilliant thing: Knowing a bunch of CTOs and technical leaders would be on holiday at Christmas, they gave very generous rate limits and discounted subscription pricing and got every technical leader at every major company hooked over the holidays. By January, everyone was telling colleagues, “This stuff really works now.” Claude Code, even though it had been out since May, hit an inflection point around December — I think it was Opus 4.5 back then — and this combination was an incredible step function in capability.

Then Codex came out shortly after. Pi, OpenCode, and a bunch of open agents are very popular. Market share-wise it would be Claude Code, then probably Codex, then OpenCode and Pi.

The most interesting trend to me isn’t even coding agents anymore. OpenClaude came out in December too, and it’s very much a developer and hobbyist tool. But Hermes Agent is an unbelievable dark horse. It came out of nowhere, and I think it now processes three or four times the tokens of OpenClaude. It’s shocking how quickly it’s risen, and it’s truly a hobbyist tool because it’s easier to use. You need less developer experience to be functional with it.

20:10 The rise of 24/7 cybersecurity agent swarms

Val: The killer app right now isn’t even coding agents for enterprises. It’s going to be 24/7 cybersecurity agents; agent swarms, or blue agent swarms.

Raz: I’m putting on the hat of a CIO listening to you. I’m convinced I need to look at my memory, my CPUs, my servers. Now I understand what KV cache is. Give me a very simple readiness plan: Should I invest in servers, GPUs, WEKA? What do I do tomorrow?

Val: I always recommend, unless you’re a Fortune 50 or 100, having a managed security service provider. MSSPs (managed security service providers) use major tools like CrowdStrike, SentinelOne, Palo Alto Networks, and so forth. What’s changing right now — and this is a quick tangent into the SaaS apocalypse — is that software is no longer just compiled code or a smart algorithm. It’s inference. Agents are continuously processing and generating tokens. If you look at what’s happening with Mythos as a category of model: In its initial release, in 31 minutes you could point it at a sophisticated, mature piece of software and it would find vulnerabilities and write the actual exploit. In 31 minutes.

21:50 The CISO playbook: Agentizing the OODA loop

Val: What a CISO is doing is essentially staffing — either contracting from an MSSP or in-house — a security operations center. Fundamentally, what a security operations center does is an OODA (observe, orient, decide, act) loop: They’re observing what’s going on throughout the organization and the supply chain, orienting based on threat intelligence signals, deciding whether there’s an incident to declare, and acting if there is one.

This OODA loop now needs to be agentized if it isn’t already because your adversary has gone from a sophisticated supply chain of human operators who can walk through a kill chain to just doing script-kiddie attacks with agents.

Raz: The challenge from a CISO perspective is that vulnerability management has been around for two or three decades. When you say Mythos, it’s something I need to manage, but does it actually relate to my business crown jewels? Does it represent a real risk reduction priority? Most CISOs I speak to say, “We don’t have a relevant plan.” The board, the CEO, the CIO, the CTO — they all ask what the plan is for Mythos. And the honest answer is: Take your ransomware playbook, replace the name “ransomware” with “Mythos,” and then accelerate the execution speed by 100x.

Val: And you know what always ends up happening in any security conversation is you don’t pay this year even if you’re exploited. You pay next year when your cyber insurance bill comes in. If you don’t have the right certifications, or if you’ve been exploited, you’ve got a very different bill. The same thing is going to happen with Mythos. Whether you get exploited or not, there’s going to be a new bar that all the insurers set in terms of your readiness for red agent swarm attacks.

24:35 The future of vulnerabilities and cyber insurance

Val: Remember what I said about DeepSeek: When you have 87x lower token costs, you can just swarm, just let it go. I think we’re going to have another two years before the Mythos moment looks like the WannaCry attack. I know that sounds alarming, but it’s taking time. It’s agile and trendy to talk about, but there is still time. My best advice to any CISO: Get your readiness program together now.

Raz: Your insurer is going to demand you have one next year. So you don’t really have the time.

Val: The vulnerabilities have already been documented, they’re already there. So the question is when you get exploited, not if. What’s fascinating is the whole concept of “zero days.” These were really valuable. You’d hold them back and only burn them when you needed to. Now they’re practically useless in that sense because you can burn zero days at will; Mythos keeps pumping more out of the backlog.

To me, this indicates the first wave of these kill chains is nation-state kill chains, and there’s a lot of dwelling happening right now. The actual public exploitation, the ransomware wave — this hasn’t happened yet, but it’s inevitable. Whether it will be some sensational moment or a series of continuous attacks, we don’t know. But we’ve learned that staying the course and waiting for something to happen is a dangerous approach.

26:14 AI factories: From pre-Model T to token warehousing

Michael: Tell us more about AI factories. What does it mean?

Val: The AI factory analogy is literal: You take raw materials — data and prompts — you’ve got an assembly line, the model, and you generate output tokens. What fascinates me is that we are pretty much at the pre-Model T, pre-assembly line part of AI factories, precisely because of the way KV cache and GPUs work. As soon as we’re out of a very short assembly line, we have to go back and re-create the same materials again because we don’t have a long enough assembly line.

The most interesting concept is what we call token warehousing. Think about Amazon: When you order from an Amazon marketplace seller, it doesn’t ship directly from the factory to you. That would be inefficient. Amazon is legendary for warehousing. That’s what we don’t yet do at scale in AI factories.

Michael: What would you warehouse, though?

Val: Technically it’s the attention, but we call it warehousing the tokens. When you do the prefill operation — the compute-intensive part of inference — you process it in KV cache and quickly evict it because you’re out of physical memory. You then have to regenerate those materials before you can ship the output. If you’re able to warehouse the product, you don’t have to regenerate it redundantly. It’s extremely efficient in terms of energy.

Michael: Make it real-world. What’s an agent harness where I’m warehousing something?

Val: Let’s say you generate an app, an interactive website with 10 pages, authentication, logging, vulnerability scanning, all the things you’d do in production. You’ve got a common code base that occupies a certain amount of memory. As soon as you spin up a second or third subtask — design, back-end, vulnerability scanning — you’re consuming enough memory that you run out and have to evict a portion of that code base. You have to regenerate the embeddings. You’ve taken the same code base, the same prompt, and you have to regenerate that portion of KV cache because something newer or fresher evicted it. Evicting and regenerating is wasteful; it adds latency, energy consumption, and it means you need more GPUs than you otherwise would if you only generated the tokens once.

With a warehouse, you do the prefill, generate the KV cache, process it, and then warehouse these tokens in memory, keeping them very fresh. You’re processing only new tokens. You’re reading the tokens you already kept in memory instead of regenerating them. You’re never evicting. The whole thing just moves forward instead of going back to the factory every single time.

29:43 Limitless memory: Bypassing the CPU DRAM bottleneck

Michael: That memory — in this KV cache model — where is it actually sitting?

Val: The trick is to get a portion of the KV cache into the GPU to do the matrix multiplication. You can do this by putting it temporarily in the server motherboard’s CPU DRAM and bringing it back, or by using storage. NVIDIA has a whole program inspired by us called CMX, or Context Memory Extensions. In the Vera Rubin era with Bluefield 4, when that ships later this year, you’ll be able to have a reference hardware architecture that tiers from high-bandwidth memory (VRAM) down to CPU DRAM (DDR5), and all the way down to SSDs.

We have an option above and beyond that: We go directly from the GPU to the storage, bypassing the CPU DRAM entirely, at maximum network bandwidth. This means you effectively have limitless memory. You just bring directly into the HBM anything you need when calculating the output token.

We documented this recently with Oracle Cloud Infrastructure (OCI). OCI ran the test (not us, because it’s in their marketplace) and demonstrated the ability to generate 10x the number of tokens, 10x the concurrency. So instead of buying 10x the cluster size or 10 extra data centers, on the same hardware, the same watts, the same energy profile, you process 10x the concurrent tokens.

Michael: So you think companies building memory are going to keep exploding because every conversation comes back to memory — VRAM, DRAM, KV cache?

Val: Yes and no. There were some interesting headlines last week that I kind of predicted with my friends at SemiAnalysis. They broke the story that NVIDIA, at the last minute for Vera Rubin, decided they don’t need as much DRAM because it’s expensive, and most manufacturing capacity is dedicated to making HBM, which is on the critical path. More HBM production means less DRAM, and it’s zero-sum.

Now with CMX coming out and Augmented Memory Grid, you’re able to go from the HBM, bypassing DRAM, directly to storage. At scale, you don’t even need as much DRAM. NVIDIA is saying they’re going to ship half the DRAM on their motherboards because of this capability, partly to lessen dependence on third-party components and partly to enable more HBM production. You can still inference at scale without a bottleneck and save on bill of materials. It’s just there’s not enough memory, and if you can sacrifice some DRAM while preserving capability, why not?

34:00 The Jevons Paradox: Why efficiency creates more demand

Val: I’m surprised we haven't used this term yet: the Jevons Paradox. It’s undefeated. When you reduce the cost of a valuable commodity, usage and demand skyrocket. Our partners at Nebius have done this math: For every 100x reduction in the unit cost of tokens, there’s been a 10,000x increase in demand and consumption. The net difference is 100x more demand every time we improve things. That’s why the bubble conversation is not black and white; it’s a wave.

34:40 The “Dumb Down” theory: Compute starvation and quantization

Michael: I’ve been trying to solve something for myself. I use both Codex and Opus on a personal project, and I’m suddenly finding Opus acting kind of stupid. Other people are saying the same thing. Is it possible that it’s being starved for compute or memory — kind of getting dumbed down — to keep things running? I keep running into API errors. Does that sound nuts?

Val: People have very subjective opinions on this, but there’s enough of them that I totally believe it, myself included. What's happening is complex. Anthropic has been very varied in their infrastructure. They’ve disclosed TPU-based training and TPU-based inference for a while, Trainium from the original AWS investment, plus a lot of NVIDIA. It’s not simple to understand exactly where you’re getting tokens from.

Having said that, it’s clear — and Dario has admitted — that he underinvested in compute. So clearly they’ve had to quantize quite aggressively. And quantizing isn't just a monolithic on/off switch. You can load some models or stream some weights that are quantized for a certain period at a certain location, and then less quantized at other times and other locations.

Michael: I’ve literally been in the middle of something in Claude Cowork and, from one task to the next, it was as if it woke up like a small child and said, “I don’t know where I am in this task anymore.” And it couldn’t come back from it. I’ve never seen Codex do that.

Val: You’ve heard of the compact feature, right? What’s happening is there are enough KV cache evictions going on that either aggressive compaction or premature compaction is happening, and you’re fundamentally losing context. It comes back to the same fundamental problem: You’re evicting so often. Both OpenAI and Anthropic are pretty explicit in their pricing; you can only buy one hour of cache rights, one hour of KV cache. After that hour, these really long-horizon agents lose context very quickly. That’s one of the things DeepSeek is able to combat, and what we’re looking to combat as well.

38:13 How “mere mortals” should use AI agents today

Michael: How would everyday people think about all this in terms of daily life? I look at my parents and they have no way of relating to this stuff. The young people see it as chat. But somewhere in there, this is fundamentally going to change the world. What do you point people to, beyond the China-versus-America memes?

Val: Just use a consumer-grade agent. Meta has an agent, Google has agents now in regular subscription programs. Everyone’s got agents. Obviously OpenAI has agents, and I think Apple just announced something today.

If you’re a consumer, just use a consumer agent. If you’re a hobbyist, use Hermes Agent. If you’re a developer, Claude Code or Codex.

Michael: Would you install Hermes Agent on a Mac Mini?

Val: You could, but the Mac Mini thing is a bit overdone. I use old laptops and Raspberry Pis. I experiment with all of them. Right now I’m probably going to convert some OpenClaude setups — which is more and more becoming NemoClaude — toward Hermes Agent, because every version update of OpenClaude tends to break things. Hermes Agent claims better memory consistency over time and has better memory harnesses by default. There are also lots of third-party memory systems you can install: Tobi Lütke from Shopify has a famous one, QMD.

Honestly, I don’t have time to tinker as much as OpenClaude encourages you to. Hermes Agent is more straightforward. And for most of my work right now, the guardrailing built into Claude Cowork is really, really good. I haven’t been burned yet.

40:15 The lethal trifecta: Security risks of autonomous agents

Val: We’re all doing something very dangerous if we’re using Claude Code — Simon Wilson’s “lethal trifecta.” We’re giving Claude access to our files. We’re authorizing changes with “always on” skip permission permanently enabled. That’s the second factor: Access to internal data and authorization to act, including delete and update. And the third is access to the internet, because prompt injection is everywhere right now. It’s so easy to embed in images, steganography, hidden fonts, anything on a web page.

So what you need is: You never release output tokens to the user until you have a guardrail as the absolute last thing in the chain.

Michael: Are they doing that well?

Val: I haven’t seen any public incidents of Claude Code or Claude Cowork suffering from the lethal trifecta yet. What I know they’re doing — and this is one of the reasons for rate limits and compute strain — is they’re splitting their token budget into at least three parts: One for actually processing the prompt and agentic multiple turns; one for quality validation loops to make sure you get what you asked for; and one for guardrail loops. They’re not taking shortcuts on the guardrail tokens. They’re running all of them, which costs time and money. That’s where you hit rate limits early and why prices are high. But the end result is higher quality and safe output.

42:48 Predictions for 2030: Recursive self-improvement

Raz: Let’s say we’re doing a second episode in 2030. What should people expect?

Val: I always have a very explicit answer here: We literally can’t imagine what’s going to happen by 2030 because of the exponential progress. Anthropic has pretty much disclosed that we’re at recursive self-improvement, at the edge of the singularity. The unreleased models in the major frontier labs are already there, or close to it. These things are writing their next versions, which means fewer humans in the loop, and the question is how much governance humans can actually exercise.

The iteration cadence of models that can fully recursively self-improve is one that we literally can’t review as humans. We’re seeing this with Claude Code right now: The number of PRs submitted exceeds human review capacity. Cloudflare just declared agent traffic now exceeds internet traffic. We’re basically at a point where we can’t supervise or keep up with the frontier of where these models are, and we’re not even halfway through 2026.

The only analogy I have: If you’d told me when Steve Jobs was on stage with the first iPhone that I’d be regularly getting into a stranger’s car and sleeping at a stranger’s house, I’d have thought you were crazy. That’s where we are right now. The kinds of applications that are coming in three and a half years? We can’t even imagine them.

45:00 Global policy: The need for AI governance

Michael: Is anyone actually policing these companies — including yours — for net-positive returns to humanity? Is the government even capable of that?

Val: Singapore actually has an AI policy now, and they’re even looking at how it’ll affect employment. If you’re relying on an assistant who barely touches this stuff to actually set policy, we’re doomed. You need people in government who deeply understand it.

Singapore has something really interesting called GovTech, which sets the stage for technology within government. There’s a CTO there who still codes, writes papers, and even wrote his own version of an OpenClaude from the ground up in Rust. That’s the kind of people in the government. So Singapore probably has some handle on this. I look at some other governments around the world and I’m less sure.

Last year I was on record saying there should not be regulation, that the industry is innovating fast and we don't want to stifle it. That was 2025. This year I’ve changed my mind. The models have advanced to a level where we absolutely need to start governing them and setting policies.

I was involved with the Linux Foundation back in the early Kubernetes days and delivered an internal talk to the Linux Leadership Forum around 2015 on the need for nonproliferation treaties for AI. If you look at what’s happening in conflicts around the world — Ukraine, Iran — AI is on the battlefront. Autonomous drones are already real, already carrying kinetic weapons, already lethal. We’re there.

48:25 The jobs “apocalypse” vs. shifting tasks

Val: The trend in software developers is revealing. When Claude Code came out in May of last year, a few people said, “Oh my god, this is really good.” By December, everyone realized it was. December was like a peak trough in software development jobs. Since then, I think we’re back — if not exceeding — where we were before Claude Code. People have realized software can be a positive-sum industry. If you can generate more software and more features, you can sell more. There isn’t a jobs apocalypse here. While there are absolutely some industries and regulated organizations where it’s zero-sum, my tagline is: The jobs don’t disappear, the tasks do. Either you figure out how to change the tasks in your job, or if your job is a menial task, it probably will be replaced. You have to move up the stack and adapt.

50:20 Advice for founders and future builders

Michael: You’re here for Super AI in Singapore. What do you tell this audience — startups, founders — to get excited about and focus on?

Val: We were in this age of innocence a few months ago where, if you could have infinite tokens and just a little patience to curate these agents, your imagination was the only limit. The whole Silicon Valley philosophy — “ideas are cheap and execution is priceless” — is inverted right now. Execution is cheap and ideas are priceless.

Think abundance, not just in outcomes but in infrastructure too. You’re not really limited by token costs if you can route your tokens properly across premium, mid-level, and low-level tiers. You effectively still have an unlimited token budget. And with an unlimited token budget, your imagination is the only limit.

Then there are the classic things. Solve your own problems first, because you know that domain best. Whatever domain you come from — medical, food industry, cybersecurity — solve your own problem first. Think Jarvis from Iron Man. Think enterprise-grade. You can do anything you want. This is a very global field right now. For example, there’s a lot of respect for Asian-based startups. Look at Manus, for example.

Michael: Would you tell someone in school not to study computer science?

Val: There was some uncertainty last year when Claude Code first came out. But to date myself — back in the ’80s when I was in high school, I was ahead of my teacher. My teacher was a mainframe guy, and I was on a PC, coding way faster. The exact same thing is happening now. If you’re in a community college or university program and you’ve vibe-coded at all, you’re typically ahead of your professor.

Don’t be afraid of computer science. CS has changed in the last year more than in the last 60, but it’s more alive and thriving than ever.

53:00 Personal insights: Moving to the water and mental health

Raz: You’re super busy: multiple conferences, different continents, keeping up with multiple models. What do you do for your mental or physical health that isn’t AI?

Val: I did something simple. I moved to the water. I lived in California for a long time, but this year I decided to move up to Victoria, British Columbia, which is literally an island surrounded by the Pacific. It’s a beautiful place. I can walk out of my house for 15 minutes and be on the Pacific Ocean, on a beach that’s not crowded, and have it almost all to myself. Connecting that way is my mental health thing. I just love the water.

Michael: Leave us with something you love to watch or listen to.

Val: I cut the cord at home a long time ago, so YouTube is what I watch when I have downtime. The algorithm knows me well and always brings an interesting mix of AI content. I really enjoy Theo, who’s a very famous developer and very prolific on YouTube. He sort of live-streams and then curates content. And the SemiAnalysis crew, Dylan Patel and the whole team. They broke the analyst mold. They didn’t try to be another Gartner. They’ve skilled up across the full stack from deep silicon supply chain all the way up to AI models and agents, and their breadth and depth have always impressed me. Very approachable, very meritocracy-based people.