Building AI Factories: Storage Architecture Defines Success

When I walked into SC25 in St. Louis this November, the energy felt different from past years. The conversations weren’t about supercomputers anymore—they were about factories. AI factories.

NVIDIA’s Jensen Huang has been pushing this concept hard over the past year, and it’s finally clicking with infrastructure leaders: data centers aren’t just housing compute anymore, they’re manufacturing intelligence. You apply energy and data as raw materials, and what comes out are tokens—the building blocks of AI.

The analogy makes perfect sense when you think about it. In the last industrial revolution, factories took in water and produced electricity. Today’s AI factories take in data and electricity, and manufacture something incredibly valuable: intelligence tokens that power everything from scientific discovery to autonomous systems.

But here’s what I kept hearing in every conversation at SC25: the factory metaphor only works if your data pipeline can keep up with production demands.

The Exponential Data Challenge Nobody’s Prepared For: 394 Zettabytes by 2028

Let me share some numbers around what we’re dealing with.

By 2025, the global datasphere is projected to reach 181 zettabytes, driven by AI-driven content, IoT devices generating over 73 zettabytes alone, and cloud computing infrastructure. By 2028, we’re looking at 394 zettabytes per year—and the “yottabyte era” is projected for the early 2030s. 

That’s not a typo. We’re talking about data growth that will double approximately every three years at current rates. 

But here’s what makes AI workloads fundamentally different: they don’t just consume data—they amplify it exponentially. At SC, I heard one estimate that a single 100-megawatt AI factory running on Hopper architecture could produce 300 million tokens per second.To break that down a bit more, each of those training runs generates:

• Terabyte-to-petabyte scale checkpoints that need rapid write and read access
• Billions of tokens stored for fine-tuning and inference
• Massive embedding stores for RAG and agentic systems
• Training artifacts retained for governance and model iteration

And that’s just for one model lifecycle. Multiply that across continuous training, fine-tuning, and inference at production scale, and you quickly understand why by 2025, 463 exabytes of data will be created daily around the world.

Why Traditional Storage Breaks the AI Factory Model

I spent most of SC25 at WEKA’s booth, and the pattern in customer conversations was remarkably consistent. Organizations had made massive GPU investments—often tens or hundreds of millions of dollars. They’d hired world-class AI talent. They had the models.

But their AI factories couldn’t scale because their storage architecture forced an impossible choice: extreme performance or manageable economics. Never both.

Here’s the fundamental problem: AI workloads need both high-performance flash for hot data access and high-capacity storage for the massive data footprint—simultaneously, in the same namespace, without tiering penalties.

Traditional storage forces you to choose:

• All high-performance TLC NVMe: Delivers the microsecond latencies and sustained throughput GPUs demand, but the capacity costs become prohibitive at AI scale
• All high-capacity QLC NVMe: Provides the density needed for exabyte-scale data, but introduces write performance penalties that leave GPUs waiting

Let me explain why this trade-off exists.

The Flash Trade-Off: Understanding TLC vs. QLC for AI Workloads

Here’s the challenge that came up in nearly every technical conversation at SC25: TLC and QLC flash technologies each excel at different things, and traditional storage architectures force you to choose between them.

High-performance TLC flash delivers the speed AI workloads demand—sub-millisecond latencies, sustained write throughput for massive checkpoint operations, and approximately 3,000 program/erase cycles. It’s built for performance but comes with higher cost per terabyte.

High-capacity QLC flash provides the density AI factories need—dramatically lower cost per terabyte and the ability to scale to 61TB+ per drive. But it introduces performance trade-offs, particularly for write operations, and handles around 1,000 program/erase cycles.

The critical difference comes down to how these drives manage data internally through something called “indirection units”—the minimum block size the drive’s internal controller uses for mapping and managing data.

High-performance drives use smaller indirection units (typically 4-8KB), which means they can handle small, random writes efficiently without significant overhead. When your AI workload writes a 4KB metadata update, the drive processes it directly without extra translation work.

High-capacity drives use larger indirection units (often 16KB, 64KB, or larger) to manage their massive address spaces efficiently. This reduces the amount of DRAM needed in the drive controller—critical when you’re managing 60TB+ of capacity. But when your workload writes data smaller than the indirection unit, the drive must perform read-modify-write operations internally, creating write amplification that degrades performance and accelerates wear.

This is why AI factories hit a wall with traditional storage:

• Training and checkpointing generate massive sequential writes that need TLC-level performance
• Metadata operations create countless small random writes that suffer on large-indirection-unit drives
• Embeddings and token histories require petabytes of capacity that only high-density drives provide economically

Traditional tiered storage tries to solve this by moving data between performance and capacity tiers based on access patterns. But that creates its own problems: data movement overhead, cache coherency complexity, and write performance penalties exactly when GPUs need maximum throughput. Every minute spent analyzing access patterns and shuffling data between tiers is time your GPUs are waiting.

How WEKA AlloyFlash Eliminates the TLC vs QLC Trade-off 

At SC25, we announced a breakthrough that fundamentally changes this equation: WEKA AlloyFlash.

AlloyFlash intelligently combines TLC and eTLC flash drives in the same system—up to 40 drives in a 2U configuration—delivering consistent performance across all operations without cache hierarchies, data movement between tiers, or write performance penalties.

Here’s what makes this different from traditional tiered storage:

1. Unified namespace, intelligent placement
Unlike legacy tiered systems that force data to move between performance and capacity tiers, AlloyFlash presents everything as a single ultra-fast namespace. NeuralMesh automatically places data based on workload characteristics—hot data on high-performance flash, warm data on high-capacity flash—without introducing I/O penalties or requiring manual management.

2. Full write performance without throttling
AlloyFlash delivers full write performance across all operations, eliminating the write throttling that plagues traditional capacity-optimized systems when SLC cache exhausts. This is critical for AI workloads where massive checkpoint writes can’t afford performance degradation.

3. Breakthrough economics at scale
The WEKApod Prime appliance with AlloyFlash achieves 65% better price-performance, with 4.6x better capacity density, 5x better write IOPS per rack unit, 4x better power density at 23,000 IOPS per kW, and 68% less power consumption per terabyte.

This isn’t about choosing between performance and cost anymore. It’s about eliminating that false trade-off entirely.

Building Production AI Factories: Infrastructure Requirements for Token Generation at Scale

I had a conversation at SC25 with an AI cloud provider managing thousands of GPUs across multiple customer deployments. Their challenge: serving diverse customer needs with SLAs that range from ultra-low-latency training to cost-optimized inference to high-throughput data preprocessing.

With traditional storage, they’d need separate infrastructure for each workload profile. With AlloyFlash on NeuralMesh, they get the flexibility to optimize each workload on a unified platform—maximizing infrastructure utilization while maintaining guaranteed performance SLAs.

This is the pattern I kept seeing: organizations need storage that adapts to AI workload diversity, not storage that forces them into architectural compromises.

Investors are now measuring AI infrastructure ROI by tokens generated per watt or per dollar, treating capex models like manufacturing cost accounting rather than traditional IT budgets. That’s exactly the shift Jensen Huang talks about—infrastructure as a production system, not just a cost center.

The Three Ingredients of a Scalable AI Factory

Based on what I saw at SC25 and countless customer deployments over the past year, successful AI factories need three foundational elements:

1. Compute fabric optimized for AI workloads
GPUs are the obvious starting point—but the entire compute stack needs to be purpose-built for AI. Modern AI factories require both enormous FLOPS for throughput and enormous bandwidth plus memory for response time—an incredibly difficult problem to solve. Organizations are deploying liquid-cooled, high-density racks as the new baseline, with NVIDIA’s Blackwell architecture delivering 10× throughput at 10× lower cost per token versus Hopper.

2. Storage architecture that eliminates GPU idle time
GPUs that sit idle waiting for data represent wasted money, stalled innovation, and runaway cost per token. This is where NeuralMesh makes the difference—microsecond latency at any scale, unified namespace from core to edge, and intelligent data placement that keeps GPUs saturated.

Our recently announced integration with NVIDIA BlueField-4 takes this further, eliminating the need for standalone CPU servers by leveraging 800 Gb/s networking bandwidth and 6x compute improvement directly on the DPU.

3. Unified data layer from ingestion to inference
AI factories generate data everywhere—edge devices, preprocessing clusters, training infrastructure, inference endpoints. Traditional data centers fragment this across multiple namespaces with duplication and complexity. A unified storage layer with global consistency becomes critical for managing data at production scale.

Looking Ahead: Storage Defines AI Factory Economics

Walking away from SC25, one thing is abundantly clear: the next wave of AI innovation won’t be limited by model architectures or GPU availability—it’ll be limited by storage systems that can’t keep pace with production demands.

The companies that learn to run their own AI factories—converting data into differentiated intelligence—will define the next era.

But “running an AI factory” means rethinking infrastructure from first principles:

• Data isn’t just an input—it’s the raw material that defines production capacity
• Storage isn’t just capacity—it’s the conveyor belt that determines throughput
• Performance isn’t measured in IOPS—it’s measured in tokens per dollar and GPU utilization

Organizations are already managing AI data at exabyte scale, with one exabyte equivalent to 7.8 million iPhones with 128 GB storage each. As we move toward 394 zettabytes per year by 2028 and the yottabyte era in the early 2030s, the storage systems we build today will determine which organizations can scale their AI factories and which will hit infrastructure limits.

That’s why technologies like AlloyFlash matter. Not because they’re incrementally better than what came before—but because they eliminate the fundamental trade-offs that make AI factories uneconomical at scale.

If you’re building AI infrastructure right now, ask yourself: Is your storage architecture a multiplier for your AI investments, or is it the bottleneck that limits everything downstream?

The AI factory era is here. The question isn’t whether you’ll build one—it’s whether your infrastructure can scale to meet production demands.

Want to dive deeper into how NeuralMesh can transform your AI factory? Check out this Solution Brief or contact us—I’m always happy to talk storage architecture and AI factory design.