Weka AI: A Transformative Solution Framework for Accelerated DataOps

Weka AI is a solutions ecosystem engineered to solve Accelerated DataOps challenges, delivering Reference Architectures and Software Development Kits with leading AI solutions partners. It provides a production-ready storage solution where the entire data pipeline workflow—ingest data, to batch feature extraction, to hyperparameter optimization, and finally to inferencing and versioning—can be run be run on the same platform, whether running on-prem or in the public cloud. Direct access to data for training and inferencing eliminates data staging at the compute layer and storage silos which results in shorter Epoch and Wall Clock time.

Learn More

REFERENCE ARCHITECTURES AND TECHNICAL BRIEFS

slider bg
tu simpleinnoviz-logo-white-300x91-1

“Weka IO was the clear choice for our DNN training…standard NAS would not scale and Weka was the most performant of all the parallel file systems we evaluated…we really liked that it was hardware-independent allowing us better control over our infrastructure costs.”

Dr. Xiaodi Hou, Co-founder and CTO

“After comparisons with legacy NFS-based NAS storage solutions, Innoviz selected WekaFS because the performance improvements with WekaFS matched the company’s needs. The storage scalability and ability to grow the infrastructure without losing performance, was a key factor in choosing the Weka file system.”

Oren Ben Ibghei, IT Manager

DATAOPS WORKFLOW AND RELATED STORAGE CHALLENGES

Different stages within AI data pipelines have distinct storage requirements for massive ingest bandwidth, need mixed read/write handling and ultra-low latency, often resulting in storage silos, for each stage. This means business and IT leaders must reconsider how they architect their storage stacks and make purchasing decisions for these new workloads.

desktop
mobile

Ingest

Needs massive concurrency, write (WR) throughput

Enrich

Needs labelling, index, search, and cloud bursting

Train

Needs massive read (RD) throughput

Validate

Needs large number of streams replay

Infer

Needs low latency access

Retain

Needs lifecycle, management, versioning, and reproducibility

SOLVING STORAGE CHALLENGES FOR CHIEF DATA OFFICERS, CHIEF ANALYTICS OFFICERS AND DATA SCIENTISTS
watch

Reduces Epoch Times, While Delivering Lowest Inference Times

icon2-1

Explainability and Reproducibility Using Snapshots

icon3-2

Industry’s Best GPUDirect Performance, 80GB/sec to a Single DGX-2 System

icon4

In-Flight and At-Rest Encryption Delivers Data Compliance and Governance

icon5-2

Enables Hybrid Workflows for Testing and Production

SOLVING STORAGE CHALLENGES FOR INFRASTRUCTURE ENGINEERS
icon11

Best TCO Leveraging NVMe for Performance and HDD for Capacity

icon22

Single Storage Platform for Entire Data Pipeline

Best Agility for Data Management Across the Edge, Core and Cloud

icon44-1

Best Scalability, Up to Exabytes of Storage and Billions of Files in a Single Directory

icon55

Best Scalability, Up to Exabytes of Storage and Billions of Files in a Single Directory

Machine Learning/AI Use Case
Use Case

Machine Learning/AI Use Case

Download
Evaluator Group: AI Technology Insight Paper
White Papers

Evaluator Group: AI Technology Insight Paper

Download
Infographic: Reduce time to insights for AI Workflows
Infographic

Infographic: Reduce time to insights for AI Workflows

Download

START SOLVING BIG PROBLEMS

get started