Artificial Intelligence, Machine and Deep Learning

High performance, low latency storage for I/O-intensive workloads like AI and machine learning.

A Transformative Solution Framework for Accelerated DataOps

WEKA is a solutions ecosystem engineered to solve Accelerated DataOps challenges, delivering Reference Architectures and Software Development Kits with leading AI solutions partners. It provides a production-ready storage solution where the entire data pipeline workflow—ingest data, to batch feature extraction, to hyperparameter optimization, and finally to inferencing and versioning—can be run on the same platform, whether running on-prem or in the public cloud. Direct access to data for training and inferencing eliminates data staging at the compute layer and storage silos which results in shorter Epoch and Wall Clock time.

Reference Architectures And Technical Briefs

tu simpleinnoviz-logo-white-300x91-1Cerence LogoWeRide-1536x593 (1)

“Weka IO was the clear choice for our DNN training…standard NAS would not scale and Weka was the most performant of all the parallel file systems we evaluated…we really liked that it was hardware-independent allowing us better control over our infrastructure costs.”

Dr. Xiaodi Hou, Co-founder and CTO

“After comparisons with legacy NFS-based NAS storage solutions, Innoviz selected WekaFS because the performance improvements with WekaFS matched the company’s needs. The storage scalability and ability to grow the infrastructure without losing performance, was a key factor in choosing the Weka file system.”

Oren Ben Ibghei, IT Manager

“We looked at our legacy architecture and instead of taking an evolutionary step and upgrading every component, we took the revolutionary approach. Weka cost-effectively enables both the use of POSIX and object storage with performance and latency that is far superior to any other solution.”

Bridget Collins, Chief Information Officer

“We built a GPU farm, and we needed a high-speed data pipe to feed it. We evaluated open source solutions, HDFS, and the public cloud. We chose Weka for its ability to provide cost-effective, high-bandwidth I/O to our GPUs, product maturity, customer references, and stellar on-demand support.”

Paul Liu, Engineering Operations Lead

DataOps Workflow And Related Storage Challenges

Different stages within AI data pipelines have distinct storage requirements for massive ingest bandwidth, need mixed read/write handling and ultra-low latency, often resulting in storage silos, for each stage. This means business and IT leaders must reconsider how they architect their storage stacks and make purchasing decisions for these new workloads.

desktop
mobile
Ingest

Needs massive concurrency, write (WR) throughput

Enrich

Needs labelling, index, search, and cloud bursting

Train

Needs massive read (RD) throughput

Validate

Needs large number of streams replay

Infer

Needs low latency access

Retain

Needs lifecycle, management, versioning, and reproducibility

Solving Storage Challenges For Chief Data Officers, Chief Analytics Officers And Data Scientists

watch

Reduces Epoch Times, While Delivering Lowest Inference Times

icon2-1

Explainability and Reproducibility Using Snapshots

icon3-2

Industry’s Best GPUDirect Performance, 113GB/sec for a single DGX-2 and 162GB/sec for a single DGX-A100

icon4

In-Flight and At-Rest Encryption Delivers Data Compliance and Governance

icon5-2

Enables Hybrid Workflows for Testing and Production

Converged Storage

More economical storage solution when running converged on the storage servers

Solving Storage Challenges For Infrastructure Engineers

icon11

Best TCO Leveraging NVMe for Performance and HDD for Capacity

Global Namespace

Single Storage Platform for Entire Data Pipeline

Best Agility for Data Management Across the Edge, Core and Cloud

Scalability

Best Scalability, Up to Exabytes of Storage and Billions of Files in a Single Directory

icon55

Simple to Setup and Manage From an Intuitive GUI, or Integrate with Other Third Party Monitoring Tools

AI Datasheet - faster deep learning for artificial intelligence and analytics
Datasheet

AI Datasheet - faster deep learning for artificial intelligence and analytics

Download
Cerence Case Study
Case Studies

Cerence Case Study

Download
10 Things to Know When Starting with AI
White Papers

10 Things to Know When Starting with AI

Download

Start Accelerating AI Data Pipelines

Schedule a Free Trial