WekaFS Cloud-First Eco-system for Accelerated DataOps

Shailesh Manjrekar. October 26, 2020
WekaFS Cloud-First Eco-system for Accelerated DataOps

IDC predicts File and Object market capacity growing at 40% capacity CAGR from 235 exabytes in 2021 to 833 exabytes in 2024 and market size growing from $22.4B in 2021 to $30B by 2024. Most of the growth is driven by uses cases like AI/ML, machine data, IoT, and life sciences projects. This data deluge cannot be just addressed by All-Flash storage alone as capacity demand vastly outstrips flash supply. There needs to be an innovative approach to how an integrated architecture comprising of flash for performance and hard disks (HDD) and object for capacity tier should be deployed, as a unified global namespace.

WekaFS for Accelerated DataOps provides a global namespace across private and public cloud ecosystem with support for cloud-native storage and data management for edge to core to cloud applications. WekaFS along with leading eco-system partners, who are now certified as part of Weka’s Technology Alliance Program (TAP)  enables DataOps, by providing operational agility, built-in data protection, and security while providing actionable intelligence.

Weka’s Technology Alliance Program (TAP)

WekaFS cloud-first eco-system now supports leading private and public cloud vendors, certified as part of its Technology Alliance Program (TAP). AWS S3, AWS Outposts, Cloudian HyperStore®, Hitachi Content Platform (HCP), IBM Cloud® Object System (IBM COS), Quantum ActiveScale™, and Scality RING are our esteemed partners and this program is growing.

The journey towards hybrid and Cloud-First

WekaFS, working with cloud-native eco-system partners, expedites time-to-market and to-value while delivering cloud-first, data anywhere and security at scale. Specifically provides the following customer benefits –

  • Ease of use with policy-based global file namespace management
    • With multi-protocol access – POSIX, NFS, SMB, S3 via gateway and NVIDIA® GPUDirect ® storage
    • Ability to scale performance and capacity tier independent of each other
    • Performance of NVMe flash and Economics of HDD
  • Deployment flexibility and workload provisioning – hybrid, dedicated, SaaS, DR modes
    • Cloud bursting, data migration between S3 namespaces
    • Data mobility from the edge to core to cloud using Snap-to-object
    • Disaster recovery and test/dev hybrid workflows with 2 copies to two S3 namespaces
    • Data portability across consumption models provide the best TCO
  • End to End security throughout the lifecycle for data anywhere
  • Scalable cost-effective namespace with the best TCO

Global namespace across hybrid cloud  – Best of both worlds

WekaFS can extend its namespace, on the fly, with a single click, over S3 REST enabled private and public cloud ecosystem, thus providing ease of management of a single namespace, with the best economics and data durability. With WekaFS filesystem metadata resides on flash, while seamlessly extending capacity over a private or public cloud namespace. All the I/O’s are serviced by the flash tier while leveraging the object tier for capacity.

Data versioning and provisioning  – important building blocks for Cloud-First strategy

WekaFS enables data versioning using snap-to-object capability. Snap-to-object captures a point in time copy of the entire file and object namespace, with a single click and pushes it to an object store and gets an object storage locator key. Snap-to-object provides data mobility and provisioning, by allowing users to create another file namespace instantly in private and public clouds. Snap-to-object is an important functionality for providing persistence in public clouds such as AWS, as well as providing versioning for “Responsible AI”. WekaFS also ensures security and encryption for these namespace snapshots throughout their lifecycle. A secondary copy can be moved to a remote object namespace, thus providing instant hybrid workflows, disaster recovery, and test/dev capabilities. WekaFS optimizes the S3 namespace by reclaiming space on snapshot deletes, thus optimizing storage efficiency.

Integrated cloud-native exascale Data Protection and DR

As data has become the single most strategic asset for businesses, data value management has become paramount. However, the datasets encountered in these new use cases have grown so big and are so agile that traditional backup and recovery approaches fall short. Businesses are looking for approaches like accelerated DataOps to meet these challenges in the AI era. Add-on and dis-jointed backup and recovery applications siloed namespaces and workflows, fall short when it comes to providing agility and protection to these datasets.

WekaFS snap-to-object knows exactly what the changed files are and captures them along with the namespace metadata to create full and incremental, versioned, immutable backups, and instant recovery. All the snapshots are space and time-efficient. Snap-to-object is done with a single click as shown above and can be recovered using the object storage locator URL as shown below


Multiple models for edge to core to cloud deployment

WekaFS disaggregates performance and capacity tiers and each can be expanded without each other. This disaggregation enables hybrid deployments to leverage private and public clouds, all orchestrated with Kubernetes orchestration.

  • Converged mode
    • WekaFS runs on GPU or CPU server performance tier and private cloud capacity tier.
  • SaaS mode
    • WekaFS runs in a public cloud EC2 performance tier with AWS S3 capacity tier.
  • Hybrid mode supported with AWS Outposts
    • WekaFS run an on-premises performance tier and capacity tier in the public cloud
  • Dedicated mode
    • WekaFS runs on-premises perf tier and capacity tier in a private cloud
  • Edge aggregation Mode
    • WekaFS runs on edge aggregation racks with CPU or GPU servers and private and public cloud
  • DR mode where a second snapshot copy is stored in the public cloud

Distributed computing and data portability across consumption models provide the best TCO

WekaFS provides data portability across consumption models.  The look, feel and performance benefits are the same, whether you run WekaFS at the edge aggregation, on AWS Outposts at the edge data center, in the datacenter with our storage server partners or in the cloud as a SaaS application. WekaFS is available on both AWS Outposts and AWS Cloud and is deployed on the EC2 instances with the ability to provide a global namespace over an AWS S3 bucket. This enables the best TCO across consumption models and across tiers while supporting the latest approaches around Distributed Computing.

To find certified unified storage solutions partner in the Weka Technology Alliance Program, go here or email us at  wtap@weka.io

Weka Technology Alliances Program (TAP)

Learn more

Related Resources

White Papers
White Papers

Guide to Selecting Scalable Storage Solutions

Solution Brief
Solution Brief

Accelerate Genomics Discovery in AWS with NVIDIA GPUs


Analyze Genome Sequence Data on AWS with WekaFS and NVIDIA Clara Parabricks