Superior performance, cost-efficiency, and maximum GPU utilization from a modern parallel file system support WeRide’s data pipeline from the edge, to the core, to the cloud.
CAMPBELL, Calif., July 20, 2020 – WekaIO™ (Weka), the innovation leader in high-performance and scalable file storage, announced today that WeRide, a smart mobility company with industry-leading autonomous driving technologies, has selected the Weka File System (WekaFS™), the world’s fastest shared parallel file system from WekaIO, to manage its artificial intelligence (AI) data pipeline from the edge to the core to the cloud. WeRide implemented WekaFS using a hybrid model to manage compute and storage resources both on-premises using commodity Intel x86-based servers and on the Amazon Web Services (AWS) Cloud. WeRide chose WekaFS because it presented a hardware-agnostic solution that was also the most cost-efficient, delivering high-bandwidth I/O to the company’s GPU farm for high performance with mixed workloads.
WeRide is a multi-faceted AI startup that works on advanced research and development (R&D) cycles for Level 4 (L4) autonomous driving vehicles and on partnerships with transportation platform providers that support robotaxi services for commuters. The company processes data at the petabyte (PB) level with a daily mix of large video and image files generated from mapping the operational design domain for the robotaxi service. The images are collected from more than 2 million kilometers of driving distance. WeRide produces millions of high-quality labeling data that is annotated at the core, trained by the AI model on the cloud-based cluster, and fed back to the on-premises AI engine.
WeRide needed a cost-effective solution that would provide high I/O bandwidth to keep the GPU farm saturated with data. It also had to deliver high performance for mixed workloads with significant volumes of metadata, enable a hybrid implementation for the organization to maximize its investment, and be hardware-agnostic to allow for flexible and scalable expansion. WeRide selected Weka as it met every one of these requirements and provided technical support expertise and positive customer references.
“We had built a GPU farm, and we needed a high-speed data pipe to feed it without significantly impacting our bottom line. After an extensive cost analysis and evaluation of several alternative solutions, including open-source and HDFS, Weka stood out as the clear leader,” says Paul Liu, engineering operations lead at WeRide. “Weka was the best choice for our hardware procurement model and was ideal for fulfilling our objective to make our storage a utility for our users—completely hardware-agnostic and transparent to the end-user. Weka’s customer references demonstrated product maturity and the technical support team proved invaluable by getting us launched on AWS.”
Beyond delivering high I/O bandwidth to data-hungry GPUs to keep them fully utilized, WekaFS is the world’s fastest and most scalable file system, perfect for data-intensive applications, whether hosted on-premises or in the public cloud. It is a POSIX file system that scales performance linearly as the GPU server farm grows, so WeRide will not have to compromise performance with future expansions. And since WeRide is running WekaFS on GPU servers in converged mode, creating a single namespace from all the locally attached NVMe drives, they will not have to invest in expensive hardware for their on-premises cluster.
“The Weka software has allowed WeRide to take maximum advantage of its investment in GPUs while achieving the required infrastructure cost efficiencies. In addition, with our file system they get the benefit of workload uniformity and mobility between their on-premises cluster and the public cloud,” said Liran Zvibel, co-founder and chief executive officer at WekaIO. “Weka is solving big problems for customers who require the flexibility of a software-only storage solution that provides all the traditional enterprise features while delivering superior performance at scale. Innovators such as WeRide share my vision to make storage a utility, it’s there and it works, and data available to anyone in the organization who needs it in a predictable time, no matter where they are.”
“Datacenters are evolving, incorporating accelerated computing technologies and cloud strategies to support new workloads such as AI or machine learning. WekaFS is cloud-optimized and architected to provide high bandwidth I/O to GPU-enabled compute clusters playing a big role in enabling digital transformation. We are pleased to have been the solution provider of the Weka software licenses for WeRide to drive their AI workflow,” added Chris Saso, CTO, at Dasher Technologies, a WIN Leader Partner.
Additional resources:
Learn more about the WIN Partner Program: https://www.weka.io/partners/
Â
About WekaIO
Weka offers WekaFS, the modern distributed file system that uniquely empowers organizations to solve the newest, biggest HPC storage problems holding back innovation. Optimized for NVMe and hybrid cloud storage, Weka handles the most demanding HPC storage challenges in the most data-intensive technical computing environments, delivering truly epic performance at any scale. Its modern architecture unlocks the full capabilities of today’s data center, allowing businesses to maximize the value of their high-powered IT investments. Weka helps industry leaders reach breakthrough innovations and solve previously unsolvable problems.
About Dasher
Dasher is more than just an end-to-end IT solution provider with expert engineers in technology infrastructure. We assess, architect, and service IT solutions that digitally transform businesses. We are the trusted technology partner for hundreds of clients. And we provide personal service to deliver positive outcomes. https://www.dasher.com/
WekaIO, WekaFS, Weka AI, Weka Innovation Network, Weka AI logo, WIN logo, and the WekaIO logo are trademarks of WekaIO, Inc.
Media Contact
WEKA Communications
media.relations@weka.io
