WekaIO and Supermicro Expedite AI Workflows and Time-to-Market
Ross Kemp. June 22, 2020
Ross Kemp, Director of Partner Sales at WekaIO (Weka), is responsible for the Supermicro relationship, and in this blog, discusses how WekaIO (Weka) and Supermicro accelerate AI processes for faster time-to-market in this blog titled “WekaIO and Supermicro Expedite AI Workflows and Time-to-Market”.
WekaIO (Weka) and Supermicro expedite AI workflows and time-to-market for companies using software running on GPU-enabled servers to process large amounts of data. These companies include those leading the charge in autonomous vehicles, financial services, and genomics research.
To stay competitive and to expedite AI workflows and time-to-market, companies need high-bandwidth I/O to keep GPU-enabled servers saturated with data while staying within IT budget constraints. The performance needs of modern data analytics demand a complete departure from legacy file structures and hard-disk-based architectures. A single high-performance GPU server can present I/O demands exceeding 80GB/sec of data transfer.
Predictable and seamless performance scaling is unattainable with traditional NAS filers because of file system protocol limitations, resulting in data starvation and poor utilization of expensive GPU server resources. Our file system, WekaFS, is built for the most extreme demands of data-intensive technical computing environments and delivers the maximum performance at any scale.
Weka used a clean-sheet design for WekaFS to leverage the performance of NVMe flash technology and fast Ethernet or InfiniBand networking to ensure the highest performance and lowest latency for the most demanding, unpredictable AI workloads. WekaFS is the world’s fastest file system and is uniquely able to meet the performance needs of AI workloads, which have highly randomized access patterns to both small and large files. WekaFS is superior to legacy NFS-based Network Attached Storage (NAS) solutions and all-flash arrays (AFAs) and can be deployed natively on-premises or in a hybrid cloud. Together, Weka and Supermicro have set industry performance records on SPEC® SFS2014 and the IO-500, the authoritative benchmark of Supercomputing.
Weka and Supermicro, a WIN Innovator partner, have collaborated closely to offer pre-configured storage appliances that deliver performance, flexibility, and value. These storage appliances with WekaFS running on Supermicro servers deliver the best economics and unmatched storage performance at scale, with optimization for maximum AI acceleration and reduced training times. AI workloads consist of billions of small files, so the storage system must be able to deliver high throughput, low latency, and excellent metadata performance at extreme scale to keep GPUs and applications saturated with data. This is unattainable with legacy storage systems but made possible with the solution from WekaFS and Supermicro.
Any company that has invested in expensive GPU resources and has AI workflows similar to those described above can benefit from the ability of Weka and Supermicro to expedite AI workflows and time-to-market.
If you have any questions or would like more information, please feel free to contact me directly at firstname.lastname@example.org.
To see a recorded webinar on the storage solution from WekaIO and Supermicro, go to: “Faster Results and a Quicker Time to Market with Weka and Supermicro“.
For more information or to locate a partner in the Weka Innovation Network, go to: https://www.weka.io/partners.
See the WekaFS on Supermicro Storage Appliances Data Sheet for more configuration details.