Cerence Chooses WEKA for HPC Storage | WEKA

overview

WEKA allows Cerence to support the use of both the POSIX file system and object storage in a cost-effective way.

Cerence is building speech language models to improve the in-car experience, specifically using AI to deliver a deep understanding of human behavior, culture, and language. Its mission is to help make automotive transport safer and human interaction with the car more natural and comfortable. The compute infrastructure at Cerence supports multiple mixed workflows: speech language models; those for rapidly growing Natural Language Processing (NLP) and Natural Language Understanding (NLU) efforts; Text to Speech; and Predictive Keyboards — a pattern recognition effort.

“We looked at our legacy architecture and instead of taking an evolutionary step and upgrading every component, we took the revolutionary approach. WEKA cost-effectively enables both the use of POSIX and object storage with performance and latency that is far superior to any other solution.”

Bridget Collins, Chief Information Officer

Challenge

In 2019, Cerence spun off and became a separate entity from parent company Nuance Communications, a leading provider of conversational AI and user of the high-performance file system GPFS on massive storage arrays. Cerence determined that using its legacy file system would result in cost-prohibitive economics at scale and began to explore alternatives. The end-users at Cerence relied on a POSIX file system for their research workflows, but they had the added constraint of having to cost-effectively manage the mountains of data that would be collected. Therefore, using an expensive, monolithic infrastructure to provide the POSIX file server was not desirable as the cost would be prohibitive at scale.

THE NEW SOLUTION HAD TO MEET SEVERAL KEY CRITERIA:

Support use of both the POSIX file system and object storage in a cost-effective way

Enable a hybrid implementation and tier data seamlessly from on-premises to the public cloud

Use modern storage technologies such as NVMe

Exscale Capacity

Allow for modular growth and scale with the growing needs of the business

SOLUTION

THE WEKA FILE SYSTEM SOFTWARE ON HPE SERVERS

WEKA has a two-tier architecture that takes NVMe flash and disk-based technologies and presents them as a single hybrid storage solution. The Cerence IT team decided to implement WEKA in a converged mode with WEKA running on GPU servers, creating a single namespace from all the locally attached NVMe drives. The team is managing 900TB of data on the WEKA file system to support the NLP and NLU workloads on a cluster consisting of 40 HPE Proliant DL360 servers, each with dual 25GbE networking adapters. The servers are interconnected with 4 switches for high availability (HA), performance, and redundancy. Each server has one network connection and two NVMe drives dedicated to WEKA and one GPU card. Cerence IT is managing 3.2PB of data on object storage with SUSE Enterprise Storage, with 900TB assigned to WEKA, running on a cluster consisting of 9 HPE Apollo 4200 servers, each with twenty-four 14TB drives. The team also utilizes an HPE Apollo 6500 server with 8 GPUs for multi-GPU processing.

Benefits and ROI

Cerence was able to realize several benefits and tremendous return on investment by choosing WEKA:

No limit on capacity scaling

Improvement in performance

Unified Access Posix COMPLIANT with others

AUTOMATED TIERING

Improved Resource Utilization

Integration with public cloud for compute elasticity

Reference Architectures

HPE DL360 Reference Architecture

Download

Technology Brief, White Papers

HPE: Accelerate Performance for Production AI Technical White Paper

Download

Start Solving the Big Problems

Schedule a Free Trial