Weka was the clear choice for our on-premises DNN training. A NAS solution would not be able to scale to the extent we would need it to, and apart from Weka being the most performant of all the parallel file systems we evaluated, we really liked the fact that it is hardware-independent, allowing us better control over our infrastructure costs. The object storage capability is much more economical than an all-flash system and allows us to efficiently scale our data catalog in a single namespace.
We needed an urgent solution to solve the fast-growing resource constraints. It was astonishing how easy the Weka implementation on AWS was versus trying to build and manage our own Lustre® file system. The improvements we got in terms of performance and costs savings have been great too. Recently we completed one of the most demanding analysis that we have ever made in less than two weeks with Weka. In contrast, with the Lustre system we built, a similar analysis a year ago took two months and we spent three times as much.
Our studio is no longer constrained by the physical location of our data, so we don’t have to have artists in the office working on content – they can be anywhere in the world. Weka with AWS allowed us to start with a small cluster and grow it as our business demands grow.
The integration of the Weka software in our infrastructure increases the efficiency of our datacenter, keeping pace with our application performance requirements while delivering exascale capacity at the best economics. With our current IT infrastructure, we are well-positioned to deliver the best IC products to the market in to the future.
The Aiden Lab cluster required a new solution to improve application performance and facilitate the deployment of a high-performance file system in a cloud computing environment. We required a solution that could support the team and their research related to Genome Architecture and felt that neither GPFS nor Lustre could keep up with our workload.
After comparisons with legacy NFS-based NAS storage solutions, Innoviz selected WekaFS because the performance improvements with WekaFS matched the company’s needs. The storage scalability and ability to grow the infrastructure without losing performance, was a key factor in choosing the Weka file system.
We needed a modern storage solution that could scale to 100s petabytes while maintaining performance scaling, and it had to be simple to manage at that scale. With its clever combination of flash for performance and object store for scale, Weka has proven to be a great solution.
We are using WekaIO technologies over InfiniBand to address the challenges of data analytics at extreme scale in life sciences, particle physics, geosciences, and other fields. That process is still ongoing but to-date we’ve already achieved some promising results.
Future-thinking companies like WekaIO, complement our core principle of accelerating research and discovery. The ability to run more concurrent high performance genomic workloads will significantly advance our time to discovery.