NeuralMesh Unifies Datasets, Accelerates Genomics Research

Spend less money on storage and more time on genomics research

Genomics pipelines move from sequencer to variant call across petabytes of raw reads. When job queues back up, that path stretches from days into weeks, and the delay hurts scientific breakthroughs. A rare disease diagnosis waits. A cohort gets re-queued behind another re-run. A grant-funded GPU cluster sits idle. Geneticists end up managing infrastructure instead of doing science. That is not how breakthroughs happen.

NeuralMesh removes data bottlenecks. Metadata is shared across every node, so a directory holding billions of files behaves like any other directory. Physical and logical tenant isolation gives each lab its own slice of the cluster, so one re-run does not stall others.

Among WEKA customers, the Oklahoma Medical Research Foundation (OMRF) leverages NeuralMesh to run GATK-based NGS pipelines on GPU-enabled servers. With NeuralMesh, OMRF cut research job times by 10x, with some jobs dropping from 70 days to just a week and others from weeks to hours. And all of these research workloads no longer required staging data to local SSDs before a run.

Our scientists can now more easily complete their data analysis without dealing with data infrastructure. With NeuralMesh, we have so much performance and expandable capacity.

Another WEKA customer, Genomics England (GE) processes 5+ million genomes in a single namespace. With NeuralMesh, GE replaced six file systems with one and cut infrastructure costs by 75%. NeuralMesh also holds the SPECstorage® Solution 2020 GENOMICS record at 5,600 jobs, up from a prior record of 2,200.

We needed something that’s much more scalable than existing NAS solutions — an infrastructure that could grow to hundreds of petabytes. Our existing solution couldn’t provide that scale and wasn’t performing as well in these magnitudes — that’s what drove us to WEKA.