# NeuralMesh Unifies Datasets, Accelerates Genomics Research

NeuralMesh™ brings all sequencing data into a single namespace to remove storage silos, feeds servers in parallel to maximize GPU and CPU utilization, and reduces cost per computation for genomics workloads.

**Type:** Solution Brief

## Challenges

- Billions of files per directory delay concurrent genomics research jobs
- Sequencing data scatters across disparate data repositories
- Idle GPUs raise research costs
- Retention mandates drive endless capacity growth
- Consortium sites wait on data copies

## Solution

NeuralMesh™ distributes every file and its metadata across all nodes, so one read pulls from many drives at once. Clients move I/O directly to GPU and CPU servers, with no staging to local SSD. Distributed metadata handles billions of files in one directory and trillions cluster-wide. S3 and POSIX run on the same physical data blocks, not a gateway, so every pipeline stage shares one copy. Data reduction extends usable capacity without slowing writes. Tiering spans flash and object storage with no manual tuning.

## Benefits

- Run more concurrent research jobs
- Consolidate disparate file systems into a single namespace
- Drive GPU utilization past 90%
- Stretch usable capacity up to 6x with guaranteed data reduction
- Federate data across hybrid cloud with on-demand hydration

## Spend less money on storage and more time on genomics research

Genomics pipelines move from sequencer to variant call across petabytes of raw reads. When job queues back up, that path stretches from days into weeks, and the delay hurts scientific breakthroughs. A rare disease diagnosis waits. A cohort gets re-queued behind another re-run. A grant-funded GPU cluster sits idle. **Geneticists end up managing infrastructure instead of doing science**. That is not how breakthroughs happen.

NeuralMesh removes data bottlenecks. Metadata is shared across every node, so a directory holding billions of files behaves like any other directory. Physical and logical tenant isolation gives each lab its own slice of the cluster, so one re-run does not stall others.

Among WEKA customers, the **[Oklahoma Medical Research Foundation](/customers/omrf/) (OMRF)** leverages NeuralMesh to run GATK-based NGS pipelines on GPU-enabled servers. With NeuralMesh, OMRF **cut research job times by 10x**, with some jobs **dropping from 70 days to just a week** and others from weeks to hours. And all of these research workloads no longer required staging data to local SSDs before a run.

```json
{
  "_key": "7f6db2983b67",
  "_type": "customerQuoteBlock",
  "markDefs": null,
  "quote": {
    "anonymizedAttribution": null,
    "anonymous": false,
    "authorName": "Stuart Glenn",
    "companyLogo": {
      "_type": "image",
      "asset": {
        "_id": "image-28cecab21ac8abebd0749524470764c1bd660ff8-150x99-svg",
        "metadata": {
          "dimensions": {
            "_type": "sanity.imageDimensions",
            "aspectRatio": 1.5151515151515151,
            "height": 99,
            "width": 150
          },
          "lqip": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsTAAALEwEAmpwYAAACmElEQVR4nJ1SS0iVURCeXkRPe1h6/3/mO2eOtxtlRSVSEYUFFoFRqYhgTygrrLByE1kuysKyB5mFQU8x7b/XClpkm9rUrnX7CEnv1Yyg9Y2TBq3vwHDmMGc+5nzfR/R/vPs+i/ozpZTK7KaX6QR1fZ5GOceH7FRKpZdQcqSSUsPbKfqu1DU4k7LZSbkB9vycT8lMGUWjtZTK7Pxbd39jirJTcgOMBkGpkSrqHa6kZLqCetMV9HQQuW8YDccpGj06uftr4/S29/V5Dbf35O/Yl0jEYvlBECxk5hnxBfG5IhJYa40GQcIYY33fn8aYZTawSyfAslPo1dhqikaap93+dC2/quEqlpdcMMYdUqDKit0lBVKMGDYq47AROWaBJhui3rHZYxknnUijETk4LkbPYD71Z7ZSNHRm9tkHzbxq3U3HuKLALWXcsoJ2G8gBFTmnLM8c46ZlvmyBFv/OMe4Z5kMmDNf7rxZSNLThL2/Pv5YvqD6xH4kVrSrmtIrcUcZjBV4ocN+JpFQw4AFUpM0yWh3jhgoeeUDHvJIold48rmp6Bz39koiVbNqmuuS8E3Pcb2gFnQr0KTCggrdOzBsPoIJeK3homVuU8cQBVxWoJooydZQcW0OvMwG1fZwDuI2OcdEympyYu1ak3YrpUOCxFelwIt1O0DnR6/A0eAqUuc4LRRSlayn6HU5oPckEZrWKOT/BzQ0/YEQaFdirQJ0Tua5iLrkQp5T5rGU0+7sylxLRZKJoqJyiX4v+uUcXa4ECFTbEkSKRGiuyzTJvFpEiESm2IjUaotqEYZkJzRbHplYZdQCcX4ioL72Woh95/zlyqvddUVAk8cLCRQDmO+fy4vH49FgsNlNVC1yBW2yMmecTQMyn96mf/gMae8wtFMlPjgAAAABJRU5ErkJggg=="
        },
        "url": "https://cdn.sanity.io/images/ult5g8gw/production/28cecab21ac8abebd0749524470764c1bd660ff8-150x99.svg"
      },
      "crop": null,
      "hotspot": null
    },
    "companyLogoMono": {
      "_type": "image",
      "asset": {
        "_id": "image-052721e5c5767b4b111782b10b171797212d33ae-150x99-svg",
        "metadata": {
          "dimensions": {
            "_type": "sanity.imageDimensions",
            "aspectRatio": 1.5151515151515151,
            "height": 99,
            "width": 150
          },
          "lqip": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsTAAALEwEAmpwYAAABeUlEQVR4nKXT3WsPcBQG8I8hmby1GWbzPlpSUkpTxmqaIoykKFshLibGzaK5wCRva/vhyktSUu5c8OfprOdXc/vbqdO379vznPOcc/jf1uAIzmMfVlqCrUAfLmEEu9COZa0CbsQQruEcTqAHy1sF3J7oys/iTM7aWgXci9uYzDoaHTehA6uxDt3Ykbud6Mzaj/1NsErrEB7jNV5hGuO4GAkO4BhuhPAhbuECJnAPY81idEa/yQDNYgbv4kVyHVP4hrd4hid59yHkRwtwCwai2XBY6tF9zOELfuIjfuFvAF7iOd7gcwAPFuBgUjodXU7hEe4kuvcBLKA/+B2AH/iUKL/iRQq60CaHsQ1ro9N0NGok3UYincf3kDSyn4oEVxOQKwErqyau4lSElXalUx8q/SKuT1Wwp7iLBylk7WvCFlqsdKvWaNrm6HkTlyPBcexJpeusUqvGP5mAimh3c6oq3fWLAKvq1Xe9IaoJqvtVGcUi7MKG+NZ49al/Xnk/6ZQ/IIgAAAAASUVORK5CYII="
        },
        "url": "https://cdn.sanity.io/images/ult5g8gw/production/052721e5c5767b4b111782b10b171797212d33ae-150x99.svg"
      },
      "crop": null,
      "hotspot": null
    },
    "companyName": "OMRF",
    "jobTitle": "Assistant Director of Infrastructure & Research Computing",
    "text": "Our scientists can now more easily complete their data analysis without dealing with data infrastructure. With NeuralMesh, we have so much performance and expandable capacity."
  }
}
```

Another WEKA customer, [**Genomics England**](/customers/genomics-england/) (GE) processes 5+ million genomes in a single namespace. With NeuralMesh, GE **replaced six file systems with one** and **cut infrastructure costs by 75%. **NeuralMesh also holds the [SPECstorage® Solution 2020](https://www.spec.org/storage2020/) GENOMICS record at 5,600 jobs, up from a prior record of 2,200.

```json
{
  "_key": "q00040",
  "_type": "customerQuoteBlock",
  "markDefs": null,
  "quote": {
    "anonymizedAttribution": null,
    "anonymous": false,
    "authorName": "David Ardley",
    "companyLogo": {
      "_type": "image",
      "asset": {
        "_id": "image-9384d0795da03f467e1ba2c5a9c7bb7dbfe7219f-300x152-svg",
        "metadata": {
          "dimensions": {
            "_type": "sanity.imageDimensions",
            "aspectRatio": 1.9736842105263157,
            "height": 152,
            "width": 300
          },
          "lqip": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsTAAALEwEAmpwYAAABwklEQVR4nJ1SW2sVQQxerQqKl1N9KNXVU8+Z3cwl2XXdPZPMntq1aksfWrwhKELRJ2nBgvbBvy/Zgoj4UB0ICZOQL8n3Zdn/vYuztr3lHBfGtPP19fbavza48JtlbdteLqibWeQdi4v0EHltGIZLYyUir1m7uAPQ31BTZGPizWldTzRWP53WE6LN1Qe0uao13vvrQEI2yKEN/N5VzHU9TMaGEOSgrGQPQurLkBaWokDo+sLFJ2WQpU5Q+Dae/afeuNRMod7wdf/IBj72KF+hkoOilnt6iswGfmMpfRg9LnccyYuS4kuL8tqGtF9Seg6Bn+p6misobc2gJcBFByhfXOCjEuOrIjQuz+VqFkJ333sxJvRzQ0OuSLbq0KAMv6Zu+qQTajzDRVciV1DHbUvyDXRC5M+66dyLGRn70/SmzqWmcMyO0pat465FGcpalgqiuZEQ4h8e0ylgOrXI33WTv1Lqvb/ivdxW9gB4QzcgirmhmKtUQmDnq/TMYTqxlI6hSidnDVN/Drm8W8myURIrCmJJHkOI21D1b13ojwDlo63kEIg/Gdc15xaiag+oAyWv9LJnsBtsiLuqhKJiVukANHd/AramZ9wwTkK9AAAAAElFTkSuQmCC"
        },
        "url": "https://cdn.sanity.io/images/ult5g8gw/production/9384d0795da03f467e1ba2c5a9c7bb7dbfe7219f-300x152.svg"
      },
      "crop": null,
      "hotspot": null
    },
    "companyLogoMono": {
      "_type": "image",
      "asset": {
        "_id": "image-5046395859d911420d6b9c813ec2615668d81cc2-300x152-svg",
        "metadata": {
          "dimensions": {
            "_type": "sanity.imageDimensions",
            "aspectRatio": 1.9736842105263157,
            "height": 152,
            "width": 300
          },
          "lqip": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsTAAALEwEAmpwYAAABD0lEQVR4nJ3S2SuFYRAG8J+1yHI4F8pSsiTFCTmO42RfckG2G1LiShSFC/++5jTqS8oy9fS+7/dNM8/zzPC/aEU/pjGJ7r8WaCkgogMT2EcdQ2j/TI5HGb2J6NyHUt5LiYFE5PRgHje4RC1zmnGMQ6yhitW8r6ORDFYK3xcxjgU84ilrjKQVznGVZ0jYwwnOcIRdbBX+bSS7ZdzjAaeYRVcUHMNUmjuaneawWWBdT4bVLFTBNl6S4V2yjDpNml9RTmm1ZHSQDRoF2cH4HW+J11TybXRiMAcWfkXnYB+IVQl5O3hOH+OMgtHsx3Vpy5WIM5ospdyL9O86p32bzH8dsXszObzYirAgrAgbwprA8Aexqx2t5CAGMQAAAABJRU5ErkJggg=="
        },
        "url": "https://cdn.sanity.io/images/ult5g8gw/production/5046395859d911420d6b9c813ec2615668d81cc2-300x152.svg"
      },
      "crop": null,
      "hotspot": null
    },
    "companyName": "Genomics England",
    "jobTitle": null,
    "text": "We needed something that’s much more scalable than existing NAS solutions — an infrastructure that could grow to hundreds of petabytes. Our existing solution couldn’t provide that scale and wasn’t performing as well in these magnitudes — that’s what drove us to WEKA."
  }
}
```

[Download PDF](/api/resource-pdf?slug=neuralmesh-accelerates-genomics-research)