Will the Real Performance Leader Please Stand Up?
David Hiatt. November 10, 2018
Many of us in the storage industry have been around long enough to remember the TV show To Tell the Truth. Three individuals each professed to be a person with a specific claim to fame; contestants asked questions of each participant to identify the person that was telling the truth. IT organizations often play that same game when it comes to vendor claims. Each vendor says they are more scalable or have better performance. The customer’s challenge in this case is to ask questions to determine the conditions under which that performance can be achieved. Typically, these conditions do not relate to those found in the only world that matters – the data center.
A couple of interesting examples are IBM’s claim to deliver throughput of 23GB/second for a single server node. Not to be outdone, DDN claimed earlier this year that its ES14KX all flash array achieves 33GB/second to Nvidia’s DGX1 system. The devil, of course, is in the details. WekaIO also makes a very bold claim, that Matrix is the World’s Fastest File System. Hmm, what’s a customer to do?
It’s important to see if the numbers are directly comparable, so let’s evaluate the claims. IBM’s 23 GB/sec is write performance for a single server node, a very respectable number. The performance is achieved using a block size of 128GiB and 8 clients. To Big Blue’s credit, they give results that at least show some of the specifics behind the numbers. DDN’s performance of 33 GB/sec is harder to evaluate because there’s no point of reference, which is a red flag to look a bit closer. Is it read or write performance? What is the file size? Was it peak or sustained bandwidth?
One must conclude that this is an aggregate bandwidth figure because the ES14KX is a separate hardware device feeding the DGX1. Consider that the theoretical maximum bandwidth per link to the device is 12.5GB/sec using a 100Gb interconnect. To achieve 33GB/sec would require three 100Gb links at a minimum, which would be unbalanced across the aging dual controller design. So, figure that four 100Gb links were used to get this bandwidth. This comes on specialized hardware and uses DDN’s Lustre file system. While Lustre is good at certain things, small files and metadata operations are not ones that come to mind.
WekaIO has achieved over 11GB/sec per 100Gb link in a customer production environment, not using a benchmark under sterile conditions. Using four 100Gb links provides 44 GB/sec of bandwidth. Need more bandwidth, add more links. Performance scales linearly with the number of links.
Equally important as understanding the conditions under which benchmark figures are achieved is understanding how much the test system costs to get that performance. Proprietary hardware using an outdated design is hardly a way to save money or position your infrastructure for tomorrow’s workloads. Matrix is a parallel file system architected for NVMe, not hard disk drives, like Lustre. Matrix runs on standard x86 hardware and standard NVMe SSDs, not proprietary hardware. In the equivalent rack space (4U), Matrix provides 45GB/sec and 3.5M IOPS and is based on your choice of hardware. Because Matrix is architected for NVMe, it provides much higher small file, random access and metadata performance than other solutions. In addition, Matrix supports snapshots, integrated tiering, cloud bursting and remote backup and DR.
So, will the real Price and Performance Leader please stand up?