How to Develop a Best-of-Breed HPC Storage Solution to Deliver Record Breaking Performance
Blog published with permission from Micron.
WekaIO is a leader in high performance, scalable file storage for data intensive applications and was just named a cool vendor by Gartner. We also caught the attention of The Register and others in the tech space when we recently achieved #1 position on SPEC SFS® 2014 benchmark results, delivering over 2x the performance of the previous highest submission. As a software-only company, WekaIO prides itself in being “hardware independent”, a true software defined storage (SDS) solution. Of course, to participate in the benchmark test, we needed to have a hardware infrastructure to prove our parallel file system could post the best performance results. We also had a goal to prove that SDS partnered with the best of breed industry standard hardware could deliver a performance system that significantly outpaced dedicated storage appliances. So how did we go about it?
Comparing SAS/SATA to NVMe
As most of you know, SSDs are currently available in three protocol types, SAS, SATA and NVMe. And while all three SSD types deliver significantly better performance than spinning disk, we chose NVMe for its superior performance. Both SAS and SATA require more interaction with the host operating system which impacts the performance from these devices. Figure 1 shows that an NVMe SSD delivers over 3.5x the performance of SAS and over 7.5x the performance of SATA. (It is important to mention that the lowest performing SSD numbers are over 180x faster when compared to the highest performance traditional spinning disk drives.)
WekaIO Matrix software supports all SSD protocols, but having just launched our native NVMe-over-fabrics driver we were keen to demonstrate the significant performance advantage of this new technology. All I/O from WekaIO Matrix software is handled at 4K level granularity so our software can interact directly with the underlying SSDs.
We chose to partner with Supermicro for the server infrastructure and Micron NVMe SSDs for the high performance persistent storage layer to prove that best of breed components can deliver superior performance to dedicated custom appliances. The benefit of using industry standard components is that you get to choose the best platform for your unique environment at the best economics.
Why Micron?
Micron has been a pioneer of NAND flash technologies, and has a broad portfolio of solutions to address a spectrum of performance needs. We chose Micron SSDs as our own internal tests had shown terrific performance on the Micron NVMe SSDs. We were also influenced by independent analyst coverage which was heralded the Micron as “the highest performing enterprise NVMe SSD….”. Micron’s NVMe SSD platform has other advantages like Micron’s FlexPro firmware architecture that lets you actively tune capacity to optimize drive performance and endurance. The product portfolio has a broad spectrum of capacity points from 1.6TB write-intensive drives to 11TB read-intensive drives so customers can pick the best fit for their unique environment.
Testing Real World Environments
Typical benchmark reports are measuring synthetic results with little relevance to real-world applications. The SPEC organization has chosen to develop benchmarks based on actual customer workloads. The SPEC2014 Software Build workload is a classic metadata intensive workload (one of the most challenging workloads in storage) that pushes the limits of I/O workloads and is a prime benchmark for putting both file systems and the underlying NVMe storage through its paces.
As evidenced in the figure above, the benchmark results proved that best of breed of NVMe drives from Micron, Supermicro servers and WekaIO Matrix software outperformed the previous benchmark results by 2x. Prior to this submission, the #1 position for published benchmark results had been captured by IBM Spectrum Scale software with HGST’s SN200 NVMe SSDs.
Engineering solutions with the best of breed components is as critical to any great storage solution and these benchmark results prove that the WekaIO engineered solution with Micron NVMe drives and Matrix software provide industry best performance and economics.
If you have additional questions about our testing you can email Micron at ssd@micron.com.
