Forget what you previously knew about high-performance storage and file systems as Intel® Optane™ DC Persistent Memory Modules (Intel® Optane™ DCPMM) devices ingeniously package non-volatile Intel Optane memory in a DDR form factor that is accessed via the memory bus. The means the CPU no longer needs to call the operating system to perform IO operations. Extremely simple to understand, the impact is profoundly changing HPC. The capacity, latency, and bandwidth benefits are too compelling. Don’t be confused by the DC (e.g. “Data Center”) in the name, Intel Optane DC Persistent Memory Modules have HPC written all over them.

Extreme performance with Intel® Optane® DCPMM

Along with terabyte main memory capacity workstations and “fat” computational nodes, HPC users can also benefit from persistent Intel Optane DCPMM memory via memory mapped files in an AppDirect optimized filesystem such as XFS, EXT4, and NTFS or directly via an AppDirect optimized application as shown below.

Figure 1: The CPU can bypass significant overhead by directly accessing Intel DCPMM from user space (Source Intel)
(Click to enlarge)

The key to performance is that the CPU issues DAX instructions to directly communicate with storage in userspace. As illustrated in Figure 1, direct userspace access eliminates many of the frustrating performance limiting aspects of traditional HPC I/O that reduces storage bandwidth and increases latency such as the operating system, page cache, block driver, and PCIe bus. To speed development, Intel provides the Persistent Memory Development Kit (PMDK) to support the transactional operations necessary to keep the data consistent and durable as Intel Optane DPCMM represents a persistent store. To protect your data, the Intel Optane DCPMM module also supports on-module encryption via a 256bit AES-XTP encryption engine.

Reports from third-parties indicate the performance is spectacular:

• Lenovo reports that “DCPMM devices can provide up to multiple tens of GB/s of throughput at nanosecond latencies versus single digit GB/s of throughput at microsecond latencies seen with high performing NAND-based flash storage devices.”, [i]
• After 330 hours of in-depth benchmarking, the UCSD Non-Volatile Systems Laboratory observed that Intel Optane DCPMM “can make key storage applications up to 17 times faster”.
• The NVSL team also concluded that Intel Optane DCPMM memory, “provides comparable performance to DRAM for many of the real world applications we explored” when used to augment the main memory of the server.[ii]

A burgeoning software ecosystem is rapidly evolving that shows what is possible for many HPC applications including those that use extremely large, globally accessible data sets. Intel optimized Apache Spark, for example, doubles throughput and reduces runtime by up to 40% for data centric workloads. Other partner frameworks such as SAP HANA, Redis Labs, AsiaInfo, Aerospike, and a number of ISVs demonstrate how the bottleneck imposed by the current popular HPC parallel distributed file-systems such as Lustre and GPFS can be eliminated.

DAOS eliminates POSIX imposed IO limitations

The Intel Optane performance is so extreme that Intel is creating an entirely new open source software ecosystem based on Distributed Asynchronous Object Storage (DAOS) that will eliminate performance bottlenecks imposed on I/O by the POSIX standard.[iii] DAOS will enable next-generation data-centric workflows for simulation, data analytics, and AI on current and future exascale machines. [iv]

Succinctly, DAOS and Intel Optane provide a more performant (i.e. lower latency, higher BW/IOPS), scalable (i.e. millions of MPI tasks and thousands of storage servers) plus capable (e.g. fine-grain data access, native object API, key-value store) computational storage that supports resilient distributed storage for HPC applications and workflows.

Unlike the traditional storage stack, DAOS is architected from the ground up to make use of new NVM technologies such as Intel Optane DCPMM. It is extremely lightweight because it operates end-to-end in user space with full operating system bypass. DAOS offers a shift away from the traditional HPC block-based, high-latency storage model to one that inherently supports fine-grained data access and unlocks the performance of next generation storage systems.

Figure 2: The DAOS software stack versus conventional storage systems (Click to enlarge)

Forget the old metrics for high-performance storage and memory capacity, Intel Optane DC persistent memory delivers advantages for HPC that are both performant and profound.  Enjoy nanosecond access latency, multiple tens of gigabytes per second of bandwidth, nonvolatile storage, and terabyte capacity main memory systems based on Intel Optane DC persistent memory modules now, and the benefits of Intel DAOS in the near future.

Find more information about how DAOS revolutionizes high-performance storage with Intel Optane DC Persistent Memory [LINKS to https://www.intel.com/content/www/us/en/high-performance-computing/daos-high-performance-storage-brief.html]

[i] https://lenovopress.com/lp1083.pdf

[ii] https://arxiv.org/pdf/1903.05714.pdf

[iii] http://pages.cs.wisc.edu/~johnbent/Pubs/lofstead_sc16.pdf

[iv] https://www.intel.com/content/www/us/en/high-performance-computing/daos-high-performance-storage-brief.html

For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at [intel.com].

Intel, the Intel logo, and Optane are trademarks of Intel Corporation or its subsidiaries.

Other names and brands may be claimed as the property of others.

© Intel Corporation