By Dr. Ismail Oukid
Dr. Ismail Oukid and a team of SAP, TU Dresden, and Intel researchers are finalists for the 2018 Memorable Paper Award for System Architecture and Applications at the 9th Annual Non-Volatile Memories Workshop (NVMW) hosted by the University of California, San Diego. Their paper, "
Memory Management Techniques for Large-Scale Persistent-Main-Memory Systems" was one of six finalists for the award selected from over 80 papers submitted to the workshop. A full citation for their work and a summary is below.
Ismail Oukid, Daniel Booss, Adrien Lespinasse, Wolfgang Lehner, Thomas Willhalm, and Grégoire Gomes. 2017. Memory management techniques for large-scale persistent-main-memory systems. Proc. VLDB Endow. 10, 11 (August 2017), 1166-1177
Paper Summary
Storage-Class Memory (SCM) is a class of novel memory technologies, best exemplified by Intel’s 3D XPoint, that combine the non-volatility, high capacity, and economic characteristics of traditional storage media (e.g., SSDs) with the byte-addressability and a latency close to that of DRAM (albeit higher). Given its characteristics, SCM can be used as main memory or as storage. However, while SCM is projected to be cheaper than DRAM, it will be too expensive to replace flash and too slow to replace DRAM. In this work we make the case for a third option: using SCM as persistent memory (PM), i.e., as main memory and storage at the same time. Specifically, we endeavor to devise persistent memory management techniques as the first necessary building block to enable PM-based software.
The programming model for main memory has been developed under the premise that a computer loses its DRAM contents across restarts, therefore, it does not account for data recoverability. PM invalidates this assumption as it retains its data across power failures, thereby maintaining the topology of existing memory blocks. Hence, the first requirement for a PM allocator is to provide a programming model that accounts for data recoverability. Furthermore, in the case of DRAM, restarting the computer is a last-resort but valid way of eliminating memory leaks and fragmentation. This is not applicable to PM given its non-volatility. Hence, a PM allocator must avoid memory leaks and minimize fragmentation, as both are potentially perpetual.
Moreover, to store data, software has traditionally assumed block-addressable devices, managed by a file system and accessed through main memory, giving the programmer full control over when data is persisted. Conversely, data is read, modified, and persisted directly in PM through the CPU caches over which the programmer has little control. This can lead to partial writes and an inconsistent program state in case of failure. Therefore, there is a need to enforce the order and durability of PM writes using persistence primitives, such as memory barriers and CPU cache line flushing instructions to achieve failure atomicity. Failure-atomicity means that logical operations, such as allocating a PM block, either execute to completion or are rolled back if a failure occurs during execution.
To solve the above challenges, we devise PAllocator, a highly scalable, failure-atomic PM allocator tailored for large-scale systems such as main-memory databases. PAllocator employs a sound PM programming model that provides a recoverable addressing scheme and avoids memory leaks. Additionally, PAllocator minimizes fragmentation by employing three different allocation strategies depending on the size of requested blocks. Notably, PAllocator is the first of its kind to embed a defragmentation algorithm.
This work has been conducted within the SAP HANA Campus – a research group in the heart of the SAP HANA development team – and is the fruit of a collaboration between TU Dresden, SAP, and Intel. Some of the concepts developed in this work have been successfully integrated in the SAP HANA database, paving the way for PM adoption.
About The Memorable Paper Award
The Memorable Paper Award recognizes the best recent research on non-volatile memories published throughout the world. It is given annually to outstanding research published in the last two years that is expected to have substantial impact on the study of non-volatile memories. To be eligible, the paper must have been published in peer-reviewed venue in the last two years and the lead researcher must have been a student at the time.
About the Non-Volatile Memories Workshop
The Non-Volatile Memories Workshop is the world’s premier venue for research into how to use non-volatile memory technology to improve the performance, reliability, and efficiency of computing systems. It was founded in 2010 by Dr. Paul Siegel and Dr. Steven Swanson of the University of California, San Diego’s Jacob School of engineering. The workshop is a co-production of the Center for Magnetic Recording Research (
http://cmrr.ucsd.edu) and the Non-Volatile Systems Laboratory (
http://nvsl.ucsd.edu) at UC San Diego. More information, including a detailed program, is available at
http://nvmw.ucsd.edu.
