A High-Performance Storage Engine for Modern Hardware

LeanStore is a high-performance OLTP storage engine optimized for many-core CPUs and NVMe SSDs. Our goal is to achieve performance comparable to in-memory systems when the data set fits into RAM, while being able to fully exploit the bandwidth of fast NVMe SSDs for large data sets. While LeanStore is currently a research prototype, we hope to make it usable in production in the future.

Highlights

Support for Very Large Data Sets on Directly-Attached NVMe Arrays

NVMe SSDs have become cheap (1 TB cost around 200 USD) and fast (achieving more than 2-7 GB/s bandwidth per device). Existing storage engines are not capable of exploiting such fast IO devices, in particular when multiple NVMe SSDs are combined in a single server. LeanStore has been designed from scratch for Directly-Attached NVMe Arrays to get the full performance from fast flash storage devices.

High In-Memory Performance

Using optimized index structures, LeanStore achieves very high in-memory performance and scalable synchronization techniques ensure that it scales very well on many-core CPUs. A lightweight buffer manager transparently keeps frequently-accessed data in RAM, while supporting arbitrarily-large data sets on SSD.

Embeddable Open Source Library

Similar to storage engines like RocksDB and WiredTiger, LeanStore can be embedded into applications by linking it as a library. It offers a C++ interface for basic data operations like point lookup, range lookup, insert, update, delete. Support for ACID transactions is currently under development. The source code is available under MIT license.

Publications

Venue Publication Link
VLDB 2023 Scalable and Robust Snapshot Isolation for High-Performance Storage Engines
Adnan Alhomssi, Viktor Leis
 
BTW 2023 The Evolution of LeanStore
Adnan Alhomssi, Michael Haubenschild, Viktor Leis
 
SIGMOD 2023 Virtual-Memory Assisted Buffer Management
Viktor Leis, Adnan Alhomssi, Tobias Ziegler, Yannick Loeck, Christian Dietrich
pdf
CIDR 2021 Contention and Space Management for B-Trees
Adnan Alhomssi, Viktor Leis
pdf - code
SIGMOD 2020 Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines
Michael Haubenschild, Caetano Sauer, Thomas Neumann, Viktor Leis
pdf
CIDR 2020 Exploiting Directly-Attached NVMe Arrays in DBMS
Gabriel Haas, Michael Haubenschild, Viktor Leis
pdf
IEEE Data Engineering Bulletin Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method
Viktor Leis, Michael Haubenschild, Thomas Neumann
pdf
ICDE 2018 LeanStore: In-Memory Data Management Beyond Main Memory
Viktor Leis, Michael Haubenschild, Alfons Kemper, Thomas Neumann
pdf