Red Hat Research Days Talk: Endure: A Robust Tuning Paradigm for LSM Trees Under Workload Uncertainty

Join Red Hat Research for the next Research Days event, “Endure: A Robust Tuning Paradigm for LSM Trees Under Workload Uncertainty ” on February 8, 2023 from 11AM to 12:30PM EST (5PM CET, 6PM IST).  

Abstract: Log-Structured Merge trees (LSM trees) are increasingly used as the storage engines behind several data systems, frequently deployed in the cloud. Similar to other database architectures, LSM trees consider information about the expected workload (e.g., reads vs. writes, point vs. range queries) to optimize their performance via tuning. However, operating in a shared infrastructure like the cloud comes with workload uncertainty due to the fast-evolving nature of modern applications. Systems with static tuning discount the variability of such hybrid workloads and hence provide inconsistent and overall suboptimal performance.In this talk we introduce Endure – a new paradigm for tuning LSM trees in the presence of workload uncertainty. We introduce the robust problem and show how it differs from the traditional tuning problem. Then we will discuss how to define uncertainty and use this definition to recommend tunings that perform on average better under the presence of a changing workload. We show the potential of robust tunings by implementing Endure in RockDB, a popular LSM engine, and analyze the performance benefits of using robust tunings.

Speaker: Andy Hyunh

This will be held remotely. Additional details to be added