Volume Storage Over Object Storage

The goal of this project is to create block storage that could combine the high performance of locally-attached storage with the features of network-mounted storage (e.g, data availability), while also benefiting from object store advantages. We exploit the fact that many cloud applications and services are already designed to be resilient to failures occurring at their locally attached storage. Therefore, using the reliability provided by current cloud block storage only results in data and operation redundancy .

To achieve our goal, we propose a hybrid storage system composed of a high-speed local device (e.g. Optane) to store short term data, along with a write-once object store (e.g, Ceph RGW) to store data blocks permanently. Data management of this storage system is done through a translation layer. Data blocks are first forwarded to local storage on write operations and then periodically coalesced and flushed to the write-once object store as large objects.