Skip to Main content Skip to Navigation
New interface
Conference papers

A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores

Abstract : Erasure codes have been widely used over the last decade to implement reliable data stores. They offer interesting trade-offs between efficiency, reliability, and storage overhead. Indeed, a distributed data store holding encoded data blocks can tolerate the failure of multiple nodes while requiring only a fraction of the space necessary for plain replication, albeit at an increased encoding and decoding cost. There exists nowadays a number of libraries implementing several variations of erasure codes, which notably differ in terms of complexity and implementation-specific optimizations.Seven years ago, Plank et al. [14] have conducted a comprehensive performance evaluation of open-source erasure coding libraries available at the time to compare their raw performance and measure the impact of different parameter configurations. In the present experimental study, we take a fresh perspective at the state of the art of erasure coding libraries. Not only do we cover a wider set of libraries running on modern hardware, but we also consider their efficiency when used in realistic settings for cloud-based storage, namely when deployed across several nodes in a data centre. Our measurements therefore account for the end-to-end costs of data accesses over several distributed nodes, including the encoding and decoding costs, and shed light on the performance one can expect from the various libraries when deployed in a real system. Our results reveal important differences in the efficiency of the different libraries, notably due to the type of coding algorithm and the use of hardware-specific optimizations.
Complete list of metadata

Cited literature [18 references]  Display  Hide  Download
Contributor : Hal Ifip Connect in order to contact the contributor
Submitted on : Friday, January 13, 2017 - 2:02:29 PM
Last modification on : Wednesday, November 28, 2018 - 2:48:22 PM
Long-term archiving on: : Friday, April 14, 2017 - 7:46:41 PM


Files produced by the author(s)


Distributed under a Creative Commons Attribution 4.0 International License



Dorian Burihabwa, Pascal Felber, Hugues Mercier, Valerio Schiavoni. A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. pp.160-173, ⟨10.1007/978-3-319-39577-7_13⟩. ⟨hal-01434792⟩



Record views


Files downloads