A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores

Abstract : Erasure codes have been widely used over the last decade to implement reliable data stores. They offer interesting trade-offs between efficiency, reliability, and storage overhead. Indeed, a distributed data store holding encoded data blocks can tolerate the failure of multiple nodes while requiring only a fraction of the space necessary for plain replication, albeit at an increased encoding and decoding cost. There exists nowadays a number of libraries implementing several variations of erasure codes, which notably differ in terms of complexity and implementation-specific optimizations.Seven years ago, Plank et al. [14] have conducted a comprehensive performance evaluation of open-source erasure coding libraries available at the time to compare their raw performance and measure the impact of different parameter configurations. In the present experimental study, we take a fresh perspective at the state of the art of erasure coding libraries. Not only do we cover a wider set of libraries running on modern hardware, but we also consider their efficiency when used in realistic settings for cloud-based storage, namely when deployed across several nodes in a data centre. Our measurements therefore account for the end-to-end costs of data accesses over several distributed nodes, including the encoding and decoding costs, and shed light on the performance one can expect from the various libraries when deployed in a real system. Our results reveal important differences in the efficiency of the different libraries, notably due to the type of coding algorithm and the use of hardware-specific optimizations.
Type de document :
Communication dans un congrès
Márk Jelasity; Evangelia Kalyvianaki. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. Lecture Notes in Computer Science, LNCS-9687, pp.160-173, 2016, Distributed Applications and Interoperable Systems. 〈10.1007/978-3-319-39577-7_13〉
Liste complète des métadonnées

Littérature citée [18 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01434792
Contributeur : Hal Ifip <>
Soumis le : vendredi 13 janvier 2017 - 14:02:29
Dernière modification le : vendredi 13 janvier 2017 - 14:05:49
Document(s) archivé(s) le : vendredi 14 avril 2017 - 19:46:41

Fichier

 Accès restreint
Fichier visible le : 2019-01-01

Connectez-vous pour demander l'accès au fichier

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Dorian Burihabwa, Pascal Felber, Hugues Mercier, Valerio Schiavoni. A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores. Márk Jelasity; Evangelia Kalyvianaki. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. Lecture Notes in Computer Science, LNCS-9687, pp.160-173, 2016, Distributed Applications and Interoperable Systems. 〈10.1007/978-3-319-39577-7_13〉. 〈hal-01434792〉

Partager

Métriques

Consultations de la notice

59