BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map-Reduce Applications

Bogdan Nicolae 1 Diana Moise 1 Gabriel Antoniu 1, * Luc Bougé 1 Matthieu Dorier 1
* Auteur correspondant
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : Hadoop is a software framework supporting the Map-Reduce programming model. It relies on the Hadoop Distributed File System (HDFS) as its primary storage system. The efficiency of HDFS is crucial for the performance of Map-Reduce applications. We substitute the original HDFS layer of Hadoop with a new, concurrency-optimized data storage layer based on the BlobSeer data management service. Thereby, the efficiency of Hadoop is significantly improved for data-intensive Map-Reduce applications, which naturally exhibit a high degree of data access concurrency. Moreover, BlobSeer's features (built-in versioning, its support for concurrent append operations) open the possibility for Hadoop to further extend its functionalities. We report on extensive experiments conducted on the Grid'5000 testbed. The results illustrate the benefits of our approach over the original HDFS-based implementation of Hadoop.
Type de document :
Communication dans un congrès
24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010), Apr 2010, Atlanta, United States. 2010, 〈http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5470433〉. 〈10.1109/IPDPS.2010.5470433〉
Liste complète des métadonnées

Littérature citée [9 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00456801
Contributeur : Luc Bougé <>
Soumis le : lundi 15 février 2010 - 17:28:20
Dernière modification le : mercredi 16 mai 2018 - 11:23:28
Document(s) archivé(s) le : jeudi 30 juin 2011 - 12:08:31

Fichier

PID1126243.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Bogdan Nicolae, Diana Moise, Gabriel Antoniu, Luc Bougé, Matthieu Dorier. BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map-Reduce Applications. 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010), Apr 2010, Atlanta, United States. 2010, 〈http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5470433〉. 〈10.1109/IPDPS.2010.5470433〉. 〈inria-00456801〉

Partager

Métriques

Consultations de la notice

801

Téléchargements de fichiers

560