BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map/Reduce Applications

Bogdan Nicolae 1 Diana Moise 1 Gabriel Antoniu 1, * Luc Bougé 1 Matthieu Dorier 1
* Corresponding author
1 KerData - Scalable Storage for Clouds and Beyond
IRISA-D1 - SYSTÈMES LARGE ÉCHELLE, Inria Rennes – Bretagne Atlantique
Abstract : Hadoop is a software framework supporting the Map/Reduce programming model. It relies on the Hadoop Distributed File System (HDFS) as its primary storage system. The efficiency of HDFS is crucial for the performance of Map/Reduce applications. We substitute the original HDFS layer of Hadoop with a new, concurrency-optimized data storage layer based on the BlobSeer data management service. Thereby, the efficiency of Hadoop is significantly improved for data-intensive Map/Reduce applications, which naturally exhibit a high degree of data access concurrency. Moreover, BlobSeer's features (built-in versioning, its support for concurrent append operations) open the possibility for Hadoop to further extend its functionalities. We report on extensive experiments conducted on the Grid'5000 testbed. The results illustrate the benefits of our approach over the original HDFS-based implementation of Hadoop.
Document type :
Reports
Complete list of metadatas

https://hal.inria.fr/inria-00440312
Contributor : Luc Bougé <>
Submitted on : Thursday, December 10, 2009 - 11:41:16 AM
Last modification on : Wednesday, October 2, 2019 - 3:58:13 PM
Long-term archiving on : Thursday, June 30, 2011 - 11:19:23 AM

Files

rr-7140.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00440312, version 1

Citation

Bogdan Nicolae, Diana Moise, Gabriel Antoniu, Luc Bougé, Matthieu Dorier. BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map/Reduce Applications. [Research Report] RR-7140, INRIA. 2009, pp.20. ⟨inria-00440312⟩

Share

Metrics

Record views

489

Files downloads

582