Towards MapReduce for Desktop Grid Computing

Abstract : MapReduce is an emerging programming model for data-intense application proposed by Google, which has attracted a lot of attention recently. MapReduce borrows from functional programming, where programmer defines Map and Reduce tasks executed on large set of distributed data. In this paper we propose an implementation of the MapReduce programming model. We present the architecture of the prototype based on Bit Dew, a middleware for large scale data management on Desktop Grid. We describe the set of features which makes our approach suitable for large scale and loosely connected Internet Desktop Grid: massive fault tolerance, replica management, barriers-free execution, latency-hiding optimisation as well as distributed result checking. We also present performance evaluation of the prototype both against micro-benchmarks and real MapReduce application. The scalability test shows that we achieve linear speedup on the classical Word Count benchmark. Several scenarios involving lagger hosts and host crashes demonstrate that the prototype is able to cope with an experimental context similar to real-world Internet.
Type de document :
Communication dans un congrès
P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC), 2010 International Conference on, Nov 2010, Fukuoka, Japan. 2010, 〈10.1109/3PGCIC.2010.33〉
Liste complète des métadonnées

https://hal.inria.fr/hal-00687553
Contributeur : Ist Rennes <>
Soumis le : vendredi 13 avril 2012 - 14:57:10
Dernière modification le : samedi 21 avril 2018 - 01:27:08

Lien texte intégral

Identifiants

Citation

Bing Tang, Mircea Moca, Stéphane Chevalier, Gilles Fedak. Towards MapReduce for Desktop Grid Computing. P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC), 2010 International Conference on, Nov 2010, Fukuoka, Japan. 2010, 〈10.1109/3PGCIC.2010.33〉. 〈hal-00687553〉

Partager

Métriques

Consultations de la notice

166