MRSG - A MapReduce Simulator over SimGrid

Abstract : MapReduce is a parallel programming model to process large datasets, and it was inspired by the Map and Reduce primitives from functional languages. Its first implementation was designed to run on large clusters of homogeneous machines. Though, in the last years, the model was ported to different types of environments, such as desktop grid and volunteer computing. To obtain a good performance in these environments, however, it is necessary to adapt some framework mechanisms, such as scheduling and data distribution algorithms. In this paper we present the MRSG simulator, which reproduces the MapReduce work-flow on top of the SimGrid simulation toolkit, and provides an API to implement and evaluate these new algorithms and policies for MapReduce. To evaluate the simulator, we compared its behavior against a real Hadoop MapReduce deployment. The results show an important similarity between the simulated and real executions.
Type de document :
Article dans une revue
Parallel Computing, Elsevier, 2013, 39 (4-5), pp.233--244. 〈10.1016/j.parco.2013.02.001〉
Liste complète des métadonnées
Contributeur : Pierre Sens <>
Soumis le : mercredi 15 janvier 2014 - 23:16:23
Dernière modification le : mardi 17 avril 2018 - 11:32:28




Wagner Kolberg, Pedro De Botelho Marcos, Julio C. S. Anjos, Alexandre K. S. Miyazaki, Claudio R. Geyer, et al.. MRSG - A MapReduce Simulator over SimGrid. Parallel Computing, Elsevier, 2013, 39 (4-5), pp.233--244. 〈10.1016/j.parco.2013.02.001〉. 〈hal-00931855〉



Consultations de la notice