Availability/Network-aware MapReduce over the Internet

Abstract : MapReduce offers an ease-of-use programming paradigm for processing large datasets. In our previous work, we have designed a MapReduce framework called BitDew-MapReduce for desktop grid and volunteer computing environment, that allows nonexpert users to run data-intensive MapReduce jobs on top of volunteer resources over the Internet. However, network distance and resource availability have great impact on MapReduce applications running over the Internet. To address this, an availability and network-aware MapReduce framework over the Internet is proposed. Simulation results show that the MapReduce job response time could be decreased by 40.05%, thanks to Weighted Naive Bayes Classifier-based availability prediction and landmark-based network estimation. The effectiveness of the new MapReduce framework is further proved by performance evaluation in a real distributed environment.
Type de document :
Article dans une revue
Information Sciences, Elsevier, 2016, 379, pp.94--111. 〈10.1016/j.ins.2016.09.030〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01426393
Contributeur : Gilles Fedak <>
Soumis le : mercredi 4 janvier 2017 - 14:42:26
Dernière modification le : vendredi 20 avril 2018 - 15:44:26

Identifiants

Collections

Citation

Bing Tang, Mingdong Tang, Gilles Fedak, Haiwu He. Availability/Network-aware MapReduce over the Internet. Information Sciences, Elsevier, 2016, 379, pp.94--111. 〈10.1016/j.ins.2016.09.030〉. 〈hal-01426393〉

Partager

Métriques

Consultations de la notice

210