Availability and Network-Aware MapReduce Task Scheduling over the Internet

Abstract : MapReduce offers an ease-of-use programming paradigm for processing large datasets. In our previous work, we have designed a MapReduce framework called BitDew-MapReduce for desktop grid and volunteer computing environment, that allows nonexpert users to run data-intensive MapReduce jobs on top of volunteer resources over the Internet. However, network distance and resource availability have great impact on MapReduce applications running over the Internet. To address this, an availability and network-aware MapReduce framework over the Internet is proposed. Simulation results show that the MapReduce job response time could be decreased by 27.15%, thanks to Naive Bayes Classifier-based availability prediction and landmark-based network estimation .
Type de document :
Communication dans un congrès
Algorithms and Architectures for Parallel Processing, Dec 2015, Zhangjiajie, China. 9528, 2015, Lecture Notes in Computer Science 〈10.1007/978-3-319-27119-4_15〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01256183
Contributeur : Gilles Fedak <>
Soumis le : jeudi 14 janvier 2016 - 14:47:39
Dernière modification le : jeudi 8 février 2018 - 11:10:04
Document(s) archivé(s) le : samedi 16 avril 2016 - 10:40:12

Fichier

ICA3PP_2015_submission_274.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité - Pas d'utilisation commerciale 4.0 International License

Identifiants

Collections

Citation

Bing Tang, Qi Xie, Haiwu He, Gilles Fedak. Availability and Network-Aware MapReduce Task Scheduling over the Internet. Algorithms and Architectures for Parallel Processing, Dec 2015, Zhangjiajie, China. 9528, 2015, Lecture Notes in Computer Science 〈10.1007/978-3-319-27119-4_15〉. 〈hal-01256183〉

Partager

Métriques

Consultations de la notice

347

Téléchargements de fichiers

128