FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads

Abstract : Originally, MapReduce implementations such as Hadoop employed First In First Out (fifo) scheduling, but such simple schemes cause job starvation. The Hadoop Fair Scheduler (hfs) is a slot-based MapReduce scheme designed to ensure a degree of fairness among the jobs, by guaranteeing each job at least some minimum number of allocated slots. Our prime contribution in this paper is a different, flexible scheduling allocation scheme, known as flex. Our goal is to optimize any of a variety of standard scheduling theory metrics (response time, stretch, makespan and Service Level Agreements (slas), among others) while ensuring the same minimum job slot guarantees as in hfs, and maximum job slot guarantees as well. The flex allocation scheduler can be regarded as an add-on module that works synergistically with hfs. We describe the mathematical basis for flex, and compare it with fifo and hfs in a variety of experiments.
Type de document :
Communication dans un congrès
Indranil Gupta; Cecilia Mascolo. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. Springer, Lecture Notes in Computer Science, LNCS-6452, pp.1-20, 2010, Middleware 2010. 〈10.1007/978-3-642-16955-7_1〉
Liste complète des métadonnées

Littérature citée [17 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01055274
Contributeur : Hal Ifip <>
Soumis le : mardi 12 août 2014 - 11:47:14
Dernière modification le : mercredi 16 août 2017 - 17:20:50
Document(s) archivé(s) le : mercredi 26 novembre 2014 - 22:41:48

Fichier

middleware10-cr.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, et al.. FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads. Indranil Gupta; Cecilia Mascolo. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. Springer, Lecture Notes in Computer Science, LNCS-6452, pp.1-20, 2010, Middleware 2010. 〈10.1007/978-3-642-16955-7_1〉. 〈hal-01055274〉

Partager

Métriques

Consultations de la notice

163

Téléchargements de fichiers

537