Elastic Phoenix: Malleable MapReduce for Shared-Memory Systems

Abstract : We present the design, implementation, and an evaluation of Elastic Phoenix. Based on the original Phoenix from Stanford, Elastic Phoenix is also a MapReduce implementation for shared-memory systems. The key new feature of Elastic Phoenix is that it supports malleable jobs: the ability add and remove worker processes during the execution of a job. With the original Phoenix, the number of processors to be used is fixed at start-up time. With Elastic Phoenix, if more resources become available (as they might on an elastic cloud computing system), they can be dynamically added to an existing job. If resources are reclaimed, they can also be removed from an existing job. The concept of malleable jobs is well known in job scheduling research, but an implementation of a malleable programming system like Elastic Phoenix is less common.We show how dynamically increasing the resources available to an Elastic Phoenix workload as it runs can reduce response time by 29% compared to a statically resourced workload. We detail the changes to the Phoenix application programming interface (API) made to support the new capability, and discuss the implementation changes to the Phoenix code base. We show that any additional run-time overheads introduced by Elastic Phoenix can be offset by the benefits of dynamically adding processor resources.
Type de document :
Communication dans un congrès
Erik Altman; Weisong Shi. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. Springer, Lecture Notes in Computer Science, LNCS-6985, pp.1-16, 2011, Network and Parallel Computing. 〈10.1007/978-3-642-24403-2_1〉
Liste complète des métadonnées

Littérature citée [10 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01593035
Contributeur : Hal Ifip <>
Soumis le : lundi 25 septembre 2017 - 16:34:27
Dernière modification le : vendredi 1 décembre 2017 - 01:09:45
Document(s) archivé(s) le : mardi 26 décembre 2017 - 14:42:17

Fichier

978-3-642-24403-2_1_Chapter.pd...
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Adam Wolfe Gordon, Paul Lu. Elastic Phoenix: Malleable MapReduce for Shared-Memory Systems. Erik Altman; Weisong Shi. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. Springer, Lecture Notes in Computer Science, LNCS-6985, pp.1-16, 2011, Network and Parallel Computing. 〈10.1007/978-3-642-24403-2_1〉. 〈hal-01593035〉

Partager

Métriques

Consultations de la notice

142

Téléchargements de fichiers

22