Self-configuration of the Number of Concurrently Running MapReduce Jobs in a Hadoop Cluster

Abstract : There is a trade-off between the number of concurrently running MapReduce jobs and their corresponding map and reduce tasks within a node in a Hadoop cluster. Leaving this trade-off statically configured to a single value can significantly reduce job response times leaving only suboptimal resource usage. To overcome this problem, we propose a feedback control loop based approach that dynamically adjusts the Hadoop resource manager configuration based on the current state of the cluster. The preliminary assessment based on workloads synthesized from real-world traces shows that the system performance can be improved by about 30% compared to default Hadoop setup.
Type de document :
Communication dans un congrès
ICAC 2015, Jul 2015, Grenoble, France. pp.149-150
Liste complète des métadonnées

Littérature citée [5 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01143157
Contributeur : Bo Zhang <>
Soumis le : mercredi 6 mai 2015 - 11:24:44
Dernière modification le : vendredi 13 octobre 2017 - 01:04:47
Document(s) archivé(s) le : mercredi 19 avril 2017 - 18:26:20

Fichier

icac15-paper.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01143157, version 1

Citation

Bo Zhang, Filip Křikava, Romain Rouvoy, Lionel Seinturier. Self-configuration of the Number of Concurrently Running MapReduce Jobs in a Hadoop Cluster. ICAC 2015, Jul 2015, Grenoble, France. pp.149-150. 〈hal-01143157〉

Partager

Métriques

Consultations de
la notice

328

Téléchargements du document

236