Towards Efficient Power Management in MapReduce: Investigation of CPU-Frequencies Scaling on Power Efficiency in Hadoop

Shadi Ibrahim 1, * Diana Moise 2 Houssem-Eddine Chihoub 3 Alexandra Carpen-Amarie 4 Luc Bougé 1 Gabriel Antoniu 1
* Auteur correspondant
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
3 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : With increasingly inexpensive cloud storage and increasingly powerful cloud processing, the cloud has rapidly become the environment to store and analyze data. Most of the large-scale data computations in the cloud heavily rely on the MapReduce paradigm and its Hadoop implementation. Nevertheless, this exponential growth in popularity has significantly impacted power consumption in cloud infrastructures. In this paper, we focus on MapReduce and we investigate the impact of dynamically scaling the frequency of compute nodes on the performance and energy consumption of a Hadoop cluster. To this end, a series of experiments are conducted to explore the implications of Dynamic Voltage Frequency scaling (DVFS) settings on power consumption in Hadoop-clusters. By adapting existing DVFS governors (i.e., performance, powersave, ondemand, conservative and userspace) in the Hadoop cluster, we observe significant variation in performance and power consumption of the cluster with different applications when applying these governors: the different DVFS settings are only sub-optimal for different MapReduce applications. Furthermore, our results reveal that the current CPU governors do not exactly reflect their design goal and may even become ineffective to manage the power consumption in Hadoop clusters. This study aims at providing more clear understanding of the interplay between per- formance and power management in Hadoop cluster and therefore offers useful insight into designing power-aware techniques for Hadoop systems.
Type de document :
Communication dans un congrès
Workshop on Adaptive Resource Management and Scheduling for Cloud Computing, Held in conjunction with PODC 2014, Jul 2014, Paris, France
Liste complète des métadonnées

Littérature citée [31 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01077285
Contributeur : Shadi Ibrahim <>
Soumis le : lundi 22 juin 2015 - 10:30:44
Dernière modification le : jeudi 24 mai 2018 - 15:59:21
Document(s) archivé(s) le : mardi 25 avril 2017 - 18:03:20

Fichier

arms-cc-paper-10.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01077285, version 1

Citation

Shadi Ibrahim, Diana Moise, Houssem-Eddine Chihoub, Alexandra Carpen-Amarie, Luc Bougé, et al.. Towards Efficient Power Management in MapReduce: Investigation of CPU-Frequencies Scaling on Power Efficiency in Hadoop . Workshop on Adaptive Resource Management and Scheduling for Cloud Computing, Held in conjunction with PODC 2014, Jul 2014, Paris, France. 〈hal-01077285〉

Partager

Métriques

Consultations de la notice

1154

Téléchargements de fichiers

286