Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis

Abstract : With increasingly inexpensive storage and growing processing power, the cloud has rapidly become the environment of choice to store and analyze data for a variety of applications. Most large-scale data computations in the cloud heavily rely on the MapReduce paradigm and on its Hadoop implementation. Nevertheless, this exponential growth in popularity has significantly impacted power consumption in cloud infrastructures. In this paper, we focus on MapReduce processing and we investigate the impact of dynamically scaling the frequency of compute nodes on the performance and energy consumption of a Hadoop cluster. To this end, a series of experiments are conducted to explore the implications of Dynamic Voltage and Frequency Scaling (DVFS) settings on power consumption in Hadoop clusters. By enabling various existing DVFS governors (i.e., performance, powersave, ondemand, conservative and userspace) in a Hadoop cluster, we observe significant variation in performance and power consumption across different applications: the different DVFS settings are only sub-optimal for several representative MapReduce applications. Furthermore, our results reveal that the current CPU governors do not exactly reflect their design goal and may even become ineffective to manage the power consumption in Hadoop clusters. This study aims at providing a clearer understanding of the interplay between performance and power management in Hadoop clusters and therefore offers useful insight into designing power-aware techniques for Hadoop systems.
Type de document :
Article dans une revue
Future Generation Computer Systems, Elsevier, 2016, pp.14. 〈10.1016/j.future.2015.01.005〉
Liste complète des métadonnées

Littérature citée [46 références]  Voir  Masquer  Télécharger
Contributeur : Shadi Ibrahim <>
Soumis le : vendredi 26 juin 2015 - 11:07:45
Dernière modification le : vendredi 16 novembre 2018 - 01:40:32
Document(s) archivé(s) le : mercredi 16 septembre 2015 - 01:02:07


Fichiers produits par l'(les) auteur(s)



Shadi Ibrahim, Tien-Dat Phan, Alexandra Carpen-Amarie, Houssem-Eddine Chihoub, Diana Moise, et al.. Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis. Future Generation Computer Systems, Elsevier, 2016, pp.14. 〈10.1016/j.future.2015.01.005〉. 〈hal-01166252〉



Consultations de la notice


Téléchargements de fichiers