Skip to Main content Skip to Navigation
New interface
Journal articles

Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis

Abstract : With increasingly inexpensive storage and growing processing power, the cloud has rapidly become the environment of choice to store and analyze data for a variety of applications. Most large-scale data computations in the cloud heavily rely on the MapReduce paradigm and on its Hadoop implementation. Nevertheless, this exponential growth in popularity has significantly impacted power consumption in cloud infrastructures. In this paper, we focus on MapReduce processing and we investigate the impact of dynamically scaling the frequency of compute nodes on the performance and energy consumption of a Hadoop cluster. To this end, a series of experiments are conducted to explore the implications of Dynamic Voltage and Frequency Scaling (DVFS) settings on power consumption in Hadoop clusters. By enabling various existing DVFS governors (i.e., performance, powersave, ondemand, conservative and userspace) in a Hadoop cluster, we observe significant variation in performance and power consumption across different applications: the different DVFS settings are only sub-optimal for several representative MapReduce applications. Furthermore, our results reveal that the current CPU governors do not exactly reflect their design goal and may even become ineffective to manage the power consumption in Hadoop clusters. This study aims at providing a clearer understanding of the interplay between performance and power management in Hadoop clusters and therefore offers useful insight into designing power-aware techniques for Hadoop systems.
Complete list of metadata

Cited literature [46 references]  Display  Hide  Download
Contributor : Shadi Ibrahim Connect in order to contact the contributor
Submitted on : Friday, June 26, 2015 - 11:07:45 AM
Last modification on : Friday, July 8, 2022 - 10:09:09 AM
Long-term archiving on: : Wednesday, September 16, 2015 - 1:02:07 AM


Files produced by the author(s)



Shadi Ibrahim, Tien-Dat Phan, Alexandra Carpen-Amarie, Houssem-Eddine Chihoub, Diana Moise, et al.. Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis. Future Generation Computer Systems, 2016, pp.14. ⟨10.1016/j.future.2015.01.005⟩. ⟨hal-01166252⟩



Record views


Files downloads