Performance and Energy Efficiency of Big Data Applications in Cloud Environments: A Hadoop Case Study - Archive ouverte HAL Access content directly
Reports (Research Report) Year : 2014

Performance and Energy Efficiency of Big Data Applications in Cloud Environments: A Hadoop Case Study

(1) , (2) , (1)
1
2

Abstract

The exponential growth of scienti c and business data has resulted in the evolution of the cloud computing environments and the MapReduce parallel programming model. The focus of cloud computing is increased utilization and power savings through consolidation while MapReduce enables large scale data analysis. Hadoop, an open source implementation of MapReduce has gained popularity in the last few years. In this paper, we evaluate Hadoop performance in both the traditional model of collocated data and compute services as well as consider the impact of separating out the services. The separation of data and compute services provides more exibility in environments where data locality might not have a considerable impact such as virtualized environments and clusters with advanced networks. In this paper, we also conduct an energy eciency evaluation of Hadoop on physical and virtual clusters in di erent con gurations. Our extensive evaluation shows that: (1) coexisting virtual machines on servers decrease the disk throughput; (2) performance on physical clusters is signi cantly better than on virtual clusters; (3) performance degradation due to separation of the services depends on the data to compute ratio; (4) application completion progress correlates with the power consumption and power consumption is heavily application speci c. Finally, we present a discussion on the implications of using cloud environments for big data analyses.
Not file

Dates and versions

hal-01102198 , version 1 (12-01-2015)

Identifiers

  • HAL Id : hal-01102198 , version 1

Cite

Eugen Feller, Lavanya Ramakrishnan, Christine Morin. Performance and Energy Efficiency of Big Data Applications in Cloud Environments: A Hadoop Case Study. [Research Report] LBNL-6861E, Lawrence Berkeley National Laboratory, California, USA. 2014. ⟨hal-01102198⟩
1566 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More