On the Performance and Energy Efficiency of Hadoop Deployment Models

Abstract : The exponential growth of scientific and business data has resulted in the evolution of the cloud computing and the MapReduce parallel programming model. Cloud computing emphasizes increased utilization and power savings through consolidation while MapReduce enables large scale data analysis. The Hadoop framework has recently evolved to the standard framework implementing the MapReduce model. In this paper, we evaluate Hadoop performance in both the traditional model of collocated data and compute services as well as consider the impact of separating out the services. The separation of data and compute services provides more flexibility in environments where data locality might not have a considerable impact such as virtualized environments and clusters with advanced networks. In this paper, we also conduct an energy efficiency evaluation of Hadoop on physical and virtual clusters in different configurations. Our extensive evaluation shows that: (1) performance on physical clusters is significantly better than on virtual clusters; (2) performance degradation due to separation of the services depends on the data to compute ratio; (3) application completion progress correlates with the power consumption and power consumption is heavily application specific.
Type de document :
Communication dans un congrès
The IEEE International Conference on Big Data 2013 (IEEE BigData 2013), Oct 2013, Santa Clara, United States. 2013
Liste complète des métadonnées

Littérature citée [20 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00856330
Contributeur : Eugen Feller <>
Soumis le : mardi 8 octobre 2013 - 01:32:59
Dernière modification le : jeudi 15 novembre 2018 - 11:57:44
Document(s) archivé(s) le : jeudi 6 avril 2017 - 11:23:25

Fichier

main.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00856330, version 1

Citation

Eugen Feller, Lavanya Ramakrishnan, Christine Morin. On the Performance and Energy Efficiency of Hadoop Deployment Models. The IEEE International Conference on Big Data 2013 (IEEE BigData 2013), Oct 2013, Santa Clara, United States. 2013. 〈hal-00856330〉

Partager

Métriques

Consultations de la notice

993

Téléchargements de fichiers

1530