Energy-Driven Straggler Mitigation in MapReduce

Tien-Dat Phan 1 Shadi Ibrahim 2, 3 Amelie Zhou 2 Guillaume Aupy 4 Gabriel Antoniu 1
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA_D1 - SYSTÈMES LARGE ÉCHELLE
2 ASCOLA - Aspect and Composition Languages
Inria Rennes – Bretagne Atlantique , LS2N - Laboratoire des Sciences du Numérique de Nantes
4 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
Abstract : Energy consumption is an important concern for large-scale data-centers, which results in huge monetary cost for data-center operators. Due to the hardware heterogeneity and contentions between concurrent workloads, straggler mitigation is important to many Big Data applications running in large-scale data-centers and the speculative execution technique is widely-used to handle stragglers. Although a large number of studies have been proposed to improve the performance of Big Data applications using speculative execution, few of them have studied the energy efficiency of their solutions. In this paper, we propose two techniques to improve the energy efficiency of speculative executions while ensuring comparable performance. Specifically, we propose a hierarchical straggler detection mechanism which can greatly reduce the number of killed speculative copies and hence save the energy consumption. We also propose an energy-aware speculative copy allocation method which considers the trade-off between performance and energy when allocating speculative copies. We implement both techniques into Hadoop and evaluate them using representative MapReduce benchmarks. Results show that our solution can reduce the energy waste on killed speculative copies by up to 100% and improve the energy efficiency by 20% compared to state-of-the-art mechanisms.
Type de document :
Communication dans un congrès
Euro-Par'17 - 23rd International European Conference on Parallel and Distributed Computing , Aug 2017, Santiago de Compostela, Spain
Liste complète des métadonnées

Littérature citée [22 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01560044
Contributeur : Shadi Ibrahim <>
Soumis le : mardi 11 juillet 2017 - 11:09:16
Dernière modification le : mardi 4 septembre 2018 - 11:04:01
Document(s) archivé(s) le : jeudi 25 janvier 2018 - 04:46:14

Fichier

EuroPar2017.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01560044, version 1

Citation

Tien-Dat Phan, Shadi Ibrahim, Amelie Zhou, Guillaume Aupy, Gabriel Antoniu. Energy-Driven Straggler Mitigation in MapReduce. Euro-Par'17 - 23rd International European Conference on Parallel and Distributed Computing , Aug 2017, Santiago de Compostela, Spain. 〈hal-01560044〉

Partager

Métriques

Consultations de la notice

815

Téléchargements de fichiers

187