Understanding Vertical Scalability of I/O Virtualization for MapReduce Workloads: Challenges and Opportunities

Bogdan Nicolae 1, *
* Auteur correspondant
Abstract : As the explosion of data sizes continues to push the limits of our abilities to efficiently store and process big data, next generation big data systems face multiple challenges. One such important challenge relates to the limited scalability of I/O, a determining factor in the overall performance of big data applications. Although paradigms like MapReduce have long been used to take advantage of local disks and avoid data movements over the network as much as possible, with increasing core count per node, local storage comes under increasing I/O pressure itself and prompts the need to equip nodes with multiple disks. However, given the rising need to virtualize large datacenters in order to provide a more flexible allocation and consolidation of physical resources (transforming them into public or private/hybrid clouds), the following questions arise: is it possible to take advantage of multiple local disks at virtual machine (VM) level in order to speed up big data analytics? If so, what are the best practices to achieve a high virtualized aggregated I/O throughput? This paper aims to answer these questions in the context of I/O intensive MapReduce workloads: it analyzes and characterizes their behavior under different virtualization scenarios in order to propose best practices for current approaches and speculate on future areas of improvement.
Type de document :
Communication dans un congrès
BigDataCloud'13: 2nd Workshop on Big Data Management in Clouds, Aug 2013, Aachen, Germany. 2013
Liste complète des métadonnées

Littérature citée [14 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00856877
Contributeur : Bogdan Nicolae <>
Soumis le : lundi 2 septembre 2013 - 16:15:50
Dernière modification le : mardi 3 septembre 2013 - 15:30:53
Document(s) archivé(s) le : mardi 3 décembre 2013 - 11:26:00

Fichier

main.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00856877, version 1

Citation

Bogdan Nicolae. Understanding Vertical Scalability of I/O Virtualization for MapReduce Workloads: Challenges and Opportunities. BigDataCloud'13: 2nd Workshop on Big Data Management in Clouds, Aug 2013, Aachen, Germany. 2013. 〈hal-00856877〉

Partager

Métriques

Consultations de la notice

327

Téléchargements de fichiers

284