Testing MapReduce-Based Systems

João Eugenio Marynowski 1 Michel Albonico 1 Eduardo Cunha de Almeida 1 Gerson Sunyé 2
2 TRISKELL - Reliable and efficient component based software engineering
IRISA - Institut de Recherche en Informatique et Systèmes Aléatoires, Inria Rennes – Bretagne Atlantique
Abstract : MapReduce (MR) is the most popular solution to build applications for large-scale data processing. These applications are often deployed on large clusters of commodity machines, where failures happen constantly due to bugs, hardware problems, and outages. Testing MR-based systems is hard, since it is needed a great eort of test harness to execute distributed test cases upon failures. In this paper, we present a novel testing solution to tackle this issue called HadoopTest. This solution is based on a scalable harness approach, where distributed tester components are hung around each map and reduce worker (i.e., node). Testers are allowed to stimulate each worker to inject failures on them, monitor their behavior, and validate testing results. HadoopTest was used to test two applications bundled into Hadoop, the Apache open source MapReduce implementation. Our initial implementation demonstrates promising results, with HadoopTest coordinating test cases across distributed MapReduce workers, and finding bugs.
Type de document :
Communication dans un congrès
SBBD - XXVI Simpósio Brasileiro de Banco de Dados, Oct 2011, Florianopolis, SC, Brazil. 2011
Liste complète des métadonnées

https://hal.inria.fr/hal-00728729
Contributeur : João Eugenio Marynowski <>
Soumis le : jeudi 6 septembre 2012 - 14:55:46
Dernière modification le : mercredi 11 avril 2018 - 02:01:04

Identifiants

  • HAL Id : hal-00728729, version 1

Citation

João Eugenio Marynowski, Michel Albonico, Eduardo Cunha de Almeida, Gerson Sunyé. Testing MapReduce-Based Systems. SBBD - XXVI Simpósio Brasileiro de Banco de Dados, Oct 2011, Florianopolis, SC, Brazil. 2011. 〈hal-00728729〉

Partager

Métriques

Consultations de la notice

232