Resilin: Elastic MapReduce over Multiple Clouds

Anca Iordache 1 Christine Morin 1 Nikos Parlavantzas 1 Pierre Riteau 1
1 MYRIADS - Design and Implementation of Autonomous Distributed Systems
IRISA-D1 - SYSTÈMES LARGE ÉCHELLE, Inria Rennes – Bretagne Atlantique
Abstract : The MapReduce programming model, introduced by Google, offers a simple and efficient way of performing distributed computation over large data sets. Although Google's implementation is proprietary, MapReduce can be leveraged by anyone using the free and open-source Apache Hadoop framework. To simplify the usage of Hadoop in the cloud, Amazon Web Services offers Elastic MapReduce, a web service enabling users to run MapReduce jobs. Elastic MapReduce takes care of resource provisioning, Hadoop configuration and performance tuning, data staging, fault tolerance, etc. This service drastically reduces the entry barrier to perform MapReduce computations in the cloud, allowing users to concentrate on the problem to solve. However, Elastic MapReduce is restricted to Amazon EC2 resources, and is provided at an additional cost. In this paper, we present Resilin, a system implementing the Elastic MapReduce API with resources from clouds other than Amazon EC2, such as private and scientific clouds. Furthermore, we explore a feature going beyond the current Amazon Elastic MapReduce offering: performing MapReduce computations over multiple distributed clouds. The evaluation of Resilin shows the benefits of running computations on more than one cloud. While not being the most efficient way to perform Hadoop computations, it solves the problem of resource availability and adds more flexibility regarding the type/price of resource.
Complete list of metadatas

Cited literature [23 references]  Display  Hide  Download

https://hal.inria.fr/hal-00737030
Contributor : Anca Iordache <>
Submitted on : Monday, October 1, 2012 - 10:30:29 AM
Last modification on : Friday, November 16, 2018 - 1:39:19 AM
Long-term archiving on : Friday, December 16, 2016 - 6:15:13 PM

File

RR-8081.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00737030, version 1

Citation

Anca Iordache, Christine Morin, Nikos Parlavantzas, Pierre Riteau. Resilin: Elastic MapReduce over Multiple Clouds. [Research Report] RR-8081, INRIA. 2012. ⟨hal-00737030⟩

Share

Metrics

Record views

895

Files downloads

994