Evaluation of Data Locality Strategies for Hybrid Cloud Bursting of Iterative MapReduce

Abstract : Hybrid cloud bursting (i.e., leasing temporary off-premise cloud resources to boost the overall capacity during peak utilization) is a popular and cost-effective way to deal with the increasing complexity of big data analytics. It is particularly promising for iterative MapReduce applications that reuse massive amounts of input data at each iteration, which compensates for the high overhead and cost of concurrent data transfers from the on-premise to the off-premise VMs over a weak inter-site link that is of limited capacity. In this paper we study how to combine various MapReduce data locality techniques designed for hybrid cloud bursting in order to achieve scalability for iterative MapReduce applications in a cost-effective fashion. This is a non-trivial problem due to the complex interaction between the data movements over the weak link and the scheduling of computational tasks that have to adapt to the shifting data distribution. We show that using the right combination of techniques, iterative MapReduce applications can scale well in a hybrid cloud bursting scenario and come even close to the scalability observed in single sites.
Complete list of metadatas

Cited literature [15 references]  Display  Hide  Download

https://hal.inria.fr/hal-01469991
Contributor : Bogdan Nicolae <>
Submitted on : Friday, February 17, 2017 - 12:31:59 AM
Last modification on : Friday, February 17, 2017 - 12:03:28 PM
Long-term archiving on : Thursday, May 18, 2017 - 1:02:04 PM

File

paper.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01469991, version 1

Citation

Francisco Clemente-Castello, Bogdan Nicolae, M. Mustafa Rafique, Rafael Mayo, Juan Carlos Fernandez. Evaluation of Data Locality Strategies for Hybrid Cloud Bursting of Iterative MapReduce. CCGrid’17: 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2017, Madrid, Spain. ⟨hal-01469991⟩

Share

Metrics

Record views

105

Files downloads

329