Skip to Main content Skip to Navigation
Conference papers

Eley: On the Effectiveness of Burst Buffers for Big Data Processing in HPC systems

Orcun Yildiz 1 Amelie Chi Zhou 2 Shadi Ibrahim 2, 3
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
2 ASCOLA - Aspect and Composition Languages
Inria Rennes – Bretagne Atlantique , LS2N - Laboratoire des Sciences du Numérique de Nantes
Abstract : Burst Buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the performance guarantees of HPC applications – which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the performance of HPC applications. In order to improve the performance of Big Data applications, Eley employs a prefetching technique that fetches the input data of these applications to be stored close to computing nodes thus reducing the latency of reading data inputs. Moreover, Eley is equipped with a full delay operator to guarantee the performance of HPC applications – as they are running independently on a HPC system. The experimental results show the effectiveness of Eley in obtaining shorter execution time of Big Data applications (shorter map phase) while guaranteeing the performance of HPC applications.
Complete list of metadatas

Cited literature [16 references]  Display  Hide  Download

https://hal.inria.fr/hal-01570737
Contributor : Shadi Ibrahim <>
Submitted on : Monday, July 31, 2017 - 3:48:35 PM
Last modification on : Friday, September 18, 2020 - 10:41:42 AM

File

Cluster2017-CR.pdf
Files produced by the author(s)

Identifiers

Citation

Orcun Yildiz, Amelie Chi Zhou, Shadi Ibrahim. Eley: On the Effectiveness of Burst Buffers for Big Data Processing in HPC systems. Cluster'17-2017 IEEE International Conference on Cluster Computing, Sep 2017, Honolulu, United States. ⟨10.1109/CLUSTER.2017.73⟩. ⟨hal-01570737⟩

Share

Metrics

Record views

2646

Files downloads

524