CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination

Matthieu Dorier 1, 2 Gabriel Antoniu 1 Robert Ross 3 Dries Kimpe 3 Shadi Ibrahim 1
1 KerData - Scalable Storage for Clouds and Beyond
IRISA-D1 - SYSTÈMES LARGE ÉCHELLE, Inria Rennes – Bretagne Atlantique
3 MCS
ANL - Argonne National Laboratory [Lemont]
Abstract : Unmatched computation and storage performance in new HPC systems have led to a plethora of I/O optimizations ranging from application-side collective I/O to network and disk-level request scheduling on the file system side. As we deal with ever larger machines, the interferences produced by multiple applications accessing a shared parallel file system in a concurrent manner become a major problem. These interferences often break single-application I/O optimizations, dramatically degrading application I/O performance and, as a result, lowering machine wide efficiency. This paper focuses on CALCioM, a framework that aims to mitigate I/O interference through the dynamic selection of appropriate scheduling policies. CALCioM allows several applications running on a supercomputer to communicate and coordinate their I/O strategy in order to avoid interfering with one another. In this work, we examine four I/O strategies that can be accommodated in this framework: serializing, interrupting, interfering and coordinating. Experiments on Argonne's BG/P Surveyor machine and on several clusters of the French Grid'5000 show how CALCioM can be used to efficiently and transparently improve the scheduling strategy between two otherwise interfering applications, given specified metrics of machine wide efficiency.
Document type :
Conference papers
Complete list of metadatas

Cited literature [32 references]  Display  Hide  Download

https://hal.inria.fr/hal-00916091
Contributor : Matthieu Dorier <>
Submitted on : Wednesday, April 9, 2014 - 10:32:06 AM
Last modification on : Thursday, November 29, 2018 - 2:58:00 PM
Long-term archiving on : Wednesday, July 9, 2014 - 10:50:54 AM

File

CALCioM.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00916091, version 1

Citation

Matthieu Dorier, Gabriel Antoniu, Robert Ross, Dries Kimpe, Shadi Ibrahim. CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination. IPDPS - International Parallel and Distributed Processing Symposium, May 2014, Phoenix, United States. ⟨hal-00916091⟩

Share

Metrics

Record views

2015

Files downloads

547