Skip to Main content Skip to Navigation
New interface
Conference papers

CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination

Matthieu Dorier 1, 2 Gabriel Antoniu 1 Robert Ross 3 Dries Kimpe 3 Shadi Ibrahim 1 
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
ANL - Argonne National Laboratory [Lemont]
Abstract : Unmatched computation and storage performance in new HPC systems have led to a plethora of I/O optimizations ranging from application-side collective I/O to network and disk-level request scheduling on the file system side. As we deal with ever larger machines, the interferences produced by multiple applications accessing a shared parallel file system in a concurrent manner become a major problem. These interferences often break single-application I/O optimizations, dramatically degrading application I/O performance and, as a result, lowering machine wide efficiency. This paper focuses on CALCioM, a framework that aims to mitigate I/O interference through the dynamic selection of appropriate scheduling policies. CALCioM allows several applications running on a supercomputer to communicate and coordinate their I/O strategy in order to avoid interfering with one another. In this work, we examine four I/O strategies that can be accommodated in this framework: serializing, interrupting, interfering and coordinating. Experiments on Argonne's BG/P Surveyor machine and on several clusters of the French Grid'5000 show how CALCioM can be used to efficiently and transparently improve the scheduling strategy between two otherwise interfering applications, given specified metrics of machine wide efficiency.
Document type :
Conference papers
Complete list of metadata

Cited literature [32 references]  Display  Hide  Download
Contributor : Matthieu Dorier Connect in order to contact the contributor
Submitted on : Wednesday, April 9, 2014 - 10:32:06 AM
Last modification on : Saturday, November 19, 2022 - 3:43:20 AM
Long-term archiving on: : Wednesday, July 9, 2014 - 10:50:54 AM


Files produced by the author(s)


  • HAL Id : hal-00916091, version 1


Matthieu Dorier, Gabriel Antoniu, Robert Ross, Dries Kimpe, Shadi Ibrahim. CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination. IPDPS - International Parallel and Distributed Processing Symposium, May 2014, Phoenix, United States. ⟨hal-00916091⟩



Record views


Files downloads