Skip to Main content Skip to Navigation
Reports

Periodic I/O scheduling for super-computers

Abstract : With the ever-growing need of data in HPC applications, the congestion at theI/O level becomes critical in super-computers. Architectural enhancement such asburst-buffers and pre-fetching are added to machines, but are not sufficient toprevent congestion. Recent online I/O scheduling strategies have been put inplace, but they add an additional congestion point and overheads in thecomputation of applications. In this work, we show how to take advantage of the periodic nature of HPCapplications in order to develop efficient periodic scheduling strategiesfor their I/O transfers. Our strategy computes once during the job scheduling phase a pattern where itdefines the I/O behavior for each application, after which the applications runindependently, transferring their I/O at the specified times. Our strategy limitsthe amount of I/O congestion at the I/O node level and can be easily integratedinto current job schedulers. We validate this model through extensive simulationsand experiments by comparing it to state-of-the-art online solutions, showing thatnot only our scheduler has the advantage of being de-centralized and thus overcoming theoverhead of online schedulers, but also that it performs better than thesesolutions, improving the application dilation up to 13% and the maximumsystem efficiency up to 18%.
Complete list of metadatas

Cited literature [31 references]  Display  Hide  Download

https://hal.inria.fr/hal-01474553
Contributor : Guillaume Pallez (aupy) <>
Submitted on : Monday, March 6, 2017 - 1:12:09 PM
Last modification on : Wednesday, November 20, 2019 - 3:03:47 AM
Long-term archiving on: : Wednesday, June 7, 2017 - 2:04:13 PM

File

RR-9037.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01474553, version 2

Citation

Guillaume Aupy, Ana Gainaru, Valentin Le Fèvre. Periodic I/O scheduling for super-computers. [Research Report] RR-9037, Inria Bordeaux Sud-Ouest. 2017. ⟨hal-01474553v2⟩

Share

Metrics

Record views

574

Files downloads

720