Scheduling the I/O of HPC Applications Under Congestion

Abstract : A significant percentage of the computing capacity of large-scale platforms is wasted because of interferences incurred by multiple applications that access a shared parallel file system concurrently. One solution to handling I/O bursts in large-scale HPC systems is to absorb them at an intermediate storage layer consisting of burst buffers. However, our analysis of the Argonne's Mira system shows that burst buffers cannot prevent congestion at all times. Consequently, I/O performance is dramatically degraded, showing in some cases a decrease in I/O throughput of 67%. In this paper, we analyze the effects of interference on application I/O bandwidth and propose several scheduling techniques to mitigate congestion. We show through extensive experiments that our global I/O scheduler is able to reduce the effects of congestion, even on systems where burst buffers are used, and can increase the overall system throughput up to 56%. We also show that it outperforms current Mira I/O schedulers.
Type de document :
Communication dans un congrès
IEEE International Parallel and Distributed Processing Symposium, IPDPS 2015, Hyderabad, India, May 25-29, 2015, May 2015, Hyderabad, India. IEEE International Parallel and Distributed Processing Symposium, IPDPS 2015, Hyderabad, India, May 25-29, 2015, 2015, 〈10.1109/IPDPS.2015.116〉
Liste complète des métadonnées

Littérature citée [29 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01251938
Contributeur : Equipe Roma <>
Soumis le : jeudi 7 janvier 2016 - 04:01:01
Dernière modification le : vendredi 20 avril 2018 - 15:44:27
Document(s) archivé(s) le : vendredi 8 avril 2016 - 13:08:21

Fichier

online.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Relations

Citation

Ana Gainaru, Guillaume Aupy, Anne Benoit, Franck Cappello, Yves Robert, et al.. Scheduling the I/O of HPC Applications Under Congestion. IEEE International Parallel and Distributed Processing Symposium, IPDPS 2015, Hyderabad, India, May 25-29, 2015, May 2015, Hyderabad, India. IEEE International Parallel and Distributed Processing Symposium, IPDPS 2015, Hyderabad, India, May 25-29, 2015, 2015, 〈10.1109/IPDPS.2015.116〉. 〈hal-01251938〉

Partager

Métriques

Consultations de la notice

1149

Téléchargements de fichiers

176