Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers

Abstract : Reading and writing data efficiently from storage systems is critical for high performance data-centric applications. These I/O systems are being increasingly characterized by complex topologies and deeper memory hierarchies. Effective parallel I/O solutions are needed to scale applications on current and future supercomputers. Data aggregation is an efficient approach consisting of electing some processes in charge of aggregating data from a set of neighbors and writing the aggregated data into storage. Thus, the bandwidth use can be optimized while the contention is reduced. In this work, we take into account the network topology for mapping aggregators and we propose an optimized buffering system in order to reduce the aggregation cost. We validate our approach using micro-benchmarks and the I/O kernel of a large-scale cosmology simulation. We show improvements up to 15× faster for I/O operations compared to a standard implementation of MPI I/O.
Type de document :
Communication dans un congrès
1st Workshop on Optimization of Communication in HPC runtime systems (IEEE COM-HPC16), Nov 2016, Salt-Lake City, United States. IEEE, pp.9
Liste complète des métadonnées

Littérature citée [18 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01394741
Contributeur : Emmanuel Jeannot <>
Soumis le : lundi 14 novembre 2016 - 11:40:16
Dernière modification le : mardi 24 octobre 2017 - 14:52:01
Document(s) archivé(s) le : mercredi 15 mars 2017 - 03:52:47

Fichier

topoIO-paper.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01394741, version 1

Collections

Citation

François Tessier, Preeti Malakar, Venkatram Vishwanath, Emmanuel Jeannot, Florin Isaila. Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers. 1st Workshop on Optimization of Communication in HPC runtime systems (IEEE COM-HPC16), Nov 2016, Salt-Lake City, United States. IEEE, pp.9. 〈hal-01394741〉

Partager

Métriques

Consultations de la notice

242

Téléchargements de fichiers

118