What Size Should your Buffers to Disks be?

Guillaume Aupy 1 Olivier Beaumont 2 Lionel Eyraud-Dubois 2
1 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
2 Realopt - Reformulations based algorithms for Combinatorial Optimization
LaBRI - Laboratoire Bordelais de Recherche en Informatique, IMB - Institut de Mathématiques de Bordeaux, Inria Bordeaux - Sud-Ouest
Abstract : Burst-Buffers are high throughput, small size intermediate storage systems typically based on SSDs or NVRAM that are designed to be used as a potential buffer between the computing nodes of a supercomputer and its main storage system consisting of hard drives. Their purpose is to absorb the bursts of I/O that many HPC applications experience (for example for saving checkpoints or data from intermediate results). In this paper, we propose a probabilistic model for evaluating the performance of Burst-Buffers. From a model of application and a data management strategy, we build a Markov chain based model of the system, that allows to quickly answer issues about dimensioning of the system: for a given set of applications, and for a given Burst-Buffer size and bandwidth, how often does the buffer overflow? We also provide extensive simulation results to validate our modeling approach.
Type de document :
Communication dans un congrès
IEEE. International Parallel and Distributed Processing Symposium (IPDPS), May 2018, Vancouver, Canada. 2018
Liste complète des métadonnées

Littérature citée [10 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01623846
Contributeur : Guillaume Aupy <>
Soumis le : vendredi 13 juillet 2018 - 12:28:32
Dernière modification le : samedi 14 juillet 2018 - 01:21:27

Fichier

main-v2.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01623846, version 2

Collections

Citation

Guillaume Aupy, Olivier Beaumont, Lionel Eyraud-Dubois. What Size Should your Buffers to Disks be?. IEEE. International Parallel and Distributed Processing Symposium (IPDPS), May 2018, Vancouver, Canada. 2018. 〈hal-01623846v2〉

Partager

Métriques

Consultations de la notice

72

Téléchargements de fichiers

4