What Size Should your Buffers to Disks be?

Guillaume Aupy 1 Olivier Beaumont 2 Lionel Eyraud-Dubois 2
1 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
2 Realopt - Reformulations based algorithms for Combinatorial Optimization
LaBRI - Laboratoire Bordelais de Recherche en Informatique, IMB - Institut de Mathématiques de Bordeaux, Inria Bordeaux - Sud-Ouest
Abstract : Burst-Buffers are high throughput, small size intermediate storage systems typically based on SSDs or NVRAM that are designed to be used as a potential buffer between the computing nodes of a supercomputer and its main storage system consisting of hard drives. Their purpose is to absorb the bursts of I/O that many HPC applications experience (for example for saving checkpoints or data from intermediate results). In this paper, we propose a probabilistic model for evaluating the performance of Burst-Buffers. From a model of application and a data management strategy, we build a Markov chain based model of the system, that allows to quickly answer issues about dimensioning of the system: for a given set of applications, and for a given Burst-Buffer size and bandwidth, how often does the buffer overflow? We also provide extensive simulation results to validate our modeling approach.
Complete list of metadatas

Cited literature [23 references]  Display  Hide  Download

https://hal.inria.fr/hal-01623846
Contributor : Guillaume Pallez (aupy) <>
Submitted on : Friday, July 13, 2018 - 12:28:32 PM
Last modification on : Thursday, May 16, 2019 - 3:40:34 PM
Long-term archiving on : Monday, October 15, 2018 - 10:27:12 AM

File

main-v2.pdf
Files produced by the author(s)

Identifiers

Citation

Guillaume Aupy, Olivier Beaumont, Lionel Eyraud-Dubois. What Size Should your Buffers to Disks be?. International Parallel and Distributed Processing Symposium (IPDPS), May 2018, Vancouver, Canada. ⟨10.1109/IPDPS.2018.00075⟩. ⟨hal-01623846v2⟩

Share

Metrics

Record views

160

Files downloads

374