Dynamic Provisioning of Storage Resources: A Case Study with Burst Buffers - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Dynamic Provisioning of Storage Resources: A Case Study with Burst Buffers

Résumé

Complex applications and workflows needs are often exclusively expressed in terms of computational resources on HPC systems. In many cases, other resources like storage or network are not allocatable and are shared across the entire HPC system. By looking at the storage resources in particular, any workflow or application should be able to select both its preferred data manager and its required storage capability or capacity. To achieve such a goal, new mechanisms should be introduced. In this work, we present such a tool that dynamically provision a data management system on top of storage devices. We propose a proof-of-concept that is able to deploy, on-demand, a parallel file-system across intermediate storage nodes on a Cray XC50 system. We show how this mechanism can be easily extended to support more data managers and any type of intermediate storage. Finally, we evaluate the performance of the provisioned storage system with a set of benchmarks.
Fichier principal
Vignette du fichier
HPS20__Dynamic_Provisioning_of_Storage_Resources__A_Case_Study_with_Burst_Buffers.pdf (476.51 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02491413 , version 1 (26-02-2020)

Identifiants

  • HAL Id : hal-02491413 , version 1

Citer

François Tessier, Maxime Martinasso, Matteo Chesi, Mark Klein, Miguel Gila. Dynamic Provisioning of Storage Resources: A Case Study with Burst Buffers. IPDPSW 2020 - IEEE International Parallel and Distributed Processing Symposium Workshops, May 2020, New Orleans, United States. ⟨hal-02491413⟩
120 Consultations
300 Téléchargements

Partager

Gmail Facebook X LinkedIn More