Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files

Abstract : Global sensitivity analysis is an important step for analyzing and validating numerical simulations. One classical approach consists in computing statistics on the outputs from well-chosen multiple simulation runs. Simulation results are stored to disk and statistics are computed postmortem. Even if supercomputers enable to run large studies, scientists are constrained to run low resolution simulations with a limited number of probes to keep the amount of intermediate storage manageable. In this paper we propose a file avoiding, adaptive, fault tolerant and elastic framework that enables high resolution global sensitivity analysis at large scale. Our approach combines iterative statistics and in transit processing to compute Sobol' indices without any intermediate storage. Statistics are updated on-the-fly as soon as the in transit parallel server receives results from one of the running simulations. For one experiment, we computed the Sobol' indices on 10M hexahedra and 100 timesteps, running 8000 parallel simulations executed in 1h27 on up to 28672 cores, avoiding 48TB of file storage.
Type de document :
Communication dans un congrès
The International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing), Nov 2017, Denver, United States. pp.1 - 14
Liste complète des métadonnées

Littérature citée [49 références]  Voir  Masquer  Télécharger


https://hal.inria.fr/hal-01607479
Contributeur : Bruno Raffin <>
Soumis le : lundi 2 octobre 2017 - 23:55:35
Dernière modification le : jeudi 18 janvier 2018 - 10:39:12

Fichiers

main-Sobol-SC-2017-HALVERSION....
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01607479, version 1

Citation

Théophile Terraz, Alejandro Ribes, Yvan Fournier, Bertrand Iooss, Bruno Raffin. Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files. The International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing), Nov 2017, Denver, United States. pp.1 - 14. 〈hal-01607479〉

Partager

Métriques

Consultations de la notice

190

Téléchargements de fichiers

82