Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files

Abstract : Global sensitivity analysis is an important step for analyzing and validating numerical simulations. One classical approach consists in computing statistics on the outputs from well-chosen multiple simulation runs. Simulation results are stored to disk and statistics are computed postmortem. Even if supercomputers enable to run large studies, scientists are constrained to run low resolution simulations with a limited number of probes to keep the amount of intermediate storage manageable. In this paper we propose a file avoiding, adaptive, fault tolerant and elastic framework that enables high resolution global sensitivity analysis at large scale. Our approach combines iterative statistics and in transit processing to compute Sobol' indices without any intermediate storage. Statistics are updated on-the-fly as soon as the in transit parallel server receives results from one of the running simulations. For one experiment, we computed the Sobol' indices on 10M hexahedra and 100 timesteps, running 8000 parallel simulations executed in 1h27 on up to 28672 cores, avoiding 48TB of file storage.
Liste complète des métadonnées

Cited literature [46 references]  Display  Hide  Download


https://hal.inria.fr/hal-01607479
Contributor : Bruno Raffin <>
Submitted on : Monday, October 2, 2017 - 11:55:35 PM
Last modification on : Friday, April 12, 2019 - 4:22:51 PM

Files

main-Sobol-SC-2017-HALVERSION....
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01607479, version 1

Citation

Théophile Terraz, Alejandro Ribes, Yvan Fournier, Bertrand Iooss, Bruno Raffin. Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files. The International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing), Nov 2017, Denver, United States. pp.1 - 14. ⟨hal-01607479⟩

Share

Metrics

Record views

1067

Files downloads

506