Skip to Main content Skip to Navigation
Reports

An elastic framework for ensemble-based large-scale data assimilation

Sebastian Friedemann 1 Bruno Raffin 1
1 DATAMOVE - Data Aware Large Scale Computing
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
Abstract : Prediction of chaotic systems relies on a floating fusion of sensor data (observations) with a numerical model to decide on a good system trajectory and to compensate nonlinear feedback effects. Ensemble-based data assimilation (DA) is a major method for this concern depending on propagating an ensemble of perturbed model realizations. In this paper we develop an elastic, online, fault-tolerant and modular framework called Melissa-DA for large-scale ensemble-based DA. Melissa-DA allows elastic addition or removal of compute resources for state propagation at runtime. Dynamic load balancing based on list scheduling ensures efficient execution. Online processing of the data produced by ensemble members enables to avoid the I/O bottleneck of file-based approaches. Our implementation embeds the PDAF parallel DA engine, enabling the use of various DA methods. Melissa-DA can support extra ensemble-based DA methods by implementing the transformation of member background states into analysis states. Experiments confirm the excellent scalability of Melissa-DA, running on up to 16,240 cores, to propagate 16,384 members for a regional hydrological critical zone assimilation relying on the ParFlow model on a domain with about 4 M grid cells.
Complete list of metadatas

https://hal.inria.fr/hal-03017033
Contributor : Sebastian Friedemann <>
Submitted on : Tuesday, November 24, 2020 - 3:19:06 PM
Last modification on : Thursday, November 26, 2020 - 3:30:49 AM

Files

RR-9377.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03017033, version 2
  • ARXIV : 2011.11635

Citation

Sebastian Friedemann, Bruno Raffin. An elastic framework for ensemble-based large-scale data assimilation. [Research Report] RR-9377, Inria Grenoble Rhône-Alpes, Université de Grenoble. 2020. ⟨hal-03017033v2⟩

Share

Metrics

Record views

14

Files downloads

26