Skip to Main content Skip to Navigation
Conference papers

Bredala: Semantic Data Redistribution for In Situ Applications

Abstract : In situ processing is a promising solution to the problem of imbalance between computational capabilities and I/O bandwidth in current and future supercomputers. Initially designed for staging I/O, in situ middleware now can support a wide range of domains such as visualization, machine learning, filtering, and feature tracking. Doing so requires in situ middleware to manage complex heterogeneous codes using different data structures. Data need to be transformed and reorganized along the data path to fit the analysis needs. However, redistributing complex data structures is difficult. In many cases, arbitrarily splitting the arrays of a data structure destroys the semantic integrity of the data. We present Bredala, a lightweight library to annotate a data model with enough information to preserve the semantic integrity of the data during a redistribution. Bredala allows developers to describe how to split and merge a data model safely, operations usually done by in situ middleware. We evaluate the cost and performance of our library in a molecular dynamics application. We show that our data model can simplify the workflow graph of large-scale applications, improve the reusability of tasks, and offer an efficient alternative to redistribute the data.
Complete list of metadata

Cited literature [28 references]  Display  Hide  Download

https://hal.inria.fr/hal-01358482
Contributor : Matthieu Dreher <>
Submitted on : Wednesday, August 31, 2016 - 10:56:37 PM
Last modification on : Tuesday, March 6, 2018 - 2:30:19 PM

File

bredala_final.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01358482, version 1

Citation

Matthieu Dreher, Tom Peterka. Bredala: Semantic Data Redistribution for In Situ Applications. IEEE Cluster 2016, IEEE, Sep 2016, Taipei, Taiwan. ⟨hal-01358482⟩

Share

Metrics

Record views

97

Files downloads

172