Skip to Main content Skip to Navigation
Conference papers

Bredala: Semantic Data Redistribution for In Situ Applications

Matthieu Dreher 1, * Tom Peterka 1 
Abstract : In situ processing is a promising solution to the problem of imbalance between computational capabilities and I/O bandwidth in current and future supercomputers. Initially designed for staging I/O, in situ middleware now can support a wide range of domains such as visualization, machine learning, filtering, and feature tracking. Doing so requires in situ middleware to manage complex heterogeneous codes using different data structures. Data need to be transformed and reorganized along the data path to fit the analysis needs. However, redistributing complex data structures is difficult. In many cases, arbitrarily splitting the arrays of a data structure destroys the semantic integrity of the data. We present Bredala, a lightweight library to annotate a data model with enough information to preserve the semantic integrity of the data during a redistribution. Bredala allows developers to describe how to split and merge a data model safely, operations usually done by in situ middleware. We evaluate the cost and performance of our library in a molecular dynamics application. We show that our data model can simplify the workflow graph of large-scale applications, improve the reusability of tasks, and offer an efficient alternative to redistribute the data.
Complete list of metadata

Cited literature [28 references]  Display  Hide  Download
Contributor : Matthieu Dreher Connect in order to contact the contributor
Submitted on : Wednesday, August 31, 2016 - 10:56:37 PM
Last modification on : Tuesday, March 6, 2018 - 2:30:19 PM


Files produced by the author(s)


  • HAL Id : hal-01358482, version 1


Matthieu Dreher, Tom Peterka. Bredala: Semantic Data Redistribution for In Situ Applications. IEEE Cluster 2016, IEEE, Sep 2016, Taipei, Taiwan. ⟨hal-01358482⟩



Record views


Files downloads