Modeling High-throughput Applications for in situ Analytics - Archive ouverte HAL Access content directly
Journal Articles International Journal of High Performance Computing Applications Year : 2019

Modeling High-throughput Applications for in situ Analytics

(1) , (1) , (1) , (2)
1
2

Abstract

With the goal of performing exascale computing, the importance of I/Omanagement becomes more and more critical to maintain system performance.While the computing capacities of machines are getting higher, the I/O capa-bilities of systems do not increase as fast. We are able to generate more databut unable to manage them eciently due to variability of I/O performance.Limiting the requests to the Parallel File System (PFS) becomes necessary. Toaddress this issue, new strategies are being developed such as online in situanalysis. The idea is to overcome the limitations of basic post-mortem dataanalysis where the data have to be stored on PFS rst and processed later.There are several software solutions that allow users to specically dedicatenodes for analysis of data and distribute the computation tasks over dier-ent sets of nodes. Thus far, they rely on a manual resource partitioning andallocation by the user of tasks (simulations, analysis).In this work, we propose a memory-constraint modelization for in situ anal-ysis. We use this model to provide dierent scheduling policies to determineboth the number of resources that should be dedicated to analysis functions,and that schedule eciently these functions. We evaluate them and show theimportance of considering memory constraints in the model. Finally, we discussthe dierent challenges that have to be addressed in order to build automatictools for in situ analytics.
Fichier principal
Vignette du fichier
paper.pdf (525.55 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02091340 , version 1 (05-04-2019)

Identifiers

Cite

Guillaume Aupy, Brice Goglin, Valentin Honoré, Bruno Raffin. Modeling High-throughput Applications for in situ Analytics. International Journal of High Performance Computing Applications, inPress, 33 (6), pp.1185-1200. ⟨10.1177/1094342019847263⟩. ⟨hal-02091340⟩
236 View
332 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More