Mapping tree‐shaped workflows on systems with different memory sizes and processor speeds - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Concurrency and Computation: Practice and Experience Year : 2023

Mapping tree‐shaped workflows on systems with different memory sizes and processor speeds

Abstract

Directed acyclic graphs are commonly used to model scientific workflows, by expressing dependencies between tasks, as well as the resource requirements of the workflow. As a special case, rooted directed trees occur in several applications, for instance in sparse matrix computations. Since typical workflows are modeled by large trees, it is crucial to schedule them efficiently, so that their execution time (or makespan) is minimized. Furthermore, it is usually beneficial to distribute the execution on several compute nodes, hence increasing the available memory, and allowing us to parallelize parts of the execution. To exploit the heterogeneity of modern clusters in this context, we investigate the partitioning and mapping of tree‐shaped workflows on two types of target architecture models: in AM1, each processor can have a different memory size, and in AM2, each processor can also have a different speed (in addition to a different memory size). We design a three‐step heuristic for AM1, which adapts and extends previous work for homogeneous clusters [Gou C, Benoit A, Marchal L. Partitioning tree‐shaped task graphs for distributed platforms with limited memory. IEEE Trans Parallel Dist Syst 2020; 31(7): 1533–1544]. The changes we propose concern the assignment to processors (accounting for the different memory sizes) and the availability of suitable processors when splitting or merging subtrees. For AM2, we extend the heuristic for AM1 with a two‐phase local search approach. Phase A is a swap‐based hill climber, while (the optional) Phase B is inspired by iterated local search. We evaluate our heuristics for AM1 and AM2 with extensive simulations, and we demonstrate that exploiting the heterogeneity in the cluster significantly reduces the makespan, compared to the state of the art for homogeneous processors.
Fichier principal
Vignette du fichier
CCPE-2023.pdf (1.78 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04397633 , version 1 (16-01-2024)

Licence

Attribution

Identifiers

Cite

Svetlana Kulagina, Henning Meyerhenke, Anne Benoit. Mapping tree‐shaped workflows on systems with different memory sizes and processor speeds. Concurrency and Computation: Practice and Experience, 2023, 35 (25), ⟨10.1002/cpe.7842⟩. ⟨hal-04397633⟩
6 View
2 Download

Altmetric

Share

Gmail Facebook X LinkedIn More