StarPU-MPI: Task Programming over Clusters of Machines Enhanced with Accelerators

Cédric Augonnet 1 Olivier Aumage 1 Nathalie Furmento 1, 2 Samuel Thibault 1 Raymond Namyst 1
1 RUNTIME - Efficient runtime systems for parallel architectures
Inria Bordeaux - Sud-Ouest, UB - Université de Bordeaux, CNRS - Centre National de la Recherche Scientifique : UMR5800
Abstract : GPUs have largely entered HPC clusters, as shown by the top entries of the latest top500 issue. Exploiting such machines is however very challenging, not only because of combining two separate paradigms, MPI and CUDA or OpenCL, but also because nodes are heterogeneous and thus require careful load balancing within nodes themselves. The current paradigms are usually limited to only offloading parts of the computation and leaving CPUs idle, or they require static work partitioning between CPUs and GPUs. To handle single-node architecture heterogeneity, we have previously proposed StarPU, a runtime system capable of dynamically scheduling tasks in an optimized way on such machines. We show here how the task paradigm of StarPU has been combined with MPI communications, and how we extended the task paradigm itself to allow mapping the task graph on MPI clusters such as to automatically achieve an optimized distributed execution. We show how a sequential-like Cholesky source code can be easily extended into a scalable distributed parallel execution, and already exhibits a speedup of 5 on 6 nodes.
Complete list of metadatas

Cited literature [28 references]  Display  Hide  Download

https://hal.inria.fr/hal-00992208
Contributor : Nathalie Furmento <>
Submitted on : Monday, May 26, 2014 - 11:09:08 AM
Last modification on : Tuesday, May 14, 2019 - 11:38:08 AM
Long-term archiving on : Tuesday, August 26, 2014 - 10:51:07 AM

File

RR-8538.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00992208, version 2

Citation

Cédric Augonnet, Olivier Aumage, Nathalie Furmento, Samuel Thibault, Raymond Namyst. StarPU-MPI: Task Programming over Clusters of Machines Enhanced with Accelerators. [Research Report] RR-8538, INRIA. 2014. ⟨hal-00992208v2⟩

Share

Metrics

Record views

855

Files downloads

853