Exploiting Concurrent GPU Operations for Efficient Work Stealing on Multi-GPUs

Abstract : The race for Exascale computing has naturally led the current technologies to converge to multi-CPU/multi-GPU computers, based on thousands of CPUs and GPUs interconnected by PCI-Express buses or interconnection networks. To exploit this high computing power, programmers have to solve the issue of scheduling parallel programs on hybrid architectures. And, since the performance of a GPU increases at a much faster rate than the throughput of a PCI bus, data transfers must be managed efficiently by the scheduler. This paper targets multi-GPU compute nodes, where several GPUs are connected to the same machine. To overcome the data transfer limitations on such platforms, the available softwares compute, usually before the execution, a mapping of the tasks that respects their dependencies and minimizes the global data transfers. Such an approach is too rigid and it cannot adapt the execution to possible variations of the system or to the application's load. We propose a solution that is orthogonal to the above mentioned: extensions of the XKaapi software stack that enable to exploit full performance of a multi-GPUs system through asynchronous GPU tasks. XKaapi schedules tasks by using a standard Work Stealing algorithm and the runtime efficiently exploits concurrent GPU operations. The runtime extensions make it possible to overlap the data transfers and the task executions on current generation of GPUs. We demonstrate that the overlapping capability is at least as important as computing a scheduling decision to reduce completion time of a parallel program. Our experiments on two dense linear algebra problems (Matrix Product and Cholesky factorization) show that our solution is highly competitive with other softwares based on static scheduling. Moreover, we are able to sustain the peak performance (310 GFlop/s) on DGEMM, even for matrices that cannot be stored entirely in one GPU memory. With eight GPUs, we archive a speed-up of 6.74 with respect to single-GPU. The performance of our Cholesky factorization, with more complex dependencies between tasks, outperforms the state of the art single-GPU MAGMA code.
Type de document :
Communication dans un congrès
24rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), Oct 2012, Columbia University, New York, United States. pp.75-82, 2012, 〈10.1109/SBAC-PAD.2012.28〉
Liste complète des métadonnées

Littérature citée [16 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00735470
Contributeur : Joao Vicente Ferreira Lima <>
Soumis le : mardi 25 septembre 2012 - 17:12:57
Dernière modification le : jeudi 11 janvier 2018 - 06:22:01
Document(s) archivé(s) le : vendredi 16 décembre 2016 - 16:12:52

Fichier

sbac-pad2012.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Joao Vicente Ferreira Lima, Thierry Gautier, Nicolas Maillard, Vincent Danjean. Exploiting Concurrent GPU Operations for Efficient Work Stealing on Multi-GPUs. 24rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), Oct 2012, Columbia University, New York, United States. pp.75-82, 2012, 〈10.1109/SBAC-PAD.2012.28〉. 〈hal-00735470〉

Partager

Métriques

Consultations de la notice

1026

Téléchargements de fichiers

431