Cache-Efficient, Intranode, Large-Message MPI Communication with MPICH2-Nemesis

Darius Buntinas 1 Brice Goglin 2, 3 David Goodell 1 Guillaume Mercier 2, 3, 4 Stéphanie Moreaud 2, 3
1 MCS - Mathematics and Computer Science Division [ANL]
2 LaBRI - Laboratoire Bordelais de Recherche en Informatique
3 RUNTIME - Efficient runtime systems for parallel architectures
Inria Bordeaux - Sud-Ouest, UB - Université de Bordeaux, CNRS - Centre National de la Recherche Scientifique : UMR5800
4 ENSEIRB - Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux
Abstract : The emergence of multicore processors raises the need to efficiently transfer large amounts of data between local processes. MPICH2 is a highly portable MPI implementation whose large-message communication schemes suffer from high CPU utilization and cache pollution because of the use of a double-buffering strategy, common to many MPI implementations. We introduce two strategies offering a kernel-assisted, single-copy model with support for noncontiguous and asynchronous transfers. The first one uses the now widely available vmsplice Linux system call; the second one further improves performance thanks to a custom kernel module called KNEM. The latter also offers I/OAT copy offload, which is dynamically enabled depending on both hardware cache characteristics and message size. These new solutions outperform the standard transfer method in the MPICH2 implementation when no cache is shared between the processing cores or when very large messages are being transferred. Collective communication operations show a dramatic improvement, and the IS NAS parallel benchmark shows a 25% speedup and better cache efficiency.
Type de document :
Communication dans un congrès
IEEE. 38th International Conference on Parallel Processing (ICPP-2009), Sep 2009, Vienne, Austria. 2009, <10.1109/ICPP.2009.22>
Liste complète des métadonnées

https://hal.inria.fr/inria-00390064
Contributeur : Brice Goglin <>
Soumis le : vendredi 5 juin 2009 - 10:23:11
Dernière modification le : jeudi 10 septembre 2015 - 01:06:34
Document(s) archivé(s) le : lundi 15 octobre 2012 - 11:27:37

Fichier

article.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

INRIA | LABRI | UNIV-BORDEAUX

Citation

Darius Buntinas, Brice Goglin, David Goodell, Guillaume Mercier, Stéphanie Moreaud. Cache-Efficient, Intranode, Large-Message MPI Communication with MPICH2-Nemesis. IEEE. 38th International Conference on Parallel Processing (ICPP-2009), Sep 2009, Vienne, Austria. 2009, <10.1109/ICPP.2009.22>. <inria-00390064>

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de
la notice

388

Téléchargements du document

172

Contact


archive-ouverte@inria.fr