HAL - INRIA :: [inria-00331209, version 1] High Throughput Intra-Node MPI Communication with Open-MX

inria-00331209, version 1

%% inria-00331209, version 1
%% http://hal.inria.fr/inria-00331209/en/
@inproceedings{GOGLIN:2009:INRIA-00331209:1,
	title = { {H}igh {T}hroughput {I}ntra-{N}ode {MPI} {C}ommunication with {O}pen-{MX}},
	author = {{G}oglin, {B}rice},
	abstract = {{T}he increasing number of cores per node in high-performance computing requires an efficient intra-node {MPI} communication subsystem. {M}ost existing {MPI} implementations rely on two copies across a shared memory-mapped file. {O}pen-{MX} offers a single-copy mechanism that is tightly integrated in its regular communication stack, making it transparently available to the {MX} backend of many {MPI} layers. {W}e describe this implementation and its offloaded copy backend using {I}/{OAT} hardware. {M}emory pinning requirements are then discussed, and overlapped pinning is introduced to enable the start of {O}pen-{MX} intra-node data transfer earlier. {P}erformance evaluation shows that this local communication stack performs better than {MPICH}2 and {O}pen~{MPI} for large messages, reaching up to 70\,\% better throughput in micro-benchmarks when using {I}/{OAT} copy offload. {T}hanks to a single-copy being involved, the {O}pen-{MX} intra-node communication throughput also does not heavily depend on cache sharing between processing cores, making these performance improvements easier to observe in real applications.},
	language = {{A}nglais},
	affiliation = {{L}aboratoire {B}ordelais de {R}echerche en {I}nformatique - {L}a{BRI} - {CNRS} : {UMR}5800 - {U}niversit{\'e} {S}ciences et {T}echnologies - {B}ordeaux {I} - {E}cole {N}ationale {S}up{\'e}rieure d'{E}lectronique, {I}nformatique et {R}adiocommunications de {B}ordeaux - {U}niversit{\'e} {V}ictor {S}egalen - {B}ordeaux {II} - {RUNTIME} - {INRIA} {B}ordeaux - {S}ud-{O}uest - {INRIA} - {CNRS} : {UMR}5800 - {U}niversit{\'e} {S}ciences et {T}echnologies - {B}ordeaux {I} - {E}cole {N}ationale {S}up{\'e}rieure d'{E}lectronique, {I}nformatique et {R}adiocommunications de {B}ordeaux },
	booktitle = {17th {E}uromicro {I}nternational {C}onference on {P}arallel, {D}istributed and {N}etwork-{B}ased {P}rocessing ({PDP}2009) },
	address = {{W}eimar {A}llemagne },
	editor = {{IEEE} },
	audience = {internationale },
	doi = {10.1109/{PDP}.2009.20 },
	year = {2009},
	URL = {dx.doi.org/10.1109/{PDP}.2009.20 http://hal.inria.fr/inria-00331209/en/},
}

%F inria-00331209, version 1
%0 Conference Proceedings
%U http://hal.inria.fr/inria-00331209/en/
%2 INFO:INFO_OS;INFO:INFO_NI
%T High Throughput Intra-Node MPI Communication with Open-MX
%A Goglin, Brice
%A Goglin, B.
%A Goglin B.
%+ Laboratoire Bordelais de Recherche en Informatique - LaBRI - CNRS : UMR5800 - Université Sciences et Technologies - Bordeaux I - Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux - Université Victor Segalen - Bordeaux II - RUNTIME - INRIA Bordeaux - Sud-Ouest - INRIA - CNRS : UMR5800 - Université Sciences et Technologies - Bordeaux I - Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux
%X The increasing number of cores per node in high-performance computing requires an efficient intra-node MPI communication subsystem. Most existing MPI implementations rely on two copies across a shared memory-mapped file. Open-MX offers a single-copy mechanism that is tightly integrated in its regular communication stack, making it transparently available to the MX backend of many MPI layers. We describe this implementation and its offloaded copy backend using I/OAT hardware. Memory pinning requirements are then discussed, and overlapped pinning is introduced to enable the start of Open-MX intra-node data transfer earlier. Performance evaluation shows that this local communication stack performs better than MPICH2 and Open~MPI for large messages, reaching up to 70\,\% better throughput in micro-benchmarks when using I/OAT copy offload. Thanks to a single-copy being involved, the Open-MX intra-node communication throughput also does not heavily depend on cache sharing between processing cores, making these performance improvements easier to observe in real applications.
%G Anglais
%8 2009
%2 internationale
%B 17th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP2009)
%2 Weimar
%2 Allemagne
%8 2009-02
%E IEEE
%R 10.1109/PDP.2009.20

High Throughput Intra-Node MPI Communication with Open-MX

Brice Goglin (

) ^1, 2

(2009)

Accès au texte intégral
PDF

17th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP2009) (2009)

Lien vers texte intégral chez l'éditeur

Résumé : The increasing number of cores per node in high-performance computing requires an efficient intra-node MPI communication subsystem. Most existing MPI implementations rely on two copies across a shared memory-mapped file. Open-MX offers a single-copy mechanism that is tightly integrated in its regular communication stack, making it transparently available to the MX backend of many MPI layers. We describe this implementation and its offloaded copy backend using I/OAT hardware. Memory pinning requirements are then discussed, and overlapped pinning is introduced to enable the start of Open-MX intra-node data transfer earlier. Performance evaluation shows that this local communication stack performs better than MPICH2 and Open~MPI for large messages, reaching up to 70\,\% better throughput in micro-benchmarks when using I/OAT copy offload. Thanks to a single-copy being involved, the Open-MX intra-node communication throughput also does not heavily depend on cache sharing between processing cores, making these performance improvements easier to observe in real applications.

1 :	Laboratoire Bordelais de Recherche en Informatique (LaBRI)
	CNRS : UMR5800 – Université Sciences et Technologies - Bordeaux I – Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux – Université Victor Segalen - Bordeaux II
2 :	RUNTIME (INRIA Bordeaux - Sud-Ouest)
	INRIA – CNRS : UMR5800 – Université Sciences et Technologies - Bordeaux I – Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux

Domaine

Informatique/Système d'exploitation

Informatique/Réseaux et télécommunications

DOI : 10.1109/PDP.2009.20


inria-00331209, version 1
http://hal.inria.fr/inria-00331209/fr/
oai:hal.inria.fr:inria-00331209_v1
Contributeur : Brice Goglin <>
Soumis le : Mercredi 15 Octobre 2008, 17:10:41
Dernière modification le : Dimanche 21 Juin 2009, 12:46:42