HAL - INRIA :: [inria-00288757, version 2] Improving Message Passing over Ethernet with I/OAT Copy Offload in Open-MX

inria-00288757, version 2

%% inria-00288757, version 2
%% http://hal.inria.fr/inria-00288757/en/
@inproceedings{GOGLIN:2008:INRIA-00288757:2,
	title = { {I}mproving {M}essage {P}assing over {E}thernet with {I}/{OAT} {C}opy {O}ffload in {O}pen-{MX}},
	author = {{G}oglin, {B}rice},
	abstract = {{O}pen-{MX} is a new message passing layer implemented on top of the generic {E}thernet stack of the {L}inux kernel. {O}pen-{MX} works on all {E}thernet hardware, but it suffers from expensive memory copy requirements on the receiver side due to the hardware's inability to deposit messages directly in the target application buffers. {T}his article presents the implementation of an asynchronous memory copy offload in the {O}pen-{MX} stack thanks to {I}ntel {I}/{O} {A}cceleration {T}echnology. {T}he overlapping of large message fragment copies with the processing increases the receive throughput by 30% while reducing the {CPU} usage by up to 40%. {I}t enables {O}pen-{MX} to reach 10 gigabit/s {E}thernet line rate for large messages. {O}pen-{MX} large intra-node communication also benefits significantly from the {I}/{OAT} hardware since the performance of its one-copy-based local communication mechanism is almost doubled by using blocking {I}/{OAT} memory copies. {B}y combining all these optimizations, the {O}pen-{MX} large message performance on top of 10{G} hardware is now able to bridge the gap with the native {M}yrinet {E}xpress stack.},
	language = {{A}nglais},
	affiliation = {{L}aboratoire {B}ordelais de {R}echerche en {I}nformatique - {L}a{BRI} - {CNRS} : {UMR}5800 - {U}niversit{\'e} {S}ciences et {T}echnologies - {B}ordeaux {I} - {E}cole {N}ationale {S}up{\'e}rieure d'{E}lectronique, {I}nformatique et {R}adiocommunications de {B}ordeaux - {U}niversit{\'e} {V}ictor {S}egalen - {B}ordeaux {II} - {RUNTIME} - {INRIA} {B}ordeaux - {S}ud-{O}uest - {INRIA} - {CNRS} : {UMR}5800 - {U}niversit{\'e} {S}ciences et {T}echnologies - {B}ordeaux {I} - {E}cole {N}ationale {S}up{\'e}rieure d'{E}lectronique, {I}nformatique et {R}adiocommunications de {B}ordeaux },
	booktitle = {{C}luster 2008 },
	address = {{T}sukuba {J}apon },
	editor = {{IEEE} },
	audience = {internationale },
	doi = {10.1109/{CLUSTR}.2008.4663775 },
	year = {2008},
	URL = {dx.doi.org/10.1109/{CLUSTR}.2008.4663775 http://hal.inria.fr/inria-00288757/en/},
}

Improving Message Passing over Ethernet with I/OAT Copy Offload in Open-MX

Brice Goglin (

)^a^, 1, 2

(2008)

Accès au texte intégral
PDF

Cluster 2008 (2008)

Lien vers texte intégral chez l'éditeur

Résumé : Open-MX is a new message passing layer implemented on top of the generic Ethernet stack of the Linux kernel. Open-MX works on all Ethernet hardware, but it suffers from expensive memory copy requirements on the receiver side due to the hardware's inability to deposit messages directly in the target application buffers. This article presents the implementation of an asynchronous memory copy offload in the Open-MX stack thanks to Intel I/O Acceleration Technology. The overlapping of large message fragment copies with the processing increases the receive throughput by 30% while reducing the CPU usage by up to 40%. It enables Open-MX to reach 10 gigabit/s Ethernet line rate for large messages. Open-MX large intra-node communication also benefits significantly from the I/OAT hardware since the performance of its one-copy-based local communication mechanism is almost doubled by using blocking I/OAT memory copies. By combining all these optimizations, the Open-MX large message performance on top of 10G hardware is now able to bridge the gap with the native Myrinet Express stack.

a –	INRIA
1 :	Laboratoire Bordelais de Recherche en Informatique (LaBRI)
	CNRS : UMR5800 – Université Sciences et Technologies - Bordeaux I – Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux – Université Victor Segalen - Bordeaux II
2 :	RUNTIME (INRIA Bordeaux - Sud-Ouest)
	INRIA – CNRS : UMR5800 – Université Sciences et Technologies - Bordeaux I – Ecole Nationale Supérieure d'Electronique, Informatique et Radiocommunications de Bordeaux

Domaine

Informatique/Système d'exploitation

DOI : 10.1109/CLUSTR.2008.4663775

Versions disponibles :

v1 (18-06-2008)

v2 (02-07-2008)


inria-00288757, version 2
http://hal.inria.fr/inria-00288757/fr/
oai:hal.inria.fr:inria-00288757_v2
Contributeur : Brice Goglin <>
Soumis le : Mercredi 2 Juillet 2008, 16:30:11
Dernière modification le : Dimanche 21 Juin 2009, 12:47:16