Kernel Assisted Collective Intra-node Communication Among Multicore and Manycore CPUs

Abstract : More memory hierarchies, NUMA architectures and network-style interconnection are widely used in modern many-core CPU design to achieve performance scalability. As a leading intra-node programming model, Message Passing Interface (MPI) implementations must exploit these architectures to provide reliable performance portability. These new architectures not only require specialized MPI point-to-point messaging protocols, they also require carefully designed and tuned algorithms for MPI collective operations. Multiple issues must be taken into account: 1) minimizing the number of copies required, 2) minimizing traffic to ''remote'' NUMA memory, and 3) carefully avoiding memory bottlenecks for ''rooted'' collective operations. In this paper, we present a kernel assisted intra-node collective module addressing those three issues on many-core systems. A kernel level inter-process memory copy module, called KNEM, is used by a novel Open MPI collective module to implement several improved strategies based on decreasing the number of intermediate memory copies and improving locality to reduce both the pressure on the memory banks and the cache pollution. The collective topology is mapped onto the NUMA topology to minimize cross traffic on inter-socket links. Experiments illustrate that the KNEM enabled Open MPI collective module can achieve up to a threefold speedup on synthetic benchmarks, resulting in a 12% improvement for a parallel graph shortest path discovery application.
Type de document :
Rapport
[Research Report] 2010, pp.11
Liste complète des métadonnées


https://hal.inria.fr/inria-00544872
Contributeur : Brice Goglin <>
Soumis le : jeudi 9 décembre 2010 - 10:40:00
Dernière modification le : samedi 17 septembre 2016 - 01:36:46
Document(s) archivé(s) le : lundi 5 novembre 2012 - 12:55:23

Fichier

kernel-assisted-comm-multicore...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00544872, version 1

Collections

Citation

Teng Ma, George Bosilca, Aurélien Bouteiller, Brice Goglin, Jeffrey Squyres, et al.. Kernel Assisted Collective Intra-node Communication Among Multicore and Manycore CPUs. [Research Report] 2010, pp.11. <inria-00544872>

Partager

Métriques

Consultations de
la notice

667

Téléchargements du document

151