Managing the Topology of Heterogeneous Cluster Nodes with Hardware Locality (hwloc)

Brice Goglin 1, 2
1 RUNTIME - Efficient runtime systems for parallel architectures
Inria Bordeaux - Sud-Ouest, UB - Université de Bordeaux, CNRS - Centre National de la Recherche Scientifique : UMR5800
Abstract : Modern computing platforms are increasingly complex, with multiple cores, shared caches, and NUMA architectures. Parallel applications developers have to take locality into account before they can expect good efficiency on these platforms. Thus there is a strong need for a portable tool gathering and exposing this information. The Hardware Locality project (hwloc) offers a tree representation of the hardware based on the inclusion and localities of the CPU and memory resources. It is already widely used for affinity-based task placement in high performance computing. In this article we present how hwloc is extended to describe more than computing and memory resources. Indeed, I/O device locality is becoming another important aspect of locality since high performance GPUs, network or InfiniBand interfaces possess privileged access to some of the cores and memory banks. hwloc integrates this knowledge into its topology representation and offers an interoperability API to extend existing libraries such as CUDA with locality information. We also describe how hwloc now helps process managers and batch schedulers to deal with the topology of multiple cluster nodes, together with compression for better scalability up to thousands of nodes.
Type de document :
Communication dans un congrès
International Conference on High Performance Computing & Simulation (HPCS 2014), Jul 2014, Bologna, Italy. IEEE, 2014


https://hal.inria.fr/hal-00985096
Contributeur : Brice Goglin <>
Soumis le : mardi 29 avril 2014 - 11:40:17
Dernière modification le : jeudi 10 septembre 2015 - 01:06:54
Document(s) archivé(s) le : mardi 29 juillet 2014 - 12:05:58

Fichier

article.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00985096, version 1

Collections

Citation

Brice Goglin. Managing the Topology of Heterogeneous Cluster Nodes with Hardware Locality (hwloc). International Conference on High Performance Computing & Simulation (HPCS 2014), Jul 2014, Bologna, Italy. IEEE, 2014. <hal-00985096>

Exporter

Partager

Métriques

Consultations de
la notice

542

Téléchargements du document

374