Towards the Structural Modeling of the Topology of next-generation heterogeneous cluster Nodes with hwloc

Brice Goglin 1, 2
Abstract : Parallel computing platforms are increasingly complex, with multiple cores, shared caches, and NUMA memory interconnects, as well as asymmetric I/O access. Upcoming architectures will add a heterogeneous memory subsystem with non-volatile and/or high-bandwidth memory banks. Parallel applications developers have to take locality into account before they can expect good efficiency on these platforms. Thus there is a strong need for a portable tool gathering and exposing this information. The Hardware Locality project (hwloc) offers a tree representation of the hardware based on the inclusion of CPU resources and localities of memory and I/O devices. It is already widely used for affinity-based task placement in high performance computing. We present how hwloc represents parallel computing nodes, from the hierarchy of computing and memory resources to I/O device locality. It builds a structural model of the hardware to help application find the best resources fitting their needs. hwloc also annotates objects to ease identification of resources from different programming points of view. We finally describe how it helps process managers and batch schedulers to deal with the topology of multiple cluster nodes, by offering different compression techniques for better management of thousands of nodes.
Liste complète des métadonnées

https://hal.inria.fr/hal-01400264
Contributeur : Brice Goglin <>
Soumis le : vendredi 25 novembre 2016 - 10:47:46
Dernière modification le : samedi 26 novembre 2016 - 01:05:33

Fichier

article.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01400264, version 2

Collections

Citation

Brice Goglin. Towards the Structural Modeling of the Topology of next-generation heterogeneous cluster Nodes with hwloc. [Research Report] Inria. 2016. <hal-01400264v2>

Partager

Métriques

Consultations de
la notice

78

Téléchargements du document

84