Memory Footprint of Locality Information on Many-Core Platforms

Brice Goglin 1
1 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
Abstract : Exploiting the power of HPC platforms requires knowledge of their increasingly complex hardware topologies. Multiple components of the software stack, for instance MPI implementations or OpenMP runtimes, now perform their own topology discovery to find out the available cores and memory, and to better place tasks based on their affinities. We study in this article the impact of this topology discovery in terms of memory footprint. Storing locality information wastes an amount of physical memory that is becoming an issue on many-core platforms on the road to exascale. We demonstrate that this information may be factorized between processes by using a shared-memory region. Our analysis of the physical and virtual memories in supercomputing architectures shows that this shared region can be mapped at the same virtual address in all processes, hence dramatically simplifying the software implementation. Our implementation in hwloc and Open MPI shows a memory footprint that does not increase with the number of MPI ranks per node anymore. Moreover the job launch time is decreased by more than a factor of 2 on an Intel Knights Landing Xeon Phi and on a 96-core NUMA platform.
Complete list of metadatas

Cited literature [21 references]  Display  Hide  Download
Contributor : Brice Goglin <>
Submitted on : Tuesday, March 13, 2018 - 2:40:01 PM
Last modification on : Wednesday, May 15, 2019 - 5:24:09 PM
Long-term archiving on : Thursday, June 14, 2018 - 2:57:08 PM


article (1).pdf
Files produced by the author(s)



Brice Goglin. Memory Footprint of Locality Information on Many-Core Platforms. 6th Workshop on Runtime and Operating Systems for the Many-core Era (ROME 2018), held in conjunction with IPDPS, May 2018, Vancouver, BC, Canada. pp.10, ⟨10.1109/IPDPSW.2018.00201⟩. ⟨hal-01644087⟩



Record views


Files downloads