Topology-aware resource management for HPC applications

Abstract : The Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for efficiently delivering computing power to applications in supercomputing environments. Its main intelligence relies on resource selection techniques to find the most adapted resources to schedule the users' jobs. Improper resource selection operations may lead to poor performance executions and global system utilization along with an increase of the system fragmentation and jobs starvation. These phenomena play a role in the increase of the platforms' total cost of ownership and should be minimized. This paper introduces a new method that takes into account the topology of the machine and the application characteristics to determine the best choice among the available nodes of the platform based upon their position within the network and taking into account the applications communication pattern. To validate our approach, we integrate this algorithm as a plugin for Slurm, a popular and widespread HPC resource and job management system (RJMS). We assess our plugin with different optimization schemes by comparing with the default topology-aware Slurm algorithm using both emulation and simulation of a large-scale platform, and by carrying out experiments in a real cluster. We show that transparently taking into account the job communication pattern and the topology allows for relevant performance gains.
Complete list of metadatas

Cited literature [24 references]  Display  Hide  Download

https://hal.inria.fr/hal-01414196
Contributor : Adèle Villiermet <>
Submitted on : Tuesday, December 13, 2016 - 6:05:39 PM
Last modification on : Tuesday, August 13, 2019 - 3:20:09 PM

Links full text

Identifiers

Collections

Relations

Citation

Yiannis Georgiou, Emmanuel Jeannot, Guillaume Mercier, Adèle Villiermet. Topology-aware resource management for HPC applications. ICDCN 2017 - 18th International Conference on Distributed Computing and Networking, Jan 2017, Hyderabad, India. ⟨10.1145/3007748.3007768⟩. ⟨hal-01414196⟩

Share

Metrics

Record views

451