Skip to Main content Skip to Navigation
Conference papers

Making Speculative Scheduling Robust to Incomplete Data

Ana Gainaru 1 Guillaume Pallez 2
2 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
Abstract : In this work, we study the robustness of Speculative Scheduling to data incompleteness. Speculative scheduling has allowed to incorporate future types of applications into the design of HPC schedulers, specifically applications whose runtime is not perfectly known but can be modeled with probability distributions. Preliminary studies show the importance of spec- ulative scheduling in dealing with stochastic applications when the application runtime model is completely known. In this work we show how one can extract enough information even from incomplete behavioral data for a given HPC applications so that speculative scheduling still performs well. Specifically, we show that for synthetic runtimes who follow usual probability distributions such as truncated normal or exponential, we can extract enough data from as little as 10 previous runs, to be within 5% of the solution which has exact information. For real traces of applications, the performance with 10 data points varies with the applications (within 20% of the full-knowledge solution), but converges fast (5% with 100 previous samples). Finally a side effect of this study is to show the importance of the theoretical results obtained on continuous probability distributions for speculative scheduling. Indeed, we observe that the solutions for such distributions are more robust to incomplete data than the solutions for discrete distributions.
Complete list of metadata

Cited literature [24 references]  Display  Hide  Download
Contributor : Guillaume Pallez (aupy) Connect in order to contact the contributor
Submitted on : Tuesday, October 29, 2019 - 8:01:55 AM
Last modification on : Tuesday, November 5, 2019 - 3:50:46 PM
Long-term archiving on: : Thursday, January 30, 2020 - 2:06:19 PM


Files produced by the author(s)


  • HAL Id : hal-02336582, version 1



Ana Gainaru, Guillaume Pallez. Making Speculative Scheduling Robust to Incomplete Data. ScalA19: 10th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, Nov 2019, Denver, United States. ⟨hal-02336582⟩



Record views


Files downloads