Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Adaptive Request Scheduling for the I/O Forwarding Layer

Abstract : In this paper, we present an approach to adapt the HPC I/O forwarding layer to the application access patterns. I/O forwarding is a technique used in most supercomputers today to alleviate contention in the access to the shared storage infrastructure. Because of its location, between processing nodes and parallel file system servers, it has been used to propose optimization techniques such as request reordering, aggregation, and scheduling. Such techniques can usually provide good results only for some of the situations, or depend on the right choice of parameter values. Our case study for this work is the TWINS request scheduling algorithm, which aims at coordinating the access of intermediate I/O nodes to the data servers. Our approach uses a neural network to classify application access patterns, and a reinforcement learning technique to empower the scheduler to learn the best parameter values to each access pattern during its execution, without the need of a previous training phase. Our evaluation of the access pattern detection neural network shows average precision of 98% during write experiments, and minimum precision of 98% during reads. The latter is an important result as most performance improvements by TWINS were observed for read experiments. Furthermore, we demonstrate that our contextual bandit strategy is able to learn the best value for the window size, achieving approximately 75% of precision-98% of the performance provided by the best window size-in the first hundreds of steps.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [36 references]  Display  Hide  Download
Contributor : Francieli Zanon Boito <>
Submitted on : Friday, January 25, 2019 - 4:05:49 PM
Last modification on : Friday, July 10, 2020 - 7:58:53 AM
Document(s) archivé(s) le : Friday, April 26, 2019 - 2:22:48 PM


CCGrid_2018___Adaptive_TWINS (...
Files produced by the author(s)


  • HAL Id : hal-01994677, version 1


Jean Luca Bez, Francieli Zanon Boito, Ramon Nou, Alberto Miranda, Toni Cortes, et al.. Adaptive Request Scheduling for the I/O Forwarding Layer. 2019. ⟨hal-01994677v1⟩



Record views


Files downloads