A Load-Aware Data Placement Policy on Cluster File System

Abstract : In a large-scale cluster system with many applications running on it, cluster-wide I/O access workload disparity and disk saturation on only some storage servers have been the severe performance bottleneck that deteriorates the system I/O performance. As a result, the system response time will increase and the throughput of the system will decrease drastically. In this paper, we present a load-aware data placement policy that will distribute data across the storage servers based on the load of each server and automatically migrate data from heavily-loaded servers to lightly-loaded servers. This policy is adaptive and self-managing. It operates without any prior knowledge of application access workload characteristics or the capabilities of storage servers. It can make full use of the aggregate disk bandwidth of all storage servers efficiently. Performance evaluation shows that our policy will improve the aggregate I/O bandwidth by 10%-20% compared with random data placement policy especially under mixed workloads.
Type de document :
Communication dans un congrès
Erik Altman; Weisong Shi. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. Springer, Lecture Notes in Computer Science, LNCS-6985, pp.17-31, 2011, Network and Parallel Computing. 〈10.1007/978-3-642-24403-2_2〉
Liste complète des métadonnées

Littérature citée [23 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01593007
Contributeur : Hal Ifip <>
Soumis le : lundi 25 septembre 2017 - 16:33:12
Dernière modification le : vendredi 1 décembre 2017 - 01:09:43
Document(s) archivé(s) le : mardi 26 décembre 2017 - 14:22:52

Fichier

978-3-642-24403-2_2_Chapter.pd...
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Yu Wang, Jing Xing, Jin Xiong, Dan Meng. A Load-Aware Data Placement Policy on Cluster File System. Erik Altman; Weisong Shi. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. Springer, Lecture Notes in Computer Science, LNCS-6985, pp.17-31, 2011, Network and Parallel Computing. 〈10.1007/978-3-642-24403-2_2〉. 〈hal-01593007〉

Partager

Métriques

Consultations de la notice

40

Téléchargements de fichiers

9