Skip to Main content Skip to Navigation
Conference papers

A Load-Aware Data Placement Policy on Cluster File System

Abstract : In a large-scale cluster system with many applications running on it, cluster-wide I/O access workload disparity and disk saturation on only some storage servers have been the severe performance bottleneck that deteriorates the system I/O performance. As a result, the system response time will increase and the throughput of the system will decrease drastically. In this paper, we present a load-aware data placement policy that will distribute data across the storage servers based on the load of each server and automatically migrate data from heavily-loaded servers to lightly-loaded servers. This policy is adaptive and self-managing. It operates without any prior knowledge of application access workload characteristics or the capabilities of storage servers. It can make full use of the aggregate disk bandwidth of all storage servers efficiently. Performance evaluation shows that our policy will improve the aggregate I/O bandwidth by 10%-20% compared with random data placement policy especially under mixed workloads.
Document type :
Conference papers
Complete list of metadata

Cited literature [23 references]  Display  Hide  Download

https://hal.inria.fr/hal-01593007
Contributor : Hal Ifip <>
Submitted on : Monday, September 25, 2017 - 4:33:12 PM
Last modification on : Thursday, March 5, 2020 - 5:40:26 PM
Long-term archiving on: : Tuesday, December 26, 2017 - 2:22:52 PM

File

978-3-642-24403-2_2_Chapter.pd...
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Yu Wang, Jing Xing, Jin Xiong, Dan Meng. A Load-Aware Data Placement Policy on Cluster File System. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. pp.17-31, ⟨10.1007/978-3-642-24403-2_2⟩. ⟨hal-01593007⟩

Share

Metrics

Record views

105

Files downloads

176