Skip to Main content Skip to Navigation
Conference papers

A Fine-Grained Performance Bottleneck Analysis Method for HDFS

Abstract : The performance issue of HDFS has always been a great concern due to its widely adoption in both production and research environments. However, a fine-grained performance analysis tool is missing to effectively identify the bottlenecks as well as to provide useful guidance for performance optimization. In this paper, we propose a fine-grained performance bottleneck analysis tool, which extends HTrace with fine-grained instrumentation points that are missing in Hadoop official distribution. In addition, we propose an effective trace merging method that improves the understandability of our analysis. We analyze the performance of HDFS under different kinds of workloads and get undiscovered insights.
Document type :
Conference papers
Complete list of metadata

Cited literature [6 references]  Display  Hide  Download

https://hal.inria.fr/hal-02279548
Contributor : Hal Ifip <>
Submitted on : Thursday, September 5, 2019 - 1:30:57 PM
Last modification on : Thursday, September 5, 2019 - 1:35:35 PM
Long-term archiving on: : Thursday, February 6, 2020 - 2:44:56 AM

File

477597_1_En_17_Chapter.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Yi Liu, Yunchun Li, Honggang Zhou, Jingyi Zhang, Hailong Yang, et al.. A Fine-Grained Performance Bottleneck Analysis Method for HDFS. 15th IFIP International Conference on Network and Parallel Computing (NPC), Nov 2018, Muroran, Japan. pp.159-163, ⟨10.1007/978-3-030-05677-3_17⟩. ⟨hal-02279548⟩

Share

Metrics

Record views

48

Files downloads

7