Skip to Main content Skip to Navigation
Conference papers

Mimir+: An Optimized Framework of MapReduce on Heterogeneous High-Performance Computing System

Abstract : In this paper, we present an optimized data processing framework: Mimir+. Mimir+ is an implementation of MapReduce over MPI. In order to take full advantage of heterogeneous computing system, we propose the concept of Pre-acceleration to reconstruct a heterogeneous workflow and implement the interfaces of GPU so that Mimir+ can facilitate data processing through reasonable tasks and data scheduling between CPU and GPU. We evaluate Mimir+ via two benchmarks (i.e. the WordCount and large-scale matrix multiplication) on the Tianhe-2 supercomputing system. Experimental results demonstrate that Mimir+ achieves excellent acceleration effect compared with original Mimir.
Document type :
Conference papers
Complete list of metadata

Cited literature [8 references]  Display  Hide  Download

https://hal.inria.fr/hal-02279555
Contributor : Hal Ifip <>
Submitted on : Thursday, September 5, 2019 - 1:31:21 PM
Last modification on : Thursday, April 30, 2020 - 1:22:02 PM
Long-term archiving on: : Thursday, February 6, 2020 - 5:30:17 AM

File

477597_1_En_18_Chapter.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Nan Hu, Zhiguang Chen, Yunfei Du, Yutong Lu. Mimir+: An Optimized Framework of MapReduce on Heterogeneous High-Performance Computing System. 15th IFIP International Conference on Network and Parallel Computing (NPC), Nov 2018, Muroran, Japan. pp.164-168, ⟨10.1007/978-3-030-05677-3_18⟩. ⟨hal-02279555⟩

Share

Metrics

Record views

62

Files downloads

24