Skip to Main content Skip to Navigation
Conference papers

2D Static Resource Allocation for Compressed Linear Algebra and Communication Constraints

Olivier Beaumont 1 Lionel Eyraud-Dubois 2, 1 Mathieu Verite 1
1 HiePACS - High-End Parallel Algorithms for Challenging Numerical Simulations
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
Abstract : This paper adresses static resource allocation problems for irregular distributed parallel applications. More precisely, we focus on two classical tiled linear algebra kernels: the Matrix Multiplication (MM) and the LU decomposition (LU) algorithms on large linear systems. In the context of parallel distributed platforms, data exchanges can dramatically degrade the performance of linear algebra kernels and in this context, compression techniques such as Block Low Rank (BLR) compression techniques are good candidates both for limiting data storage on each processor and data exchanges between processors. On the other hand, the use of BLR representation makes the static allocation problem of tiles to processors more complex. Indeed, the load associated to each tile depends on its compression factor, which induces an heterogeneous load balancing problem. In turn, solving this load balancing problem optimally might lead to complex allocation schemes, where the tiles allocated to a given processor are scattered all over the matrix. This in turn induces communication costs, since matrix multiplication and LU decompositions heavily rely on broadcasting operations along rows and columns of processors, so that the communication volume is minimized when the maximal number of different processors on any row and column is minimized. In the fully homogeneous case, 2D Block Cyclic (BC) allocation solves both load balancing and communication minimization issues simultaneously , but it might lead to bad load balancing in the heterogeneous case. Our goal in this paper is to propose data allocation schemes dedicated to BLR format and to prove that it is possible to obtain good performance on makespan when simultaneously balancing the load and minimizing the maximal number of different processor in any row or column.
Complete list of metadata

Cited literature [49 references]  Display  Hide  Download

https://hal.inria.fr/hal-02900244
Contributor : Mathieu Verite <>
Submitted on : Friday, July 24, 2020 - 5:05:37 PM
Last modification on : Tuesday, February 9, 2021 - 2:48:42 PM

File

HIPC-RR-v2.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02900244, version 2

Collections

Citation

Olivier Beaumont, Lionel Eyraud-Dubois, Mathieu Verite. 2D Static Resource Allocation for Compressed Linear Algebra and Communication Constraints. HIPC 2020: 27th IEEE International Conference on High Performance Computing, Data, and Analytics, Dec 2020, (virtual), India. ⟨hal-02900244v2⟩

Share