AutoParallel: A Python module for automatic parallelization and distributed execution of affine loop nests

Abstract : The last improvements in programming languages, programming models, and frameworks have focused on abstracting the users from many programming issues. Among others, recent programming frameworks include simpler syntax , automatic memory management and garbage collection, which simplifies code re-usage through library packages, and easily configurable tools for deployment. For instance, Python has risen to the top of the list of the programming languages due to the simplicity of its syntax, while still achieving a good performance even being an interpreted language. Moreover, the community has helped to develop a large number of libraries and modules, tuning the most commonly used to obtain great performance. However, there is still room for improvement when preventing users from dealing directly with distributed and parallel computing issues. This paper proposes and evaluates AutoPar-allel, a Python module to automatically find an appropriate task-based parallelization of affine loop nests to execute them in parallel in a distributed computing infrastructure. This parallelization can also include the building of data blocks to increase task granularity in order to achieve a good execution performance. Moreover, AutoParallel is based on sequential programming and only contains a small annotation in the form of a Python decorator so that anyone with little programming skills can scale up an application to hundreds of cores.
Document type :
Conference papers
Complete list of metadatas

https://hal.inria.fr/hal-01936351
Contributor : Philippe Clauss <>
Submitted on : Tuesday, November 27, 2018 - 1:30:50 PM
Last modification on : Saturday, June 1, 2019 - 11:18:15 AM
Long-term archiving on : Thursday, February 28, 2019 - 3:06:34 PM

File

AutoParallel.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01936351, version 1

Citation

Cristian Ramon-Cortes, Ramon Amela, Jorge Ejarque, Philippe Clauss, Rosa Badia. AutoParallel: A Python module for automatic parallelization and distributed execution of affine loop nests. PyHPC 2018 - 8th Workshop on Python for High-Performance and Scientific Computing, Nov 2018, Dallas, TX, United States. ⟨hal-01936351⟩

Share

Metrics

Record views

67

Files downloads

186