An RDMA Middleware for Asynchronous Multi-stage Shuffling in Analytical Processing

Abstract : A key component in large scale distributed analytical processing is shuffling, the distribution of data to multiple nodes such that the computation can be done in parallel. In this paper we describe the design and implementation of a communication middleware to support data shuffling for executing multi-stage analytical processing operations in parallel. The middleware relies on RDMA (Remote Direct Memory Access) to provide basic operations to asynchronously exchange data among multiple machines. Experimental results show that the RDMA-based middleware developed can provide a 75 % reduction of the costs of communication operations on parallel analytical processing tasks, when compared with a sockets middleware.
Type de document :
Communication dans un congrès
Márk Jelasity; Evangelia Kalyvianaki. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. Lecture Notes in Computer Science, LNCS-9687, pp.61-74, 2016, Distributed Applications and Interoperable Systems. 〈10.1007/978-3-319-39577-7_5〉
Liste complète des métadonnées

Littérature citée [16 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01434800
Contributeur : Hal Ifip <>
Soumis le : vendredi 13 janvier 2017 - 14:02:44
Dernière modification le : vendredi 13 janvier 2017 - 14:05:48
Document(s) archivé(s) le : vendredi 14 avril 2017 - 19:12:43

Fichier

 Accès restreint
Fichier visible le : 2019-01-01

Connectez-vous pour demander l'accès au fichier

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Rui Gonçalves, José Pereira, Ricardo Jiménez-Peris. An RDMA Middleware for Asynchronous Multi-stage Shuffling in Analytical Processing. Márk Jelasity; Evangelia Kalyvianaki. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. Lecture Notes in Computer Science, LNCS-9687, pp.61-74, 2016, Distributed Applications and Interoperable Systems. 〈10.1007/978-3-319-39577-7_5〉. 〈hal-01434800〉

Partager

Métriques

Consultations de la notice

24