Dealing with Skewed Data in Structured Overlays using Variable Hash Functions

Maeva Antoine 1 Fabrice Huet 1
1 SCALE - Safe Composition of Autonomous applications with Large-SCALE Execution environment
CRISAM - Inria Sophia Antipolis - Méditerranée , COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
Abstract : Storing highly skewed data in a distributed system has become a very frequent issue, in particular with the emergence of semantic web and Big Data. This often leads to biased data dissemination among nodes. Addressing load imbalance is necessary, especially to minimize response time and avoid workload being handled by only one or few nodes. Our contribution aims at dynamically managing load imbalance by allowing multiple hash functions on different peers, while maintaining consistency of the overlay. Our experiments, on highly skewed data sets from the semantic web, show we can distribute data on at least 300 times more peers than when not using any load balancing strategy.
Type de document :
Communication dans un congrès
The 15th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), Dec 2014, Hong Kong, Hong Kong SAR China. IEEE Computer Society, 2014 15th International Conference on Parallel and Distributed Computing, Applications and Technologies, pp.42-48, <10.1109/PDCAT.2014.15>


https://hal.inria.fr/hal-01101678
Contributeur : Maeva Antoine <>
Soumis le : vendredi 9 janvier 2015 - 13:25:37
Dernière modification le : lundi 5 octobre 2015 - 17:01:49
Document(s) archivé(s) le : samedi 12 septembre 2015 - 01:10:23

Fichier

can.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Maeva Antoine, Fabrice Huet. Dealing with Skewed Data in Structured Overlays using Variable Hash Functions. The 15th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), Dec 2014, Hong Kong, Hong Kong SAR China. IEEE Computer Society, 2014 15th International Conference on Parallel and Distributed Computing, Applications and Technologies, pp.42-48, <10.1109/PDCAT.2014.15>. <hal-01101678>

Partager

Métriques

Consultations de
la notice

120

Téléchargements du document

146