BitDew: A Programmable Environment for Large-Scale Data Management and Distribution

Gilles Fedak 1 Haiwu He 1 Franck Cappello 1
1 GRAND-LARGE - Global parallel and distributed computing
LRI - Laboratoire de Recherche en Informatique, LIFL - Laboratoire d'Informatique Fondamentale de Lille, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
Abstract : Desktop Grids use the computing, network and storage resources from idle desktop PC's distributed over multiple-LAN's or the Internet to compute a large variety of resource-demanding distributed applications. While these applications need to access, compute, store and circulate large volumes of data, little attention has been paid to data management in such large-scale, dynamic, heterogeneous, volatile and highly distributed Grids. In most cases, data management relies on ad-hoc solutions, and providing general approach is still a challenging issue. To address this problem, we propose the BitDew framework, a programmable environment for automatic and transparent data management on computational Desktop Grids. This paper describes the BitDew programming interface, its architecture, and the performance evaluation of its runtime components. BitDew relies on a specific set of meta-data to drive key data management operations, namely life cycle, distribution, placement, replication and fault-tolerance with a high level of abstraction. The Bitdew runtime environment is a flexible distributed service architecture that integrates modular P2P components such as DHT's for a distributed data catalog and collaborative transport protocols for data distribution. Through several examples, we describe how application programmers and Bitdew users can exploit Bitdew's features. The performance evaluation demonstrates that the high level of abstraction and transparency is obtained with a reasonable overhead, while offering the benefit of scalability, performance and fault tolerance with little programming cost.
Type de document :
Rapport
[Research Report] INRIA. 2008, pp.24
Liste complète des métadonnées

Littérature citée [38 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00216126
Contributeur : Gilles Fedak <>
Soumis le : mardi 22 avril 2008 - 16:54:26
Dernière modification le : jeudi 5 avril 2018 - 12:30:12
Document(s) archivé(s) le : vendredi 25 novembre 2016 - 22:59:26

Fichier

RR-6427.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00216126, version 4

Citation

Gilles Fedak, Haiwu He, Franck Cappello. BitDew: A Programmable Environment for Large-Scale Data Management and Distribution. [Research Report] INRIA. 2008, pp.24. 〈inria-00216126v4〉

Partager

Métriques

Consultations de la notice

664

Téléchargements de fichiers

229