A Dynamic Data Middleware Cache for Rapidly-Growing Scientific Repositories

Abstract : Modern scientific repositories are growing rapidly in size. Scientists are increasingly interested in viewing the latest data as part of query results. Current scientific middleware cache systems, however, assume repositories are static. Thus, they cannot answer scientific queries with the latest data. The queries, instead, are routed to the repository until data at the cache is refreshed. In data-intensive scientific disciplines, such as astronomy, indiscriminate query routing or data refreshing often results in runaway network costs. This severely affects the performance and scalability of the repositories and makes poor use of the cache system. We present Delta a dynamic data middleware cache system for rapidly-growing scientific repositories. Delta's key component is a decision framework that adaptively decouples data objects--choosing to keep some data object at the cache, when they are heavily queried, and keeping some data objects at the repository, when they are heavily updated. Our algorithm profiles incoming workload to search for optimal data decoupling that reduces network costs. It leverages formal concepts from the network flow problem, and is robust to evolving scientific workloads. We evaluate the efficacy of Delta, through a prototype implementation, by running query traces collected from a real astronomy survey.
Type de document :
Communication dans un congrès
Indranil Gupta; Cecilia Mascolo. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. Springer, Lecture Notes in Computer Science, LNCS-6452, pp.64-84, 2010, Middleware 2010. 〈10.1007/978-3-642-16955-7_4〉
Liste complète des métadonnées

Littérature citée [38 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01055271
Contributeur : Hal Ifip <>
Soumis le : mardi 12 août 2014 - 11:53:46
Dernière modification le : mercredi 21 mars 2018 - 16:54:03
Document(s) archivé(s) le : mercredi 26 novembre 2014 - 22:41:18

Fichier

dache_fv1.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Tanu Malik, Xiaodan Wang, Philip Little, Amitabh Chaudhary, Ani Thakar. A Dynamic Data Middleware Cache for Rapidly-Growing Scientific Repositories. Indranil Gupta; Cecilia Mascolo. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. Springer, Lecture Notes in Computer Science, LNCS-6452, pp.64-84, 2010, Middleware 2010. 〈10.1007/978-3-642-16955-7_4〉. 〈hal-01055271〉

Partager

Métriques

Consultations de la notice

215

Téléchargements de fichiers

56