JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers

Abstract : The easily-accessible computation power offered by cloud infrastructures coupled with the revolution of Big Data are expanding the scale and speed at which data analysis is per-formed. In their quest for finding the Value in the 3 Vs of Big Data, applications process larger data sets, within and across clouds. Enabling fast data transfers across geograph-ically distributed sites becomes particularly important for applications which manage continuous streams of events in real time. Scientific applications (e.g. the Ocean Observa-tory Initiative or the ATLAS experiment) as well as com-mercial ones (e.g. Microsoft's Bing and Office 365 large-scale services) operate on tens of data-centers around the globe and follow similar patterns: they aggregate monitoring data, assess the QoS or run global data mining queries based on in-ter site event stream processing. In this paper, we propose a set of strategies for efficient transfers of events between cloud data-centers and we introduce JetStream: a prototype implementing these strategies as a high performance batch-based streaming middleware. JetStream is able to self-adapt to the streaming conditions by modeling and monitoring a set of context parameters. It further aggregates the available bandwidth by enabling multi-route streaming across cloud sites. The prototype was validated on tens of nodes from US and Europe data-centers of the Windows Azure cloud using synthetic benchmarks and with application code from the context of the Alice experiment at CERN. The results show an increase in transfer rate of 250 times over individual event streaming. Besides, introducing an adaptive transfer strategy brings an additional 25% gain. Finally, the transfer rate can further be tripled thanks to the use of multi-route streaming.
Type de document :
Communication dans un congrès
DEBS'14 - The 8th ACM International Conference on Distributed Event-Based Systems, May 2014, Mumbai, India. ACM, pp.23 - 34, 2014, 〈10.1145/2611286.2611298〉
Liste complète des métadonnées

Littérature citée [40 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01090281
Contributeur : Radu Tudoran <>
Soumis le : vendredi 19 décembre 2014 - 16:58:38
Dernière modification le : jeudi 15 novembre 2018 - 11:57:44
Document(s) archivé(s) le : samedi 15 avril 2017 - 01:49:53

Fichier

DEBS2014 Camera Ready.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Radu Tudoran, Olivier Nano, Ivo Santos, Alexandru Costan, Hakan Soncu, et al.. JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers. DEBS'14 - The 8th ACM International Conference on Distributed Event-Based Systems, May 2014, Mumbai, India. ACM, pp.23 - 34, 2014, 〈10.1145/2611286.2611298〉. 〈hal-01090281〉

Partager

Métriques

Consultations de la notice

860

Téléchargements de fichiers

664