JetStream: Enabling high throughput live event streaming on multi-site clouds

Abstract : Scientific and commercial applications operate nowadays on tens of cloud datacenters around the globe, following similar patterns: they aggregate monitoring or sensor data, assess the QoS or run global data mining queries based on inter-site event stream processing. Enabling fast data transfers across geographically distributed sites allows such applications to manage the continuous streams of events in real time and quickly react to changes. However, traditional event processing engines often consider data resources as second-class citizens and support access to data only as a side-effect of computation (i.e. they are not concerned by the transfer of events from their source to the processing site). This is an efficient approach as long as the processing is executed in a single cluster where nodes are interconnected by low latency networks. In a distributed environment, consisting of multiple datacenters, with orders of magnitude differences in capabilities and connected by a WAN, this will undoubtedly lead to significant latency and performance variations. This is namely the challenge we address in this paper, by proposing JetStream, a high performance batch-based streaming middleware for efficient transfers of events between cloud datacenters. JetStream is able to self-adapt to the streaming conditions by modeling and monitoring a set of context parameters. It further aggregates the available bandwidth by enabling multi-route streaming across cloud sites, while at the same time optimizing resource utilization and increasing cost efficiency. The prototype was validated on tens of nodes from US and Europe datacenters of the Windows Azure cloud with synthetic benchmarks and a real-life application monitoring the ALICE experiment at CERN. The results show a 3x increase of the transfer rate using the adaptive multi-route streaming, compared to state of the art solutions.
Type de document :
Article dans une revue
Future Generation Computer Systems, Elsevier, 2016, 54, 〈10.1016/j.future.2015.01.016〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01239124
Contributeur : Alexandru Costan <>
Soumis le : lundi 14 décembre 2015 - 14:01:31
Dernière modification le : mercredi 16 mai 2018 - 11:23:28
Document(s) archivé(s) le : samedi 29 avril 2017 - 10:21:25

Fichier

fgcs.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Domaine public

Identifiants

Citation

Radu Tudoran, Alexandru Costan, Olivier Nano, Ivo Santos, Hakan Soncu, et al.. JetStream: Enabling high throughput live event streaming on multi-site clouds. Future Generation Computer Systems, Elsevier, 2016, 54, 〈10.1016/j.future.2015.01.016〉. 〈hal-01239124〉

Partager

Métriques

Consultations de la notice

867

Téléchargements de fichiers

217