Skip to Main content Skip to Navigation
Conference papers

A Fully Decentralized Autoscaling Algorithm for Stream Processing Applications

Mehdi Belkhiria 1 Cédric Tedeschi 1
1 MYRIADS - Design and Implementation of Autonomous Distributed Systems
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : Stream Processing deals with the efficient, real-time processing of continuous streams of data. Stream Processing engines ease the development and deployment of such applications which are commonly pipelines of operators to be traversed by each data item. Due to the varying velocity of the streams, autoscaling is needed to dynamically adapt the number of instances of each operator. With the advent of geographically-dispersed computing platforms such as Fog platforms, operators are dispersed accordingly, and autoscaling needs to be decentralized as well. In this paper, we propose an algorithm allowing for scaling decisions to be taken and enforced in a fully-decentralized way. In particular, in spite of scaling actions being triggered concurrently, each operator maintains a view of its neighbours in the graph so as no data message is lost. The protocol is detailed and its correctness discussed. Its performance is captured through early simulation experiments.
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download
Contributor : Cédric Tedeschi Connect in order to contact the contributor
Submitted on : Tuesday, July 2, 2019 - 4:50:57 PM
Last modification on : Wednesday, November 3, 2021 - 8:15:46 AM


Files produced by the author(s)


  • HAL Id : hal-02171172, version 1


Mehdi Belkhiria, Cédric Tedeschi. A Fully Decentralized Autoscaling Algorithm for Stream Processing Applications. Auto-DaSP 2019 - Third International Workshop on Autonomic Solutions for Parallel and Distributed Data Stream Processing, Aug 2019, Göttingen, Germany. pp.1-12. ⟨hal-02171172⟩



Les métriques sont temporairement indisponibles