Skip to Main content Skip to Navigation
Conference papers

Scalable Joint Optimization of Placement and Parallelism of Data Stream Processing Applications on Cloud-Edge Infrastructure

Abstract : The Internet of Things has enabled many application scenarios where a large number of connected devices generate unbounded streams of data, often processed by data stream processing frameworks deployed in the cloud. Edge computing enables offloading processing from the cloud and placing it close to where the data is generated, whereby reducing both the time to process data events and deployment costs. However , edge resources are more computationally constrained than their cloud counterparts. This gives rise to two interrelated issues, namely deciding on the parallelism of processing tasks (a.k.a. operators) and their mapping onto available resources. In this work, we formulate the scenario of operator placement and parallelism as an optimal mixed integer linear programming problem. To overcome the issue of scalability with the optimal model, we devise a resource selection technique that reduces the number of resources evaluated during placement and parallelization decisions. Experimental results using discrete-event simulation demonstrate that the proposed model coupled with the resource selection technique is 94% faster than solving the optimal model alone, and it produces solutions that are only 12% worse than the optimal, yet it performs better than state-of-the-art approaches.
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download

https://hal.inria.fr/hal-02926464
Contributor : Felipe Rodrigo de Souza <>
Submitted on : Monday, August 31, 2020 - 5:49:33 PM
Last modification on : Thursday, September 24, 2020 - 3:53:44 PM
Long-term archiving on: : Tuesday, December 1, 2020 - 12:58:40 PM

File

icsoc2020_final.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02926464, version 1

Collections

Citation

Felipe Rodrigo de Souza, Alexandre da Silva Veith, Marcos Dias de Assuncao, Eddy Caron. Scalable Joint Optimization of Placement and Parallelism of Data Stream Processing Applications on Cloud-Edge Infrastructure. ICSOC 2020 - 18th International Conference on Service Oriented Computing, Dec 2020, Dubai, United Arab Emirates. ⟨hal-02926464⟩

Share

Metrics

Record views

163

Files downloads

187