Skip to Main content Skip to Navigation
Conference papers

Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems

Abstract : Stream Processing Engines (SPEs) have to support high data ingestion to ensure the quality and efficiency for the end-user or a system administrator. The data flow processed by SPE fluctuates over time, and requires real-time or near real-time resource pool adjustments (network, memory, CPU and other). This scenario leads to the problem known as skewed data production caused by the non-uniform incoming flow at specific points on the environment, resulting in slow down of applications caused by network bottlenecks and inefficient load balance. This work proposes Aten as a solution to overcome unbalanced data flows processed by Big Data Stream applications in heterogeneous systems. Aten manages data aggregation and data streams within message queues, assuming different algorithms as strategies to partition data flow over all the available computational resources. The paper presents preliminary results indicating that is possible to maximize the throughput and also provide low latency levels for SPEs.
Complete list of metadatas

Cited literature [42 references]  Display  Hide  Download

https://hal.inria.fr/hal-01876973
Contributor : Alexandre da Silva Veith <>
Submitted on : Wednesday, September 19, 2018 - 10:01:58 AM
Last modification on : Wednesday, November 20, 2019 - 2:29:54 AM

File

Aten_HPCS_18.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Paulo de Souza Junior, Kassiano Matteussi, Julio dos Anjos, Jobe Santos, Alexandre da Silva Veith, et al.. Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems. 2018 International Conference on High Performance Computing Simulation (HPCS), Jul 2018, Orléans, France. pp.585-592, ⟨10.1109/HPCS.2018.00098⟩. ⟨hal-01876973⟩

Share

Metrics

Record views

1085

Files downloads

325