Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems - Archive ouverte HAL Access content directly
Conference Papers Year :

Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems

(1) , (1) , (1) , (1) , (2) , (1)
1
2

Abstract

Stream Processing Engines (SPEs) have to support high data ingestion to ensure the quality and efficiency for the end-user or a system administrator. The data flow processed by SPE fluctuates over time, and requires real-time or near real-time resource pool adjustments (network, memory, CPU and other). This scenario leads to the problem known as skewed data production caused by the non-uniform incoming flow at specific points on the environment, resulting in slow down of applications caused by network bottlenecks and inefficient load balance. This work proposes Aten as a solution to overcome unbalanced data flows processed by Big Data Stream applications in heterogeneous systems. Aten manages data aggregation and data streams within message queues, assuming different algorithms as strategies to partition data flow over all the available computational resources. The paper presents preliminary results indicating that is possible to maximize the throughput and also provide low latency levels for SPEs.
Fichier principal
Vignette du fichier
Aten_HPCS_18.pdf (515.92 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01876973 , version 1 (19-09-2018)

Identifiers

Cite

Paulo de Souza Junior, Kassiano J Matteussi, Julio C S dos Anjos, Jobe D D Dos Santos, Alexandre da Silva Veith, et al.. Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems. 2018 International Conference on High Performance Computing Simulation (HPCS), Jul 2018, Orléans, France. pp.585-592, ⟨10.1109/HPCS.2018.00098⟩. ⟨hal-01876973⟩
213 View
204 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More