, Data Never Sleeps

, Apache Spark Streaming Project

P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi et al., Apache flink: Stream and batch processing in a single engine, Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, vol.36, issue.4, 2015.

A. Toshniwal, S. Taneja, A. Shukla, K. Ramasamy, J. M. Patel et al., Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD'14, pp.147-156, 2014.

S. Kulkarni, N. Bhagat, M. Fu, V. Kedigehalli, C. Kellogg et al., Twitter heron: Stream processing at scale, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD'15, pp.239-250, 2015.

, Alibaba JStorm Project

, JStorm in Alibaba

N. Project,

B. Peng, M. Hosseini, Z. Hong, R. Farivar, and R. Campbell, Rstorm: Resource-aware scheduling in storm, Proceedings of the 16th ACM/IFIP/USENIX International Conference on Middleware, ser. Middleware'15, pp.149-161, 2015.

P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M. Welsh et al., Network-aware operator placement for stream-processing systems, Proceedings of the 22nd International Conference on Data Engineering, ser. ICDE'06, pp.49-49, 2006.

J. Wolf, N. Bansal, K. Hildrum, S. Parekh, D. Rajan et al.,

L. Wu and . Fleischer, Soda: An optimizing scheduler for large-scale stream-based distributed computer systems, Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware, ser. Middleware'08, pp.306-325, 2008.

Y. Liu, X. Shi, and H. Jin, Runtime-aware adaptive scheduling in stream processing, Concurrency and Computation: Practice and Experience, vol.28, issue.14, pp.3830-3843, 2016.

J. Xu, Z. Chen, J. Tang, and S. Su, T-storm: Traffic-aware online scheduling in storm, Proceedings of the 2014 IEEE 34th International Conference on Distributed Computing Systems, ser. ICDCS'14, pp.535-544, 2014.

D. Buntinas, B. Goglin, D. Goodell, G. Mercier, and S. Moreaud, Cache-efficient, intranode, large-message MPI communication with MPICH2-Nemesis, Proceedings of the 2009 International Conference on Parallel Processing, ser. ICPP'09, pp.462-469, 2009.
URL : https://hal.archives-ouvertes.fr/inria-00390064

H. Jin and D. K. Panda, Limic: Support for high-performance MPI intra-node communication on linux cluster, Proceedings of the 2005 International Conference on Parallel Processing, ser. ICPP'05, pp.184-191, 2005.

T. Ma, G. Bosilca, A. Bouteiller, and J. J. Dongarra, Locality and topology aware intra-node communication among multicore CPUs, Recent Advances in the Message Passing Interface, pp.265-274, 2010.

, Yahoo Streaming Benchmarks

S. Intel and . Benchmark,

M. Yang and R. T. Ma, Smooth task migration in apache storm, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD'15, pp.2067-2068, 2015.

L. Aniello, R. Baldoni, and L. Querzoni, Adaptive online scheduling in storm, Proceedings of the 7th ACM International Conference on Distributed Event-based Systems, ser. DEBS'13, pp.207-218, 2013.

J. D. Valois, Lock-free linked lists using compare-and-swap, Proceedings of the Fourteenth Annual ACM Symposium on Principles of Distributed Computing, pp.214-222, 1995.

S. Schneider, H. Andrade, B. Gedik, A. Biem, and K. Wu, Elastic scaling of data parallel operators in stream processing, Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing, ser. IPDPS'09, pp.1-12, 2009.

R. C. Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch, Integrating scale out and fault tolerance in stream processing using operator state management, Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD'13, pp.725-736, 2013.

R. C. Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch, Making state explicit for imperative big data processing, Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference, ser. USENIX ATC'14, pp.49-60, 2014.

T. Heinze, Z. Jerzak, G. Hackenbroich, and C. Fetzer, Latencyaware elastic scaling for distributed data stream processing systems, Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ser. DEBS'14, pp.13-22, 2014.

, Storm Throughput Test

, Apache Hadoop Project

N. S. Islam, M. W. Rahman, J. Jose, R. Rajachandrasekar, H. Wang et al., High performance RDMA-based design of HDFS over infiniband, Proceedings of the 2012 International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC'12, pp.1-12, 2012.

Y. Wang, C. Xu, X. Li, and W. Yu, JVM-bypass for efficient hadoop shuffling, Proceedings of 2013 IEEE 27th International Symposium on Parallel and Distributed Processing, pp.569-578, 2013.

D. Buntinas, G. Mercier, and W. Gropp, Design and evaluation of nemesis, a scalable, low-latency, message-passing communication subsystem, Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid, ser. CCGRID'06
URL : https://hal.archives-ouvertes.fr/hal-00344350

D. C. Washington and . Usa, , pp.521-530, 2006.

W. Gropp, E. Lusk, N. Doss, and A. Skjellum, A high-performance, portable implementation of the MPI message passing interface standard, Parallel Computing, vol.22, issue.6, pp.789-828, 1996.

T. Buddhika, R. Stern, K. Lindburg, K. Ericson, and S. Pallickara, Online scheduling and interference alleviation for low-latency, highthroughput processing of data streams, IEEE Transactions on Parallel and Distributed Systems, vol.28, issue.12, pp.3553-3569, 2017.

Z. Weng, Q. Guo, C. Wang, X. Meng, and B. He, Adastorm: Resource efficient storm with adaptive configuration, Proceedings of 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pp.1363-1364, 2017.