HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province

Abstract : Aiming at the demand of real-time massive data processing of Intelligent Agriculture in Jilin Province, this paper studies the big data processing of Intelligent Agriculture in Jilin Province based on Spark platform by acquiring real-time data through monitoring platform. This study first conducted the performance comparison experiment of Hadoop and Spark data processing platform, then used the Spark distributed cluster computing platform, real-time processing the big data of monitoring area. The experimental results show that the Spark platform speeds up 11.4 times faster than the Hadoop platform in the case of 100 million data sizes; and based on the Spark platform for real-time processing of big data intelligent agricultural monitoring network, not only provides memory calculations to reduce IO overhead, but also the results are faster and more accurate. The research results provide strong support for the implementation of precision agriculture technology in intelligent agriculture.
Document type :
Conference papers
Complete list of metadata

Cited literature [22 references]  Display  Hide  Download

https://hal.inria.fr/hal-02111532
Contributor : Hal Ifip Connect in order to contact the contributor
Submitted on : Friday, April 26, 2019 - 9:59:14 AM
Last modification on : Friday, April 26, 2019 - 10:47:29 AM

File

478293_1_En_1_Chapter.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Siwei Fu, Guifen Chen, Shan Zhao, Enze Xiao. Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province. 11th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Aug 2017, Jilin, China. pp.1-12, ⟨10.1007/978-3-030-06179-1_1⟩. ⟨hal-02111532⟩

Share

Metrics

Record views

40

Files downloads

7