Skip to Main content Skip to Navigation
Conference papers

An Efficient Data Indexing Approach on Hadoop Using Java Persistence API

Abstract : Data indexing is common in data mining when working with high-dimensional, large-scale data sets. Hadoop, a cloud computing project using the MapReduce framework in Java, has become of significant interest in distributed data mining. To resolve problems of globalization, random-write and duration in Hadoop, a data indexing approach on Hadoop using the Java Persistence API (JPA) is elaborated in the implementation of a KD-tree algorithm on Hadoop. An improved intersection algorithm for distributed data indexing on Hadoop is proposed, it performs O(M+logN), and is suitable for occasions of multiple intersections. We compare the data indexing algorithm on open dataset and synthetic dataset in a modest cloud environment. The results show the algorithms are feasible in large-scale data mining.
Document type :
Conference papers
Complete list of metadata

Cited literature [13 references]  Display  Hide  Download

https://hal.inria.fr/hal-01055056
Contributor : Hal Ifip <>
Submitted on : Monday, August 11, 2014 - 1:10:02 PM
Last modification on : Thursday, March 5, 2020 - 5:43:12 PM
Long-term archiving on: : Wednesday, November 26, 2014 - 9:55:58 PM

File

An_Efficient_Data_Indexing_App...
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Yang Lai, Shi Zhongzhi. An Efficient Data Indexing Approach on Hadoop Using Java Persistence API. 6th IFIP TC 12 International Conference on Intelligent Information Processing (IIP), Oct 2010, Manchester, United Kingdom. pp.213-224, ⟨10.1007/978-3-642-16327-2_27⟩. ⟨hal-01055056⟩

Share

Metrics

Record views

474

Files downloads

748