Skip to Main content Skip to Navigation
Conference papers

A New Approach to Determine the Optimal Number of Clusters Based on the Gap Statistic

Abstract : Data clustering is one of the most important unsupervised classification method. It aims at organizing objects into groups (or clusters), in such a way that members in the same cluster are similar in some way and members belonging to different cluster are distinctive. Among other general clustering method, k-means is arguably the most popular one. However, it still has some inherent weaknesses. One of the biggest challenges when using k-means is to determine the optimal number of clusters, k. Although many approaches have been suggested in the literature, this is still considered as an unsolved problem. In this study, we propose a new technique to improve the gap statistic approach for selecting k. It has been tested on different datasets, on which it yields superior results compared to the original gap statistic. We expect our new method to also work well on other clustering algorithms where the number k is required. This is because our new approach, like the gap statistic, can work with any clustering method.
Complete list of metadata

https://hal.inria.fr/hal-03266454
Contributor : Hal Ifip Connect in order to contact the contributor
Submitted on : Monday, June 21, 2021 - 5:31:20 PM
Last modification on : Friday, July 30, 2021 - 2:50:13 PM
Long-term archiving on: : Wednesday, September 22, 2021 - 7:01:06 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2023-01-01

Please log in to resquest access to the document

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Jaekyung yang, Jong-yeong Lee, Myoungjin Choi, yeongin Joo. A New Approach to Determine the Optimal Number of Clusters Based on the Gap Statistic. 2nd International Conference on Machine Learning for Networking (MLN), Dec 2019, Paris, France. pp.227-239, ⟨10.1007/978-3-030-45778-5_15⟩. ⟨hal-03266454⟩

Share

Metrics

Record views

27