Streaming, Memory Limited Algorithms for Community Detection

Abstract : In this paper, we consider sparse networks consisting of a finite number of non-overlapping communities, i.e. disjoint clusters, so that there is higher density within clusters than across clusters. Both the intra- and inter-cluster edge densities vanish when the size of the graph grows large, making the cluster reconstruction problem nosier and hence difficult to solve. We are interested in scenarios where the network size is very large, so that the adjacency matrix of the graph is hard to manipulate and store. The data stream model in which columns of the adjacency matrix are revealed sequentially constitutes a natural framework in this setting. For this model, we develop two novel clustering algorithms that extract the clusters asymptotically accurately. The first algorithm is {\it offline}, as it needs to store and keep the assignments of nodes to clusters, and requires a memory that scales linearly with the network size. The second algorithm is {\it online}, as it may classify a node when the corresponding column is revealed and then discard this information. This algorithm requires a memory growing sub-linearly with the network size. To construct these efficient streaming memory-limited clustering algorithms, we first address the problem of clustering with partial information, where only a small proportion of the columns of the adjacency matrix is observed and develop, for this setting, a new spectral algorithm which is of independent interest.
Type de document :
Communication dans un congrès
NIPS 2014, Dec 2014, Montreal, Canada
Liste complète des métadonnées

https://hal.inria.fr/hal-01090818
Contributeur : Marc Lelarge <>
Soumis le : jeudi 4 décembre 2014 - 11:26:34
Dernière modification le : jeudi 22 novembre 2018 - 14:11:19

Lien texte intégral

Identifiants

  • HAL Id : hal-01090818, version 1
  • ARXIV : 1411.1279

Citation

Se-Young Yun, Marc Lelarge, Alexandre Proutière. Streaming, Memory Limited Algorithms for Community Detection. NIPS 2014, Dec 2014, Montreal, Canada. 〈hal-01090818〉

Partager

Métriques

Consultations de la notice

402