A Comparison of Techniques for Sampling Web Pages

Eda Baykan 1 Monika Henzinger 1, 2 Stefan F. Keller 3 Sebastian De Castelberg 3 Markus Kinzler 3
1 EPFL - Ecole Polytechnique Fédérale de Lausanne
2 Google
3 University of Applied Science Rapperswil
Abstract : As the World Wide Web is growing rapidly, it is getting increasingly challenging to gather representative information about it. Instead of crawling the web exhaustively one has to resort to other techniques like sampling to determine the properties of the web. A uniform random sample of the web would be useful to determine the percentage of web pages in a specific language, on a topic or in a top level domain. Unfortunately, no approach has been shown to sample the web pages in an unbiased way. Three promising web sampling algorithms are based on random walks. They each have been evaluated individually, but making a comparison on different data sets is not possible. We directly compare these algorithms in this paper. We performed three random walks on the web under the same conditions and analyzed their outcomes in detail. We discuss the strengths and the weaknesses of each algorithm and propose improvements based on experimental results.
keyword : Random walks sampling web pages
Type de document :
Communication dans un congrès
Susanne Albers and Jean-Yves Marion. 26th International Symposium on Theoretical Aspects of Computer Science STACS 2009, Feb 2009, Freiburg, Germany. IBFI Schloss Dagstuhl, pp.13-30, 2009, Proceedings of the 26th Annual Symposium on the Theoretical Aspects of Computer Science
Liste complète des métadonnées

Littérature citée [13 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/inria-00359670
Contributeur : Publications Loria <>
Soumis le : lundi 9 février 2009 - 10:41:14
Dernière modification le : mardi 28 novembre 2017 - 16:50:01
Document(s) archivé(s) le : mardi 8 juin 2010 - 22:05:04

Fichiers

paper_new.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00359670, version 1
  • ARXIV : 0902.1604

Collections

STACS2009

Citation

Eda Baykan, Monika Henzinger, Stefan F. Keller, Sebastian De Castelberg, Markus Kinzler. A Comparison of Techniques for Sampling Web Pages. Susanne Albers and Jean-Yves Marion. 26th International Symposium on Theoretical Aspects of Computer Science STACS 2009, Feb 2009, Freiburg, Germany. IBFI Schloss Dagstuhl, pp.13-30, 2009, Proceedings of the 26th Annual Symposium on the Theoretical Aspects of Computer Science. 〈inria-00359670〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

90

Téléchargements de fichiers

587

Contact


archive-ouverte@inria.fr