Skip to Main content Skip to Navigation
Conference papers

Use of language models for document stream segmentation

Abstract : Page stream segmentation into single documents is a very common task which is practiced in companies and administrations when processing their incoming mail. It is not a straightforward task because the limits of the documents are not always obvious, and it is not always easy to find common features between the pages of the same document. In this paper, we seek to compare existing segmentation models and propose a new segmentation one based on GRUs (Gated Recurrent Unit) and an attention mechanism, named AGRU. This model uses the text content of the previous page and the current page to determine if both pages belong to the same document. So, due to its attention mechanism, this model is capable to recognize words that define the first page of a document. Training and evaluation are carried out on two datasets: Tobacco-800 and READ-Corpus. The former is a public dataset on which our model reaches an F1 score equal to 90%, and the later is private for which our model reaches an F1 score equal to 96%.
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.inria.fr/hal-02975046
Contributor : Yolande Belaid <>
Submitted on : Thursday, October 22, 2020 - 12:34:21 PM
Last modification on : Monday, October 26, 2020 - 10:56:47 AM

File

version_soumise.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02975046, version 1

Collections

Citation

Chems Neche, Yolande Belaïd, Abdel Belaïd. Use of language models for document stream segmentation. International Conference on Pattern Recognition Applications and Methods, Feb 2020, Valleta, Malta. ⟨hal-02975046⟩

Share

Metrics

Record views

17

Files downloads

92