HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph

Abstract : The rise of digitization of cultural documents offers large-scale contents, opening the road for development of AI systems in order to preserve, search, and deliver cultural heritage. To organize such cultural content also means to classify them, a task that is very familiar to modern computer science. Contextual information is often the key to structure such real world data, and we propose to use it in form of a knowledge graph. Such a knowledge graph, combined with content analysis, enhances the notion of proximity between artworks so it improves the performances in classification tasks. In this paper, we propose a novel use of a knowledge graph, that is constructed on annotated data and pseudo-labeled data. With label propagation, we boost artwork classification by training a model using a graph convolutional network, relying on the relationships between entities of the knowledge graph. Following a transductive learning framework, our experiments show that relying on a knowledge graph modeling the relations between labeled data and unlabeled data allows to achieve state-of-the-art results on multiple classification tasks on a dataset of paintings, and on a dataset of Buddha statues. Additionally, we show state-of-the-art results for the difficult case of dealing with unbalanced data, with the limitation of disregarding classes with extremely low degrees in the knowledge graph.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03228787
Contributor : Cheikh Brahim El Vaigh Connect in order to contact the contributor
Submitted on : Tuesday, May 18, 2021 - 1:42:12 PM
Last modification on : Monday, April 4, 2022 - 9:28:31 AM

File

GCNBoost-hal.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03228787, version 1

Citation

Cheikh Brahim El Vaigh, Noa Garcia, Benjamin Renoust, Chenhui Chu, Yuta Nakashima, et al.. GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph. ICMR 2021 - ACM International Conference on Multimedia Retrieval, Aug 2021, Taipei, Taiwan. ⟨hal-03228787⟩

Share

Metrics

Record views

63

Files downloads

84