Skip to Main content Skip to Navigation
Journal articles

On the Correlation of Automatic Audio and Visual Segmentations of Music Videos

Abstract : The study of the associations between audio and video content has numerous important applications in the fields of information retrieval and multimedia content authoring. In this work, we focus on music videos which exhibit a broad range of structural and semantic relationships between the music and the video content. To identify such relationships, a two-level automatic struc-turing of the music and the video is achieved separately. Note onsets are detected from the music signal, along with section changes. The latter is achieved by a novel algorithm which makes use of feature selection and statistical novelty detection approaches based on kernel methods. The video stream is independently segmented to detect changes in motion activity, as well as shot boundaries. Based on this two-level segmentation of both streams, four audiovisual correlation measures are computed. The usefulness of these correlation measures is illustrated by a query by video experiment on a 100 music video database, which also exhibits interesting genre dependencies.
Document type :
Journal articles
Complete list of metadata

https://hal.inria.fr/hal-02652635
Contributor : Gaël Richard <>
Submitted on : Friday, May 29, 2020 - 7:51:53 PM
Last modification on : Wednesday, September 30, 2020 - 8:54:15 AM

Identifiers

Collections

Citation

Olivier Gillet, Slim Essid, Gael Richard. On the Correlation of Automatic Audio and Visual Segmentations of Music Videos. IEEE Transactions on Circuits and Systems for Video Technology, Institute of Electrical and Electronics Engineers, 2007, 17, ⟨10.1109/TCSVT.2007.890831⟩. ⟨hal-02652635⟩

Share

Metrics

Record views

43