Ordinal Regression for Interaction Quality Prediction

Abstract : The automatic prediction of the quality of a dialogue is useful to keep track of a spoken dialogue system’s performance and, if necessary, adapt its behaviour. Classifiers and regression models have been suggested to make this prediction. The parameters of these models are learnt from a corpus of dialogues evaluated by users or experts. In this paper, we propose to model this task as an ordinal regression problem. We apply support vector machines for ordinal regression on a corpus of dialogues where each system-user exchange was given a rate on a scale of 1 to 5 by experts. Compared to previous models proposed in the literature, the ordinal regression predictor has significantly better results according to the following evaluation metrics: Cohen’s agreement rate with experts ratings, Spearman’s rank correlation coefficient, and Euclidean and Manhattan errors.
Type de document :
Communication dans un congrès
39th International Conference on Acoustics, Speech and Signal Processing (ICASSP2014), May 2014, Florence, Italy. IEEE, pp.3221 - 3225, 〈10.1109/ICASSP.2014.6854195〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01107499
Contributeur : Olivier Pietquin <>
Soumis le : mardi 20 janvier 2015 - 18:29:39
Dernière modification le : jeudi 11 janvier 2018 - 06:21:19

Identifiants

Citation

Layla El Asri, Hatim Khouzaimi, Romain Laroche, Olivier Pietquin. Ordinal Regression for Interaction Quality Prediction. 39th International Conference on Acoustics, Speech and Signal Processing (ICASSP2014), May 2014, Florence, Italy. IEEE, pp.3221 - 3225, 〈10.1109/ICASSP.2014.6854195〉. 〈hal-01107499〉

Partager

Métriques

Consultations de la notice

127