An End-to-End Evaluation of Two Situated Dialog Systems.

Lina Maria Rojas Barahona 1 Alejandra Lorenzo 1 Claire Gardent 1
1 SYNALP - Natural Language Processing : representations, inference and semantics
LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : We present and evaluate two state-of-the art dialogue systems developed to support dialog with French speaking virtual characters in the context of a serious game: one hybrid statistical/symbolic and one purely statistical. We conducted a quantitative evaluation where we compare the accuracy of the interpreter and of the dialog manager used by each system; a user based evaluation based on 22 subjects using both the statistical and the hybrid system; and a corpus based evaluation where we examine such criteria as dialog coherence, dialog success, interpretation and generation errors in the corpus of Human-System interactions collected during the user-based evaluation. We show that although the statistical approach is slightly more robust, the hybrid strategy seems to be better at guiding the player through the game.
Document type :
Conference papers
Complete list of metadatas

Cited literature [25 references]  Display  Hide  Download

https://hal.inria.fr/hal-00726723
Contributor : Lina Maria Rojas Barahona <>
Submitted on : Friday, August 31, 2012 - 10:26:02 AM
Last modification on : Tuesday, December 18, 2018 - 4:38:01 PM
Long-term archiving on : Saturday, December 1, 2012 - 3:30:39 AM

File

emospeech_sigdial.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00726723, version 1

Collections

Citation

Lina Maria Rojas Barahona, Alejandra Lorenzo, Claire Gardent. An End-to-End Evaluation of Two Situated Dialog Systems.. Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Association for Computational Linguistics, Jul 2012, Seoul, North Korea. pp.10-19. ⟨hal-00726723⟩

Share

Metrics

Record views

583

Files downloads

343