Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora

Fabrice Lefevre; Djamel Mostefa; Laurent Besacier; Yannick Estève; Matthieu Quignard; Nathalie Camelin; Benoit Favre; Bassam Jabaian; Lina Maria Rojas Barahona

hal-00683433, version 1

Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora

Fabrice Lefevre ¹, Djamel Mostefa^a^, 2, Laurent Besacier^b^, 3, Yannick Estève ⁴, Matthieu Quignard ⁵, Nathalie Camelin^c^, 4, Benoit Favre^d^, 6, Bassam Jabaian^e^, 1, 3, Lina Maria Rojas Barahona (, ) ⁵

The International Conference on Language Resources and Evaluation (2012)

Résumé : The PORTMEDIA project is intended to develop new corpora for the evaluation of spoken language understanding systems. The newly collected data are in the field of human-machine dialogue systems for tourist information in French in line with the MEDIA corpus. Transcriptions and semantic annotations, obtained by low-cost procedures, are provided to allow a thorough evaluation of the systems' capabilities in terms of robustness and portability across languages and domains. A new test set with some adaptation data is prepared for each case: in Italian as an example of a new language, for ticket reservation as an example of a new domain. Finally the work is complemented by the proposition of a new high level semantic annotation scheme well-suited to dialogue data.

a – Evaluation and Language resources Distribution Agency
b – LIG, Grenoble, France
c – LIUM
d – LIF, Marseille, France
e – LIG
1 : Laboratoire Informatique d'Avignon (LIA)
Université d'Avignon – Centre d'Enseignement et de Recherche en Informatique - CERI
2 : Evaluations and Language resources Distribution Agency (ELDA)
ELDA
3 : Laboratoire d'Informatique de Grenoble (LIG)
Université Joseph Fourier - Grenoble I – Institut polytechnique de Grenoble (Grenoble INP) – Université Pierre-Mendès-France - Grenoble II – CNRS : UMR5217
4 : Laboratoire d'Informatique de l'Université du Maine (LIUM)
Université du Maine : EA4023
5 : SYNALP (LORIA)
Université de Lorraine – CNRS : UMR7503 – INRIA
6 : Laboratoire d'informatique Fondamentale de Marseille (LIF)
CNRS : UMR6166 – Université de la Méditerranée - Aix-Marseille II – Université de Provence - Aix-Marseille I

hal-00683433, version 1
http://hal.inria.fr/hal-00683433
oai:hal.inria.fr:hal-00683433
Contributeur : Lina Maria Rojas Barahona <>
Soumis le : Mercredi 28 Mars 2012, 16:53:42
Dernière modification le : Mercredi 18 Avril 2012, 08:35:39

Voir la fiche détaillée

Documents associés

PDF :

Exporter

Bibtex EndNote TEI RefWorks

%% hal-00683433, version 1
%% http://hal.inria.fr/hal-00683433
@inproceedings{lefevre:hal-00683433,
    hal_id = {hal-00683433},
    url = {http://hal.inria.fr/hal-00683433},
    title = {{Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora}},
    author = {Lefevre, Fabrice and Mostefa, Djamel and Besacier, Laurent and Est{\`e}ve, Yannick and Quignard, Matthieu and Camelin, Nathalie and Favre, Benoit and Jabaian, Bassam and Rojas Barahona, Lina Maria},
    abstract = {{The PORTMEDIA project is intended to develop new corpora for the evaluation of spoken language understanding systems. The newly collected data are in the field of human-machine dialogue systems for tourist information in French in line with the MEDIA corpus. Transcriptions and semantic annotations, obtained by low-cost procedures, are provided to allow a thorough evaluation of the systems' capabilities in terms of robustness and portability across languages and domains. A new test set with some adaptation data is prepared for each case: in Italian as an example of a new language, for ticket reservation as an example of a new domain. Finally the work is complemented by the proposition of a new high level semantic annotation scheme well-suited to dialogue data.}},
    keywords = {spoken language understanding system; human-machine dialogue corpus; portability},
    language = {Anglais},
    affiliation = {Laboratoire Informatique d'Avignon - LIA , Evaluations and Language resources Distribution Agency - ELDA , Laboratoire d'Informatique de Grenoble - LIG , Laboratoire d'Informatique de l'Universit{\'e} du Maine - LIUM , SYNALP - LORIA , Laboratoire d'informatique Fondamentale de Marseille - LIF},
    booktitle = {{The International Conference on Language Resources and Evaluation}},
    address = {Istanbul, Turquie},
    audience = {internationale },
    year = {2012},
    month = May,
    pdf = {http://hal.inria.fr/hal-00683433/PDF/751\_Paper.pdf},
}

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="inproceedings"><head>Communications avec actes</head>
<biblStruct type="inproceedings" xml:id="hal-00683433"><analytic>
<author><persName><forename>Fabrice</forename><surname>Lefevre</surname></persName>
<affiliation>
<orgName type="laboratory">LIA</orgName>
<orgName type="team">Laboratoire Informatique d'Avignon</orgName>
<country>FR</country>
<orgName type="institution">Université d'Avignon</orgName>
<orgName type="institution">Centre d'Enseignement et de Recherche en Informatique - CERI</orgName>
</affiliation>
</author>
<author><persName><forename>Djamel</forename><surname>Mostefa</surname></persName>
<affiliation>
<orgName type="laboratory">ELDA</orgName>
<orgName type="team">Evaluations and Language resources Distribution Agency</orgName>
<country>FR</country>
<orgName type="institution">ELDA</orgName>
</affiliation>
</author>
<author><persName><forename>Laurent</forename><surname>Besacier</surname></persName>
<affiliation>
<orgName type="laboratory">LIG</orgName>
<orgName type="team">Laboratoire d'Informatique de Grenoble</orgName>
<country>FR</country>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut polytechnique de Grenoble (Grenoble INP)</orgName>
<orgName type="institution">Université Pierre-Mendès-France - Grenoble II</orgName>
<orgName type="institution">CNRS : UMR5217</orgName>
</affiliation>
</author>
<author><persName><forename>Yannick</forename><surname>Estève</surname></persName>
<affiliation>
<orgName type="laboratory">LIUM</orgName>
<orgName type="team">Laboratoire d'Informatique de l'Université du Maine</orgName>
<country>FR</country>
<orgName type="institution">Université du Maine : EA4023</orgName>
</affiliation>
</author>
<author><persName><forename>Matthieu</forename><surname>Quignard</surname></persName>
<affiliation>
<orgName type="laboratory">LORIA</orgName>
<orgName type="team">SYNALP</orgName>
<country>FR</country>
<orgName type="institution">Université de Lorraine</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">INRIA</orgName>
</affiliation>
</author>
<author><persName><forename>Nathalie</forename><surname>Camelin</surname></persName>
<affiliation>
<orgName type="laboratory">LIUM</orgName>
<orgName type="team">Laboratoire d'Informatique de l'Université du Maine</orgName>
<country>FR</country>
<orgName type="institution">Université du Maine : EA4023</orgName>
</affiliation>
</author>
<author><persName><forename>Benoit</forename><surname>Favre</surname></persName>
<affiliation>
<orgName type="laboratory">LIF</orgName>
<orgName type="team">Laboratoire d'informatique Fondamentale de Marseille</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR6166</orgName>
<orgName type="institution">Université de la Méditerranée - Aix-Marseille II</orgName>
<orgName type="institution">Université de Provence - Aix-Marseille I</orgName>
</affiliation>
</author>
<author><persName><forename>Bassam</forename><surname>Jabaian</surname></persName>
<affiliation>
<orgName type="laboratory">LIA</orgName>
<orgName type="team">Laboratoire Informatique d'Avignon</orgName>
<country>FR</country>
<orgName type="institution">Université d'Avignon</orgName>
<orgName type="institution">Centre d'Enseignement et de Recherche en Informatique - CERI</orgName>
</affiliation>
<affiliation>
<orgName type="laboratory">LIG</orgName>
<orgName type="team">Laboratoire d'Informatique de Grenoble</orgName>
<country>FR</country>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut polytechnique de Grenoble (Grenoble INP)</orgName>
<orgName type="institution">Université Pierre-Mendès-France - Grenoble II</orgName>
<orgName type="institution">CNRS : UMR5217</orgName>
</affiliation>
</author>
<author><persName><forename>Lina Maria</forename><surname>Rojas Barahona</surname></persName>
<affiliation>
<orgName type="laboratory">LORIA</orgName>
<orgName type="team">SYNALP</orgName>
<country>FR</country>
<orgName type="institution">Université de Lorraine</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">INRIA</orgName>
</affiliation>
</author>
<title type="main" level="a">Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora</title></analytic><monogr><meeting>The International Conference on Language Resources and Evaluation</meeting><imprint><pubPlace><settlement>Istanbul</settlement><country>Turquie</country></pubPlace><biblScope type="publicationDate">2012-05-23</biblScope></imprint></monogr><idno type="HALid">http://hal.inria.fr/hal-00683433</idno><idno type="HALFile">http://hal.inria.fr/hal-00683433/PDF/751_Paper.pdf</idno></biblStruct>
</listBibl>
</listBibl>