Skip to Main content Skip to Navigation
New interface
Conference papers

Benchmarking for syntax-based sentential inference

Paul Bedaride 1 Claire Gardent 1 
1 TALARIS - Natural Language Processing: representation, inference and semantics
Inria Nancy - Grand Est, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : We propose a methodology for investigat- ing how well NLP systems handle mean- ing preserving syntactic variations. We start by presenting a method for the semi automated creation of a benchmark where entailment is mediated solely by meaning preserving syntactic variations. We then use this benchmark to compare a seman- tic role labeller and two grammar based RTE systems. We argue that the proposed methodology (i) supports a modular eval- uation of the ability of NLP systems to handle the syntax/semantic interface and (ii) permits focused error mining and er- ror analysis.
Document type :
Conference papers
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download
Contributor : Paul Bedaride Connect in order to contact the contributor
Submitted on : Monday, November 15, 2010 - 9:44:09 AM
Last modification on : Friday, February 4, 2022 - 3:32:52 AM
Long-term archiving on: : Friday, October 26, 2012 - 3:40:19 PM


Files produced by the author(s)


  • HAL Id : inria-00536022, version 1



Paul Bedaride, Claire Gardent. Benchmarking for syntax-based sentential inference. The 23rd International Conference on Computational Linguistics - COLING 2010, Aug 2010, Beijing, China. ⟨inria-00536022⟩



Record views


Files downloads