Argumentation Quality Assessment: an Argument Mining Approach - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Argumentation Quality Assessment: an Argument Mining Approach

Résumé

Argumentation is used by people both internally, by evaluating arguments and counterarguments to make a decision, and externally, e.g., by exchanging arguments to reach an agreement or to promote a position. A major component of the argumentation process concerns the assessment of a set of arguments and of their conclusions in order to establish their justification status, and therefore compute their acceptability degree. The assessment of the justification status of the statements supported by arguments allows the agent to decide what to believe and what to do. Argumentation semantics provide formal criteria to determine which sets of arguments (i.e., extensions) can be regarded as collectively acceptable (Baroni, Caminada, and Giacomin 2011). However, the assessment of the arguments acceptability is only a (basic) part of the complex assessment tasks required in argumentative processes in many everyday life applications, e.g., in medicine and education. The issue of assessing an argumentation is particularly critical when considering the different aspects of artificial argumentation, from the identification of real natural language arguments and their relations in text, to the computation of the justification status of abstract arguments, to the gradual assessment of arguments. Despite some approaches addressing the automatic assessment of natural language arguments (Wachsmuth et al. 2017, 2020; Saveleva et al. 2021), this issue remains largely unsolved. In this paper, we address this open issue and we answer the following research question: what are the basic quality dimensions to characterize natural language argumentation and how to automatically assess them? More precisely, we propose an Argument Mining (AM) approach to identify and classify natural language arguments along with quality dimensions. In artificial argumentation, Argument(ation) Mining aims at extracting arguments from text and at analyzing them (Cabrio and Villata 2018). In this paper, we decide to characterize argument quality along with three quality dimensions for natural language argumentation, i.e., cogency, rhetoric and reasonableness. Cogency estimates the acceptability of the premises that are relevant to the argument's conclusion and their sufficiency to draw the conclusion, rhetoric determines the rhetorical strategy employed in the argument's conclusion (if any) from the three options of ethos, logos and pathos, and reasonableness rates if the argument adequately rebuts its counterarguments, assessing therefore the dialectical quality dimension of the argumentation. Our interest focuses on the education scenario, where students are asked to interact with our AM system to assess the quality of their persuasive essays with respect to these three quality dimensions. To train our AM model, we annotated an existing dataset of 402 student persuasive essays (Stab and Gurevych 2017) with these quality dimensions. We then propose a new deep learning AM method based on a transformer architecture, exploiting the structure of the argumentation graph through graph embeddings. Our approach addresses in an automatic way the evaluation process proposed in social science by Stapleton and Wu (2015), through a scoring rubric for persuasive writing that integrates the assessment of both argumentative structural elements and reasoning quality. The obtained results are satisfactory and outperform standard baselines and similar approaches in the literature.
Fichier principal
Vignette du fichier
ECA2022_template_Word.pdf (532.83 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03934466 , version 1 (28-08-2023)

Identifiants

  • HAL Id : hal-03934466 , version 1

Citer

Santiago Marro, Elena Cabrio, Serena Villata. Argumentation Quality Assessment: an Argument Mining Approach. ECA 2022 - European conference on argumentation, Oct 2022, Rome, Italy. ⟨hal-03934466⟩
60 Consultations
72 Téléchargements

Partager

Gmail Facebook X LinkedIn More