Skip to Main content Skip to Navigation
New interface
Master thesis

Mitigating Unintended Bias in Masked Language Models

Mohammed Rameez Rameez Qureshi 1, 2, 3 
2 ORPAILLEUR - Knowledge representation, reasonning
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
3 LACODAM - Large Scale Collaborative Data Mining
Inria Rennes – Bretagne Atlantique , IRISA-D7 - GESTION DES DONNÉES ET DE LA CONNAISSANCE
Abstract : Algorithmic fairness is currently one of the most debated topics in artificial intelligence. Primarily because of its massive importance and critical impact on various aspects of human lives. One of its highly discussed drawbacks is the unintended bias we observe in complex machine learning models, and that carries adverse effects on various fields ranging from healthcare to legal policing. This work focused on devising novel methodologies for mitigating unintended bias found in language models. Our work mainly concerned the gender bias associated with occupations. However, it can be extended to other kinds such as demographic, racial, etc. As the definition of bias is subjective and varies case by case, it is challenging to measure and reduce biases in pre-trained language models. This work proposes an advanced architecture based on Deep Reinforcement Learning to mitigate unintended bias in pre-trained language models. The proposed architecture tackles withstanding challenges without compromising performance and defines a system that is portable and easy to adapt. The thesis is organised as follows. After defining the problem statement in Chapter 1, we lay down the pre-requisites required for this study in Chapter 2. In the following two chapters, we detail the proposed architecture and the experimental setup used for evaluation. Chapter 5 comprises a discussion based on the results and possible explanations of the model’s behaviour. Finally, in Chapter 6, we discuss the limitations as well as promising perspectives for future work.
Document type :
Master thesis
Complete list of metadata

https://hal.inria.fr/hal-03363353
Contributor : Mohammed Rameez Qureshi Connect in order to contact the contributor
Submitted on : Sunday, October 10, 2021 - 1:29:57 PM
Last modification on : Thursday, September 1, 2022 - 3:57:10 AM

File

IDMC_thesis.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03363353, version 1

Citation

Mohammed Rameez Rameez Qureshi. Mitigating Unintended Bias in Masked Language Models. Computer Science [cs]. 2021. ⟨hal-03363353⟩

Share

Metrics

Record views

117

Files downloads

125