Skip to Main content Skip to Navigation
Journal articles

SOAP: Semantic Outliers Automatic Preprocessing

Leonardo Trujillo 1 Uriel Lopez 1 Pierrick Legrand 2
2 CQFD - Quality control and dynamic reliability
IMB - Institut de Mathématiques de Bordeaux, Inria Bordeaux - Sud-Ouest
Abstract : Genetic Programming (GP) is an evolutionary algorithm for the automatic generation of symbolic models expressed as syntax trees. GP has been successfully applied in many domain, but most research in this area has not considered the presence of outliers in the training set. Outliers make supervised learning problems difficult, and sometimes impossible, to solve. For instance, robust regression methods cannot handle more than 50% of outlier contamination, referred to as their breakdown point. This paper studies problems where outlier contamination is high, reaching up to 90% contamination levels, extreme cases that can appear in some domains. This work shows, for the first time, that a random population of GP individuals can detect outliers in the output variable. From this property, a new filtering algorithm is proposed called Semantic Outlier Automatic Preprocessing (SOAP), which can be used with any learning algorithm to differentiate between inliers and outliers. Since the method uses a GP population, the algorithm can be carried out for free in a GP symbolic regression system. The approach is the only method that can perform such an automatic cleaning of a dataset without incurring an exponential cost as the percentage of outliers in the dataset increases.
Document type :
Journal articles
Complete list of metadatas
Contributor : Pierrick Legrand <>
Submitted on : Wednesday, April 22, 2020 - 5:21:03 PM
Last modification on : Wednesday, October 28, 2020 - 3:28:00 PM





Leonardo Trujillo, Uriel Lopez, Pierrick Legrand. SOAP: Semantic Outliers Automatic Preprocessing. Information Sciences, Elsevier, 2020, 526 (81-101), pp.20. ⟨10.1016/j.ins.2020.03.071⟩. ⟨hal-02551161⟩



Record views


Données de recherche

doi: web.