Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, Epiciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Conference papers

KANDINSKY Patterns as IQ-Test for Machine Learning

Abstract : AI follows the notion of human intelligence which is unfortunately not a clearly defined term. The most common definition given by cognitive science as mental capability, includes, among others, the ability to think abstract, to reason, and to solve problems from the real world. A hot topic in current AI/machine learning research is to find out whether and to what extent algorithms are able to learn abstract thinking and reasoning similarly as humans can do – or whether the learning outcome remains on purely statistical correlation. In this paper we provide some background on testing intelligence, report some preliminary results from 271 participants of our online study on explainability, and propose to use our Kandinsky Patterns as an IQ-Test for machines. Kandinsky Patterns are mathematically describable, simple, self-contained hence controllable test data sets for the development, validation and training of explainability in AI. Kandinsky Patterns are at the same time easily distinguishable from human observers. Consequently, controlled patterns can be described by both humans and computers. The results of our study show that the majority of human explanations was made based on the properties of individual elements in an image (i.e., shape, color, size) and the appearance of individual objects (number). Comparisons of elements (e.g., more, less, bigger, smaller, etc.) were significantly less likely and the location of objects, interestingly, played almost no role in the explanation of the images. The next step is to compare these explanations with machine explanations.
Document type :
Conference papers
Complete list of metadata

Cited literature [53 references]  Display  Hide  Download

https://hal.inria.fr/hal-02520058
Contributor : Hal Ifip Connect in order to contact the contributor
Submitted on : Thursday, March 26, 2020 - 1:51:34 PM
Last modification on : Tuesday, March 31, 2020 - 3:50:26 PM
Long-term archiving on: : Saturday, June 27, 2020 - 2:20:45 PM

File

485369_1_En_1_Chapter.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Andreas Holzinger, Michael Kickmeier-Rust, Heimo Müller. KANDINSKY Patterns as IQ-Test for Machine Learning. 3rd International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Aug 2019, Canterbury, United Kingdom. pp.1-14, ⟨10.1007/978-3-030-29726-8_1⟩. ⟨hal-02520058⟩

Share

Metrics

Record views

99

Files downloads

24