Skip to Main content Skip to Navigation
Conference papers

KANDINSKY Patterns as IQ-Test for Machine Learning

Abstract : AI follows the notion of human intelligence which is unfortunately not a clearly defined term. The most common definition given by cognitive science as mental capability, includes, among others, the ability to think abstract, to reason, and to solve problems from the real world. A hot topic in current AI/machine learning research is to find out whether and to what extent algorithms are able to learn abstract thinking and reasoning similarly as humans can do – or whether the learning outcome remains on purely statistical correlation. In this paper we provide some background on testing intelligence, report some preliminary results from 271 participants of our online study on explainability, and propose to use our Kandinsky Patterns as an IQ-Test for machines. Kandinsky Patterns are mathematically describable, simple, self-contained hence controllable test data sets for the development, validation and training of explainability in AI. Kandinsky Patterns are at the same time easily distinguishable from human observers. Consequently, controlled patterns can be described by both humans and computers. The results of our study show that the majority of human explanations was made based on the properties of individual elements in an image (i.e., shape, color, size) and the appearance of individual objects (number). Comparisons of elements (e.g., more, less, bigger, smaller, etc.) were significantly less likely and the location of objects, interestingly, played almost no role in the explanation of the images. The next step is to compare these explanations with machine explanations.
Document type :
Conference papers
Complete list of metadata

Cited literature [53 references]  Display  Hide  Download

https://hal.inria.fr/hal-02520058
Contributor : Hal Ifip <>
Submitted on : Thursday, March 26, 2020 - 1:51:34 PM
Last modification on : Tuesday, March 31, 2020 - 3:50:26 PM
Long-term archiving on: : Saturday, June 27, 2020 - 2:20:45 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2022-01-01

Please log in to resquest access to the document

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Andreas Holzinger, Michael Kickmeier-Rust, Heimo Müller. KANDINSKY Patterns as IQ-Test for Machine Learning. 3rd International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Aug 2019, Canterbury, United Kingdom. pp.1-14, ⟨10.1007/978-3-030-29726-8_1⟩. ⟨hal-02520058⟩

Share

Metrics

Record views

97