An Ontology to Formalize a Creative Problem Solving Activity

Our study is set in an educational context: to better teach and assess 21st-century skills, such as computational thinking or creative problem solving, we propose to formalize a specific activity that involves these competencies. This activity, referred to as #CreaCube, is presented as an open-ended problem which consists of assembling a set of robotic cubes into an autonomous vehicle. We not only anchor our formalization in classical learning science frameworks but we also propose to draw on neuro-cognitive models to describe the observed behaviors of learners engaged in this activity. The chosen formalism is symbolic and is aligned on upper ontologies to ensure that the vocabulary is well specified. This allows for a better communication between the summoned research fields, namely, learning science, cognitive neuroscience, and computational modeling. Beyond this specification purpose, we suggest performing inferences using available reasoners to better guide the analysis of the observables collected during the experiments. This operationalization of a creative problem-solving activity is part of an exploratory research action. In addition, an effective proof of concept is described in this study.


I. INTRODUCTION
We propose to formalize creative problem solving using symbolic knowledge representation to set up the basis of learner modeling in the precise context of a specific activity, by representing shareable and reusable knowledge across three domains: learning science, cognitive neuroscience and computational modeling.The task that is reviewed here is presented as an open-ended problem that aims to initiate the participants to computational thinking and invites them to solve a problem appealing to creativity using tangible artifacts.Let us first review the context and research objectives, to define the competencies we aim to better understand and justify the choice of the activity before highlighting the related challenges of such a formalization.
Educational Context: Transversal competencies, which are sometimes referred to as "twenty-first century skills", include (but are not limited to) creativity, problem solving and computational thinking [1].Integrating these skills in the K-12 curricula is a key challenge in today's learning science field [1], [2].We postulate that teaching and assessing transversal competencies could benefit from a better understanding of the learners' behaviors in specific activities that require these competencies.These skills are intrinsically related to each other, which leads to interdependent definitions.In our context, creativity refers to the process of producing something new and adequate with regard to the task being performed (e.g., a problem being solved) [3]- [5].Problem solving has been Chloé Mercier is with Mnemosyne, Inria, Bordeaux, France.
formalized at a computational level [6] and defined as finding a path from an initial state (the current situation) to a final state (the goal), while taking into account some constraints (e.g., task requirements).However, problem solving in everyday life takes many different forms, depending on the task and context.Realistic problems are most often ill-defined, as we will further clarify in this study.Computational thinking uses tools inspired by the field of computer science (e.g., algorithms, information coding and data representation) to (for example) solve problems or design systems [7] (see also [8] for a recent review).By extension, computational thinking also includes technology literacy.Furthermore, techno-creative activities [9] have the potential to develop computational thinking, while they should also engage learners in critical thinking [2].We have chosen to focus on an activity involving robotic artifacts, as developed in the sequel.
These transversal competencies also need to be considered in relation to (meta-)cognitive processes.The awareness of one's own thought processes during learning is key not only to solve ill-defined problems but also to be able to transfer skills from a given problem-solving task to another [10].Here, we hypothesize that better formalizing and understanding the learner upstream to meta-learning may contribute to better pedagogical resource design and help teach this critical awareness of one's own learning process.
Ill-defined Problem Solving: Research on problem solving has focused on a diversity of tasks but most of all on well-defined problems, such as the chess game or the Tower of Hanoi, which allows us to define task models and problem-solving methods.In short, the research objective in problem solving is to represent the problem space, including the positions of the initial state and of the desired goal (possibly a region of the space if the goal is not defined by a unique position but specified through instructions), and to define a path between the positions.This can be done in both forward and backward directions, where knowledge can provide rules for navigating the space while taking constraints into account, and requires exploration within this space when considering open problems, as already stated in the pioneering work of [6].This also can be solved at a numerical level which is biologically more plausible (see, e.g., [11]).Nonetheless, problem solving in everyday life is typically ill-defined in the sense that the goal is only partially defined or may have to be changed throughout the resolution, the initial state is only partially observable, and the problem constraints are discovered on the way towards the solution.Consequently, solving these problems requires us to consider not only prior knowledge but also the contextual knowledge that is built throughout the task process based on the different operations that the subject performs [12].Moreover, the exploration of the means given to solve the problem should facilitate the generation of new stimuli, which can then help the learner to make progress towards the goal by fulfilling sub-goals and validating or invalidating hypotheses.This is exactly what we would like to consider here.In particular, we aim to consider the problem-solving process instead of considering only performance scores of the task.Therefore, we need to take into account the set of knowledge required to solve a particular task and how it is managed.We hypothesize that ontology representation is a powerful and relatively accessible formalism to this end.
Learning with Tangible Objects: At a more concrete level, we aim to study creative problem solving through screen-less activities, also known as unplugged activities [13]; for instance, using tangible artifacts, such as pedagogical robots or connected objects [14], or even every day objects.This paradigm has demonstrated its efficiency, especially for computational thinking initiation [13].However, in contrast to online activities on a computer, a non-trivial setup is required to track the learning process through the activity.One of the challenges is the use of efficient measurement devices of reasonable cost.It also appears that a precise model of the task allows us to define specific observables that are thus more robust to estimate [14].Another obstacle is the bias that is induced by the fact that the learner is observed and monitored.One solution is to take this as a chance to involve the learner in their own learning process, which is the case with Open Learner Models [15].This is yet to be properly studied in this context, see [16] for a discussion on these points.Ethical issues of collecting human data in a way that ensures data privacy also needs to be considered, to ensure that only required data is effectively collected.For all these reasons, the construction of a precise model of the task and of the learner involved in this task is a lever to take up this challenge.Using tangible material allows us to observe behavioral patterns [14] which, if properly characterized, may help us infer cognitive processes.
The Observation Relevance Challenge: Properly characterizing observable behavioral patterns can be difficult because we are working on a relatively small batch of data (around a hundred individuals, compared to the thousands of data used in classical statistical methods).For the results to be meaningful, we propose to introduce prior information upstream to the analysis of the observables, which results in highly structured data.This has led us to choose ontology modeling, which is an appropriate tool to describe structured knowledge.This prior information can come from existing theoretical models in the learning sciences.
The Learner Modeling Challenges: This model is intended to be applicable to the observables and learning analytics, and to allow us to interpret them.Indeed, phenomenological models already exist -for example, the activity theory [17] provides a framework that is appropriate to the context of this study: it specifies an activity as performed by a subject through the mediation of a tool to achieve a certain outcome.Based on such phenomenological frameworks, our present study is positioned at an operational level.We also hypothesize that such an operationalization could benefit from a multi-disciplinary perspective and we thus draw on behavioral models taken from neuro-cognitive science.
The finality of developing a computational model of a computational thinking task [1] aims to contribute, as a longterm perspective, to a better understanding, at a very concrete level, of the fundamental notions regarding natural versus artificial so-called intelligence, as studied in [2], including a better understanding of our human intelligence.We focus on a specific learning task instead of considering human learning at a general level to offer a precise and operational formalization that is translatable to other learning tasks.
This paper is an extension of a work that was originally reported in Proceedings of the 2021 IEEE International Conference on Development and Learning (ICDL) [18].

II. RELATED WORK
Ontologies to Model Learning Activities: A recent systematic review [19] of ontology use in education has observed a growing trend in the contribution of ontologies to educational systems in the last five years, and has identified the following common use cases: • Describing learning domains, • Describing learner data, • Modeling and managing curricula, • Describing learning services.In our case, we are particularly interested in works that attempt to model both the learning domain and the learner data.This has been done especially in intelligent tutoring systems and adaptive learning systems, as reviewed in [20], where semantic inference is performed on learning analytics and the learning content to tune the resources that are presented to the learner (e.g., in [21]).These studies are inspiring but they differ from our approach in some important aspects.
Most reviewed approaches attempt to give a general model the learner (i.e., in a diversity of learning tasks).From that perspective, learner modeling is often directed towards classification of learners into several learner profiles with regard to a set of characteristics, and by doing so to select the more adequate resources for a given learner (e.g., according to learning styles in [22]).This is also the case for knowledge representation beyond ontology approaches, such as those related to the xAPI and game-based learning profiles [23].The cognitive trajectories introduced by [24] are an interesting approach to better understand the learner behavior, at a general level or in a specific task.
Another point is that a large majority of these tasks are well defined (e.g., well defined knowledge or how-to acquisition), whereas we consider here a more open ill-defined problem solving task.We can, however, cite the work of the Ludo Game Model Ontology (LUDO) [25] to describe serious games.These activities can be presented as open problems, and (despite being not really reusable here, because it does not involve tangible artifacts) the ontology itself includes both a model of the game (task) and a model of the player (learner) behavior.
Eventually, the goal of the existing ontologies is to improve the learner's performance [20], which is less easy to define in our problem solving context (solving the problem faster does not mean that we learn a lot from it) and is not our main purpose, which is to better understand how we solve such an open-ended problem.
With respect to these works, we introduce here computational and cognitive neuroscience knowledge in the modeling process, which does not seem to be a current practice, and we also consider open-ended problem solving, which is seldom addressed.
Ontologies in Neuroscience and Cognitive Science: The Cognitive Atlas 1 [26] is a collaborative initiative that aims to unify the vocabulary used in cognitive science and indexes cognitive tasks described in the literature.It includes over 800 cognitive concepts but does not encompass much logic, aiming for a more flexible approach.Its task model is partially based on CogPO [27], which is another ontology that focuses on the description of experimental paradigms used in neuroscience (mainly fMRI) tasks.While CogPO might be slightly too specific and medical-oriented for our problem, the Cognitive Atlas encloses many of the cognitive concepts that we may need to refer to in subsequent work.It also seems that the concepts we target here are not (to the best of our knowledge) addressed at the level of ontology modeling.
Beyond ontology-based methods, a wide spectrum of approaches, out of the scope of the present work, have been recruited to model ill-defined creative problem solving; for instance, using description logic, extended with probabilistic approaches [28].This example is interesting because the authors have made the distinction between different types of knowledge and memories, and considered its integration in a cognitive architecture [29].
Considering Foundational Ontologies: A good practice in ontology modeling is to base our model on a foundational ontology to better specify the design choices by relating our concepts to well-established formalized semantics and facilitate the integration with other ontologies.Neuroscience ontologies often rely on recommendations set by the Open Biological and Biomedical Ontology (OBO) Foundry, which is itself built on the principles of Basic Formal Ontology (BFO), as is the case for CogPO.However, the ontological choices made in BFO are meant to represent real-world elements and not, for example, mental states.Integration to biomedical ontologies is not necessarily a requirement for us, and the Cognitive Atlas, in that matter, is rather agnostic; therefore, a more appropriate choice for us would likely be the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) [30] which is general enough to be linked with both the cognitive and learning science aspects of our formalization work.In brief, it is a descriptive ontology of particulars that "aims at capturing the ontological stands that shape natural language and human cognition, based on the assumption that the surface structure of natural language and the so-called commonsense have ontological relevance" [30], which may better fit which our paradigm. 1See http://www.cognitiveatlas.org/for a browsable version, and on BioPortal for a reusable ontology version.

A. The #CreaCube Activity
This task has been designed considering the cognitive science literature regarding problem solving tasks [6], [31], but addressing here considerations regarding ill-defined problems.
The #CreaCube study2 focuses on problem-solving strategies using modular robotic cubes, targeting children between 8 and 12 years old.The task is explained to the participants as "building an autonomous vehicle, composed of four items, able to move from a point A to another point B", while a set of four modular robotic cubes that differ in their appearances (e.g., different colors) and features (e.g., wheels, switch etc.) is given.The learner is expected to understand that the four items are the four cubes on the table, to discover their behavior, and to assemble these cubes into a certain configuration that will be able to move autonomously.It is worth noting that this problem does not have a unique solution because several configurations may satisfy the goal requirements.Furthermore, additional goals may emerge, such as understanding the cubes, exploring alternative outcomes with those items, preserving the ego with respect to the task achievement, experiencing playful pleasure, and so on.The task is relatively easy in the sense that most participants come up with a solution in less than 15 minutes but complex to model because of the large problem space and the lack of specification (e.g., the possible actions, i.e operators on the problem space states, are not clearly stated) [12], which makes it an ill-defined problem.
In this activity, the participant has likely no prior experience of the modular robotic cubes.To complete the task, there is thus a need for exploration, defined here from a learning science perspective, as a way to gather information from the environment [32].Throughout the task, the participant moves towards the goal by mobilizing prior knowledge that appears to make sense in relation to their experience with the material being manipulated and the requirements defined by the guideline.This includes understanding the artifacts, their physical and technological characteristics, leading to formulate hypotheses on the material and define sub-goals to solve the problem.This process is better explained using the notion of affordances [33], as detailed in section III-D.
The problem is modeled based not only on the knowledge needed to solve it but also on the initial states of the hardware and the final state for its success, as detailed in Fig. 1.We can see the different observables taken into account (i.e., the possible configurations of the cubes-disassembled or assembled into a certain shape), the identification and conceptualisation of affordances (i.e., the practical possibilities offered by the cubes, as we will further develop), the outcomes of actions performed (e.g., at the motion level), and also elements regarding the subject's emotions or attitude (perseverance, abandonment) towards the task.These observables will serve as a basis for modeling the activity in the form of structured knowledge.
To allow for the analysis of the learner's activity in the context of the task, a formalization of the learning activity is developed in the form of an ontology in the computational sense that we will now define.

B. The Ontology Approach
Terminology: Ontologies allow us to represent structured knowledge by defining concepts, as well as relationships and hierarchies between these concepts.In the following, we use the Web Ontology Language (OWL) terminology, as used in the Semantic Web: (i) Individuals represent atomic, real-world objects.(ii) Classes represent concepts; a class may therefore be a collection of individuals (which are called instances of the class).(iii) Individuals may be linked by relationships, which are labeled by properties.Properties may have a domain (i.e., the class that they can be applied to) and a range (i.e., the class that they can take values from).
OWL also defines both class hierarchy and property hierarchy.This formalism allows us to structure and specify data that can be provided in a standard such as the Resource Description Framework (RDF).
OWL specifications are based on description logics, providing in some cases4 computational completeness and decidability (a didactic introduction can be found in this guide [36]).This makes it possible to perform logical reasoning to validate the model, as well as logical inference to find all of the assertions that can be deduced from the user-defined statements.
This representation could be improved using a numerical weighting of RDF statements based on the notions of possibility and necessity (in the sense of [37]) as proposed in [38], but we shall not discuss this any further here.
Methodology: [39] describes a simple methodology to build an ontology in seven steps: (f) Define the facets of each slot (i.e., the possible values that a property can take).(g) Create instances.
Considering (a) the scope defined in the previous paragraph, we have chosen to (b) rely on a lightweight version of the DOLCE foundational principles (DOLCE+DnS UltraLite) and reuse concepts defined in the Cognitive Atlas.This step (b) had been overlooked in the first version of our work [18], which we have thus revised to better integrate existing ontologies.In the following, we describe (c,d,e,f) the terms used to define the concepts and properties in our ontology (the slots refer here to the properties shared by a class, and the facets to the possible values that a property can take).Finally, we explicitize (g) how we instantiated those classes.
Based on these elements, the ontology has been designed at two levels.At the task material level, all cubes configurations and other physical elements of the setup have been identified and encoded, factorizing equivalent configurations (e.g., up to a rotation), and reducing the state space to a few qualitative relevant values.At the learner modeling level, considering the previously published studies [12], [14], [40], our work has been to formalize what has been experimentally determined by learning scientists and education experts, including the chosen pertinent observables.In order to do so, we have attempted to rely on theoretical learning science and cognitive science frameworks, and to identify the concepts that were the most relevant for our context.Observing experiment courses and discussing with the learning experimenters allowed us to capture those different aspects, both at the task level and at the learner level, and specify them as ontology entities.

C. Borrowing from Learning Sciences: Learner Modeling Framework and the Activity Theory
As previously mentioned, the growing use of intelligent tutoring systems (ITS) led to a need for the development of consistent learner models.Dillenbourg and Self [41] proposed a general framework for learner modeling in problem-solving activities, considering two dimensions: on the one hand, they distinguished the system from the learner and defined concepts regarding the system's representation of the learner; and on the other hand, they established a distinction between behavior, behavioral knowledge, and conceptual knowledge.Behavioral knowledge refers to a set of rules and primitives that can be used to infer some behavior for a given problem P, whereas conceptual knowledge contains the definition of the concepts underlying the behavioral knowledge.This representation allows us to precisely define, for example, misconceptions (i.e., the difference between the learner's and the system's conceptual knowledge) or bugs (i.e., the difference between the learner's and the system's behavioral knowledge).In our case, even if we are not implementing a tutoring system, we are in the situation of a system observing the learner's behavior and inferring their knowledge.Moreover, the problem is illdefined from the point of view of the learner, but it can be well-specified from the point of view of the system to guide the analysis of the observables, which is what we are trying to achieve here.
This framework does not really address the use of tangible artifacts in problem-solving activities.The activity theory [17] fills in the blank by introducing the notion of tools that play the role of mediator for the activity, which enables the learner (subject) to pursue a goal (object).Fig. 2 shows how we incorporate these notions into the learner modeling framework proposed by Dillenbourg & Self.In #CreaCube, the tools are the cubes and the waypoints, and the main object is to solve the problem (e.g., build a vehicle).In what follows, we will see how this main goal can be broken down into subgoals (which can be in concurrence).We will also explicitize how the aforementioned conceptual and behavioral knowledge can be understood in light of the distinction that is made in cognitive neuroscience between different types of memory.

D. Borrowing from Cognitive Sciences
Memories in Interaction: The role of memory in learning, and more precisely the interactions between memories that are of several types [42], is of primary importance at the neuroscience cognitive level.Traditionally, a distinction has been made between short-term (i.e., working) memory and long-term explicit (or declarative) and implicit memories, with regard to how long the information was allegedly stored in the brain.Working memory is considered not simply as a system for the transitory storage of information but also as a processing system [43]).Incoming stimuli perceived by sensory modalities are encoded in a sensory "buffer" (sometimes called sensory memory, e.g.[44]) where they persist very briefly.After being filtered through attentional mechanisms, some of them can be temporarily retained in working memory for future use and manipulation.For example, when solving a problem, working memory makes it possible to store traces useful to the processes involved in carrying out the task, as well as updating them as the task progresses.Because these processes often require prior information, working memory also enables retrieval from longer-term memories.Procedural memory is an example of implicit memory: it stores know-how and integrated procedures that have been automatized, such as motor skills.These procedures are gradually learned through a transfer from declarative and working memories.Declarative memories can themselves be distinguished between semantic and episodic.Semantic memory stores general knowledge about facts and concepts, while episodic memory refers to the subjective experience of the individual: it stores traces of past events that are contextualized in time and space, and associated to emotional data.They are stored as sequences of events called episodes, which can later be replayed forwards or backwards or rearranged, thus providing off-line learning.Eventually, recurrent patterns in these episodes can be consolidated into more generalized traces in semantic memory or procedural memory.Episodic memory is therefore believed to play a crucial role in human learning [45].These concepts form the basic architecture of the learner cognitive modeling proposed here.
Goals and Related Executive Cognitive Functions: Solving a problem requires one to set some goals.The main goal is to complete the task, but in order to do so, this task usually needs to be broken down in several sub-tasks, thus yielding several sub-goals.In the context of #CreaCube, the main goal given by the guideline translates into the concrete realization of a vehicle, using the four cubes presented on the table, that is able to move autonomously from the red point to the black point.Assembling the cubes into a valid configuration is a goal in itself, while making it move in the right direction is another.These high-level goals yield new sub-goals on the sensorimotor level-to assemble two cubes, one needs to localize them and mobilize specific motor skills to grasp them and put them together.
We consider here a framework of cognitive architecture in which these goals can be organized around four fundamental questions [46] that relate to the learner's cognitive functions: (i) What is the object of the goal?(ii) Why does it answer some current motivation or specification?(iii) Where is it located and how can it be accessed?And, (iv) how can it be manipulated and more generally addressed by a skill?This description of the goals represent an operational way to deal with all semantic, motivational and sensori-motor aspects of a goal to be considered in problem solving, consistent with what is known about associative neural circuits.
We also wish to consider goal orientation [47], distinguishing between mastery goals and performance goals.Participants concerned with mastery are oriented to develop task-related self-improvement [48], while those focusing on performance aim to achieve the task objective using "known ways to quickly implement knowledge and skills that have already been mastered" [49].
Creative Use of Affordances: Building with modular bricks is based on visuo-spatial constructive play objects [50] used for engaging learners into tangible programming brick games [51], [52] with modular educational robots [40], [53].In the context of an ill-defined problem solving task [31], the affordances of the robotic elements will influence the actions of the participant [54] towards the objects (observing, grasping, assembling).Indeed, as opposed to a well-defined Fig. 2: Framework chosen for the analysis of the observables, adapted from Dillenbourg & Self [41] completed by the activity theory [17].This framework corresponds to the learning science concepts taken into account for the construction of the ontology.The observables are the ones described in Fig. 1.See text for more details.
problem, the legal operators that can be considered to act on the problem states are not specified here.Instead, they are suggested by the physical features of the artifacts: for example, the blue cube has a switch on one of its faces, appealing to be activated.The exploration of one's environment (as developed in the next paragraph) aims to discover such affordances, as defined by Gibson [33] as "opportunities of actions" offered by the physical properties of this environment.Through their interactions with the task material, the learner relates the perceived physical properties of the artifacts to prior knowledge stored in long-term memories (e.g., a switch can be flipped to activate a device) and generates new hypotheses.Affordances are a way to make sense of stimuli, and a key point in the creative process is to be able to think of different appropriate ways to use the material to solve the problem, using mechanisms such as analogy to transfer knowledge from a task to another [3], [4], [10].
Exploration vs Exploitation: While the notion of exploration and exploitation is well known in cognitive science and in machine learning at the level of reinforcement learning, they are two distinct conceptualizations that we will now make explicit.In problem-solving activities, subjects alternate between two main strategies: exploration, which aims at experimenting with new alternatives or acting on the environment to generate new stimuli [32], and exploitation, which is the use of existing knowledge (semantic or procedural) in a given situation.At this level of description, we can draw parallels with these notions in reinforcement-based machine learning, but in this second domain exploration is mostly random, whereas in problem solving as studied in learning sciences, exploration is seen as a strategy for generating new stimuli or new ideas or behaviors by recombining prior knowledge or behaviors, or by manipulating the environment.These are therefore two quite distinct notions: one corresponding to an internal process of generating new internal representations, and the other to a behavior that is only defined with regard to a direct interaction with the environment.
Convergent vs Divergent Thinking: In relation to the previous distinction between two strategies -exploitation and exploration -, we can also introduce an alternation between two modes of thinking -convergent and divergent.Divergent thinking is directed towards idea generation (fostering originality and novelty), while convergent thinking focuses on selecting an effective candidate solution to the problem (ensuring adequation).These two processes, consistent with the previous definition of creativity, have been defined to be central to creative cognition [3] and are characterized by measurable indicators [55].This brings another perspective to the goal definition that was addressed earlier: during the divergent phase, goals and their sub-goals thus unfold into tree structures that take shape through stimuli and experiences, and are reunited during the convergent phase so as to achieve the final goals.

E. Concept Selection and Instantiation Methodology for the Ontology Implementation
In order to implement the Learner model of the ontology, we selected several concepts in relation to the previous theoretical frameworks: Perceptions are internal representations of stimuli formed by the learner.Traces of these perceptions are processed in working memory and may be stored in the long term in episodic memory.Conceptual knowledge as defined in III-C is for the most part stored in semantic memory and can be retrieved in working memory when it is relevant.Behavioral knowledge as defined in III-C consists of procedures and inference rules that are not convenient to represent in the OWL formalism because they are not declarative (e.g., procedural).However, we can represent the result of such inferences, which are drawn by the learner by contextualizing pieces of conceptual knowledge with regard to their perceptions.We have called these inferences hypotheses: they are temporary representations formed by the learner to make sense of the environment.Hypotheses are only relevant for a short time interval within the task (e.g., the execution of a plan associated with a goal) and we can assume they are only temporarily stored and used by the working memory.When they have been verified (thus believed by the learner as true), they can be maintained in the working memory to pursue the task but they are not questioned anymore (unless a new stimulus raises a contradiction).We can assume that hypotheses that were useful to complete the task may have a more durable print and be stored in episodic memory.The learner's goals are broken down into the four queries described in III-D (namely, What, Why, Where, How) and towards mastery vs performance goal orientation.
These concepts have been instantiated step by step, based on the design of the task itself and preliminary video observations of the first participants [56].Further details were collected during a post-activity survey: the participants were asked several questions about what they did and what they learned about the activity material.Some elements were also noted throughout the task when the learners were speaking out loud, to themselves or to the experimenter (there was no directive to the participants about whether they should speak during the task or not; the experimenters were, on their side, required not to answer the participants' questions).
(a) Observe all of the possible actions and break them down into elementary actions (e.g., assembling three cubes decomposes into assembling two cubes and connecting a third one to the resulting structure).(b) Identify the contexts in which each action was executed to determine which are the preconditions (necessary or optional) and in which state the system is after the action has been performed (post-conditions).(c) Identify, from the material changes, what stimuli are raised and how they are perceived (e.g., when the power switch is on, a diode lights up).(d) Specify the hypotheses and knowledge: • Conceptual knowledge relevant to the task was mostly identified prior to the experiment, as part of the activity design (e.g., what is considered as a vehicle, what are its main elements).• Hypothesis instances are issued from both preliminary observations by the experimenters of the resolution by different subjects and evaluation interviews after the activity, as discussed for instance in [40], [53].(e) Finally, identify how these hypotheses and knowledge are pre-requisite for others and might drive the participant's goals.(f) Associate the actions listed in (b) with the learner goals, issued from the activity observation, while also considering goal orientation and broken down into elementary goals corresponding to the four cognitive queries.These are clearly preliminary prior assumptions to bootstrap a first version of the ontology, with the perspective to adapt them from further observation by learning sciences experts and logical validation by ontological reasoners.

IV. ONTOLOGY IMPLEMENTATION
We implemented the ontology5 using the software Protégé 6 .First, we formalized the classes corresponding to the material environment of the task, and the classes referring to the learner's cognition and behavior.Then, we have defined specific classes and properties to include the observables of the task, which are situated in time, such as events and situations (in the sense of DOLCE+DnS Ultralite).Finally, we have instantiated these classes, listing their individuals based on the task design itself and the observations collected during the experiments, according to the methodology described in III-E.

A. Modeling the Activity Process
The #CreaCube activity itself is a dul:Process; that is to say, considered in its evolution.Each scene of interest annotated in the videos is a dul:Situation, which we call an ActivityState.
Activity states are transitional and situated in time.To account for this transitionality, we use the Description & Situation ontology pattern 7 of DOLCE+DnS Ultralite (we now use the prefix dul: for pre-defined entities).
A dul:Situation is, as per the entity specification, a relational context that is created by an observer on the basis of a dul:Description.Consequently, a dul:Description is itself a conceptualization that provides an interpretation for a set of entities in a given situation.
We consider here two aspects of the activity situation: the learner's mental representation (their representation of the scene) and the material states, from the point of view of an expert of the activity (cf Fig2).We therefore assign, for each ActivityState, two kinds of dul:Description: a MaterialStat-eDescription and a LearnerMentalRepresentation.

B. Modeling the Material States
The task involves several tools (in the sense of the activity theory), which can consist in dul:PhysicalObjects (cubes, waypoints) or dul:SocialObjects (guideline recording).Physical objects have features that can themselves be considered as physical objects constitutive of the cubes (e.g., switch, wheels, magnets etc.).The cubes can be assembled into multiple configurations (encoded by a series of letters and numbers as per Fig. 1) characterizing the material states.As opposed to transitional material states, physical objects have a proper In green are the other classes related to the observables.Those in blue characterize the learner's behavior and mental representations.Some classes extend the ones defined in the Cognitive Atlas (prefixed by ca:) and/or in the DOLCE+DnS Ultralite ontology (prefixed by dul:).The symbol "<" in labels stands for "isA" when the node is not expanded for concision purposes.space region and mass.Their state is represented by the values taken by certain properties called dul:Quality (e.g., color) in a given qualitative dul:Region (e.g., red): for example, a switch has the quality of being either ON or OFF.
Material states are characterized by a specific configurations of those physical objects, their quality values, and correspond to the observables shown on Fig 3 .We have also listed the possible permutations of the cubes within each configuration shape, so that each configuration encoding (e.g., "F000-BSIW-T") is characterized by its connections.These connections are themselves specified by the cubes they connect along what spatial axis.This allows us to give a better description of the material states and relates them to their JSON encodings.
These material states represent the problem space of the task in the sense of [6].Indeed, the goal states of the task are those where the cubes are in a configuration that satisfies the guideline (which is itself a dul:Description describing the requirements for a state to be a solution state).The participants may interact with the material and change its state through their actions: this is formalized as follows in the sequel.

C. Modeling the Learner's Behavior and Mental States
Actions and Affordances: Some actions are only available to the learner in some particular states of the material environment.These states define the preconditions of the action.Similarly, post-conditions define changes of state that an action results in.These pre-and post-conditions also constitute knowledge to acquire.They are represented in the ontology by properties linking a dul:Task (referring to the execution of an action) to an ActivityState (i.e. the dul:Situation before and after the action has been performed).Actions are suggested by the physical objects-or their features-that the learner relates to their knowledge, which we introduced earlier as Affordances.Affordances allow a dul:Task to be executed.When appropriate, we related those affordances to the ones defined in AfNet [57], a semantic network aiming at categorizing structural affordances based on physical properties of objects.
Stimuli and Perceptions: Throughout the activity, the learners receive various stimuli from the material environment.Stimuli are sometimes modeled as events in DOLCE-based ontologies, as is the case in the Stimulus-Sensor-Observation ontology [58].Indeed, in the #CreaCube activity, stimuli are most often the result of events occurring when participants proactively interact with the environment.For instance, when they flip the switch on the blue cube, a diode lights up as a response to their action.However, the learner may not always perceive stimuli as soon as a triggering event occurs, and stimuli might remain in the environment for some time before the learner focuses their attention on it (e.g., the diode that lights up is very small and some participants only notice it later).Therefore, we prefer to view them as sensory information input provided to the learner, rather than events themselves; hence, extending the InformationObject class.When they are filtered by the attentional focus of the learner, traces of these stimuli are kept active during the task before being eventually consolidated as knowledge.These traces constitute Perceptions: they are a conceptualization of the perceived stimuli, thus a kind of dul:Description.
Knowledge Retrieval, Hypotheses and Goals: To complete the task, the learner needs to retrieve knowledge that has been acquired in the course of past experiences and learning.Most of the time, this knowledge is activated through the discovery of affordances.For instance, after finding of the wheels, the learner might retrieve the fact that wheels on a vehicle must be in contact with the ground to allow it to move.It can also be activated by the concepts evoked in the guideline.
As we explained in section III-C, we previously distinguished between conceptual knowledge and behavioral knowledge before deciding to represent only ConceptualKnowledge in the ontology because behavioral knowledge consists of procedures and inference rules that are, in our case, not declarative.However, we accounted for the result of inferences with the class Hypothesis, representing hypotheses drawn by the learner by contextualizing pieces of conceptual knowledge with regard to their perceptions.
ConceptualKnowledge, Perceptions and Hypotheses are all mental representations that the learner forms to make sense of the environment.These items extend the class dul:Description and are interdependent (i.e.some of them can be inferred or induced from others).These dependencies are represented by ontological properties, such as requires (logical necessity), induces (inductive reasoning) and isEvidenceFor/hasEvidenceIn (e.g., when a perception tends to confirm a hypothesis).
Instead of directly identifying discrepancies at the behavioral knowledge level (which were called bugs in [41] and on Fig 2), we identified misperceptions and misinterpretations.Misperceptions happen when internal representations of stimuli formed by the learner are erroneous; for example, if the learner misheard the guideline, did not see a visible feature on the cubes, or mistook the dark blue cube for another black cube.Misinterpretations refer to erroneous hypotheses that can be inferred, either from misperceptions or from some conceptual knowledge not transferable to the situation at hand.For example, the fact that wheels automatically start to turn after the user has pulled the vehicle backwards is only applicable in certain situations-most often, this mechanism is found in children car toys.However, in this task, trying to press down on the white cube by manually rolling it backwards will not cause it to move forward autonomously.Such misinterpretations can be corrected after verification showing a lack of evidence in favor of the hypothesis.Nonetheless, if it comes from a misconception that the underlying conceptual principle is a general truth that is valid in any situation, then it might hinder the completion of the task: as long as the learner keeps on trying to make the wheels work this way, they might not look for other clues.
This makes it clear that the learner needs to understand which pieces of knowledge are effectively applicable in the context of the activity and relevant to complete it.Testing hypotheses thus defines sub-goals that will help to solve the problem.As described in section III-D, goals are a way for the participant to organize their understanding of the task and plan their actions hierarchically.Global goals (essentially re-sponding to What and Why, in relation to high-level executive functions [46]) may be instructed by an external directionin this case, the guideline-or a metacognitive reflection, inducing (Where, How) sensori-motor mechanisms [46] thus yielding sub-goals at a more local level.This goal hierarchy is represented in our ontology through the property hasSubGoal.

A. Modeling as an Interdisciplinary Tool
This work allowed us to confirm the multiple interest of working with ontological modeling when we bring together three disciplines (educational, cognitive and computer sciences) to tackle a problem, as follows: • Terminological interest to begin with: beyond periphrases and phenomenological descriptions in each discipline, constraining us to define things through lexical choices and fully specified properties and relations between these key words, forcing us to clearly posit what we are talking about.
• Interest in formalization: this approach allows, without even using algorithmic reasoning skills at this stage, to take stock of what can be defined rigorously to formalize completely.Defining a well-formulated ontology is therefore already by itself a structuring exercise, an epistemological tool in a way, before even using it.As with any modeling, it gives an exhaustive and explicit view of what is to retain compared to what is neglected.Once the specification has been set, the use of available reasoners, such as Pellet or Hermit, allows to check its consistency and query the data on the consequences that may or may not be drawn from it.
This being done, we also have an immediate check of the model properties, in particular its logical coherence and soundness.This not only allows us to verify the absence of contradictory statements but we also experimented that if our definitions are not sufficiently specified, then we can easily deduce spurious consequences (e.g., mixing categories of concepts).We thus have an operational tool to verify the completeness of our formalization.
An illustrative example of how a JSON video annotation can be translated into this formalism is given here.

B. Reasoning About the Model
A query language, such as SPARQL in the case of RDF data, combined with a reasoner (in our case, Pellet8 ), allows us to retrieve specific information -asserted or inferred-from our model.This raises some perspectives of application both at the material level and at the learner level.Let us illustrate each aspect in the following examples, the former being already operational and the latter being still at an exploratory stage.
Link the Material Configuration Encodings to the Material States: At the material level, this ontology can be used to retrieve physical information from configuration encodings.In the JSON annotations of the videos, configurations tested by the subject are encoded as strings of characters and letters (e.g., "F000-BSIW-T") that are not intelligible at a first glance, and hard to relate to real-world connections between the cubes.The ontology does this strenuous work for us: by querying a configuration by its encoding, we can retrieve the associated connections between the cubes.Corresponding ontology assertions are based on simple geometric considerations and the design of the cubes provided by the manufacturer.Conversely, we can also query the knowledge base on configurations which have, for instance, a connection between the blue cube and the red cube.Integrating an ontology-based module to the annotation interface might help to avoid encoding mistakes.A step further, we could also add some transformation rules between those configurations to validate the pre-and postconditions of the learner's actions to verify the consistency of the annotations on a sequential scale.For example, if the learner has disconnected two cubes from each other, then we should see that there is one less connection in the new configuration.

Inferring an Internal Representation of the Learner from an Observable:
At the learner behavior level, the main perspective of this work would be to interpret observables and infer, for example, the participants' goals and associated plans or motivations.Although more data and further investigation is needed to validate the relevance of the ontology inferences on this aspect, we propose here the general idea of such an application.
Storing domain knowledge about behavioral processes in an ontology allows to consistently interpret new data.For example, we may observe a new action which changes the state of the switch, and wonder what motivated the learner to execute this action.As illustrated by Fig. 5, a SPARQL query on the model helps us retrieve that the action may have been either a simple reaction to the stimulus of seeing the switch, in the case of a stimulus-driven behavior, or, in a goal-driven mode, the result of a more elaborate reasoning, such as an attempt to understand the function of the blue cube (thus testing the hypothesis that it is indeed a switch) or, this hypothesis being verified, to activate the movement of the vehicle.Characterizing whether the learner is rather in a goal-driven or stimulus-driven mode, as precisely defined in [18], [56], for a given sequence of observables might be inferred from the timestamps recorded in the data.This helps us decide which statements of the ontology are more appropriate to describe the observed situation.For example, we could assume that once the learner has identified a goal, and have the means and resources at their disposal to fulfill it, then the flow of their actions might be smoother; whereas, in an exploration situation, there might be more pauses corresponding to the delay between new stimuli.However, this link between timestamps and switch of strategy is not yet well understood and needs further examination.Another difficulty is the high computational complexity of assertions regarding the learner behavior, which does not scale well with a large knowledge base and may lead to poor reasoning performances.
Predicting the Learner's Behavior: Instead of querying the model using backward chaining as in the previous example, we could also use forward chaining to simulate the learner's deduction, hence predicting their behavior.For example, we might wonder how the learner would react to the visual stimulus of seeing the switch.In a stimulus-driven mode, they would likely touch it or try to flip it (assuming that the knowledge relative to switches being flipped has become a procedural knowledge, that is to say automated into an almost reflex behavior).In a goaldriven mode, we may infer that the learner will elaborate the hypothesis that the switch can be used to activate the vehicle, which will result in planning goals (flip the switch, connect the cube to the structure) and sub-goals (they have to grasp the cube before connecting it to the others).
This ontology thus provides a subset of plausible behaviors.However, we still need a way to decide which of these behaviors to choose to simulate the learner.Mechanisms to explain what the learner's attention is more focused on in a given situation are still to be developed.Moreover, we need to take into account inter-individual differences: some participants might be more interested in achieving a performance goal, thus appealing for a predominant exploitation strategy, whereas others might pursue a mastery goal and try to understand as much as possible about the environment of the task, displaying more exploratory behaviors.

VI. DISCUSSION AND CONCLUSION
This study introduces the operationalization of a creative problem-solving task.Within this type of task, we need to consider both exploration and exploitation strategies directed toward different concurrent goals (e.g., performance versus mastery).These goals refer to an objective (What) but we also need to consider the Why; that is, extrinsic motivation (e.g., task performance) and intrinsic motivation (e.g., increase knowledge).They result in concrete plans to achieve them (Where and How).The execution of such plans is the observable behavior of the learner that we attempt to capture  and formalize in our ontology.Through this process, we take into account the stimuli received during the interactions of the learner with the modular robotic cubes, and the perception and interpretation of the different technological affordances that will support the problem-solving activity through concrete actions to be executed.The learner's prior knowledge is mobilised to understand current perceptions and transform knowledge into a way that makes sense within the context of activity [59], formulating and testing in-task hypotheses.
This formalization was achieved step by step with the aim of encapsulating domain knowledge used by learning science researchers when they analyze such an experiment, in order to make it more robust and reproducible.The purpose of this description is not to remain phenomenological; that is, driven by a theoretical framework expressed in human language to guide the analysis of the data.We hereby reformulate these notions at a computational level to give us the means to process them in a more systematic way using an algorithmic approach, as discussed here, formalizing the concepts invoked in a precise language.
As stated in the introduction, we model the learning behavior engaged in a problem-solving activity that is ill-defined at the participant level, while the activity state space is entirely formalized at the observer level.As such, our approach is voluntarily specific, while the proposed methodology is easy to generalize, as pointed out in [16] where three other computational thinking learning tasks have been also considered.From the paradigm choice to rely on an ontology modeling to the general knowledge about cognition when engaged in a creative problem solving task, an important part of this formalization translates to another learning task.Relying on a robust foundational ontology such as DOLCE also ensures compatibility and facilitates such knowledge transfer.Besides, our paradigm is based on frameworks that are already widely used in learning sciences such as the activity theory and the ITS learner/observer representation, combined with a cognitive basis.A step further, the query-based specification of goals and the hypothetico-deductive process might also be reconcilable with, e.g, the Belief-Desire-Intention (BDI) model commonly used in AI and inspired by psychology to formalize rational agents [60], where knowledge and hypotheses would constitute beliefs with various degrees of confidence, desires may correspond to the What/Why of the learner's goals, and intentions to the procedures considered to reach these goals, i.e.Where/How.
For obvious ethical reasons, the learner is aware of being engaged in a task with learning analytic evaluation.To what extends this could perturb the outcomes has been addressed in [40], in fact only marginally.Besides, this could be turned into a significant advantage, since it is a way to engage the participants in understanding their own learning process, opening opportunities to meta-learning.This is only a first step but it already shows that formalizing such a problem-solving activity is a complex but attainable work.To complete this formal description, one of the next steps would be to better formalize the relationship between affordances and knowledge (e.g., we could draw inspiration from [61] who proposed a design pattern involving the use of frames).We also consider to address the emotional aspect, such as how to identify observable emotions (e.g., the ones listed in Fig. 1) and link inferred cognitive states to emotional states.Finally, we need to make use of reasoning mechanisms to better infer the subject internal motivation and behaviors from the experiment observables (e.g., learner state observation, post-activity evaluation) and conduct statistical analyses on the results to assess the validity of such an application.
With respect to the state of the art, briefly reviewed previously, our approach applies to tangible activities, modeling both the task and the learner in the task, and targeting a better understanding of the learning process, more than increasing the learning performance.This approach is rather disruptive with respect to learning science because, to the best of our knowledge, there has not been such modeling of an illdefined task using an ontology, including a triple-disciplinary contribution from learning science, cognitive neuroscience and computer science, from which could emerge computational learning science [62].
(a) Determine the domain and scope of the ontology.(b) Consider reusing existing ontologies.(c) Enumerate important terms in the ontology.(d) Define the classes and the class hierarchy.(e) Define the slots (i.e., the properties shared by a class).

Fig. 3 :
Fig.3: A partial diagram exposing the main classes defined in our ontology.The tools mediating the activity are highlighted in red boxes.In green are the other classes related to the observables.Those in blue characterize the learner's behavior and mental representations.Some classes extend the ones defined in the Cognitive Atlas (prefixed by ca:) and/or in the DOLCE+DnS Ultralite ontology (prefixed by dul:).The symbol "<" in labels stands for "isA" when the node is not expanded for concision purposes.

Fig. 4 :
Fig. 4: Examples of a SPARQL query 9 on the material states: Retrieving connections and cubes for a given configuration.

( a )
An example SPARQL query (b) SPARQL results returned for the query (a): We enquired about which actions change the state of the blue switch, what stimuli may have provoked them, or what goal they might be aiming at, and if that goal might be the sub-goal of another one, as well as what knowledge they are using and what hypothesis might be tested.