Design-Based Learning in Classrooms Using Playful Digital Toolkits

. The goal of this paper is to explore how to implement Design Based Learning (DBL) with digital toolkits to teach 21 st century skills in (Dutch) schools. It describes the outcomes of a literature study and two design case studies in which such a DBL approach with digital toolkits was iteratively developed. The outcome is described in the form of a framework that explains how to consider different perspectives, such as the DBL process, the role of the teacher, the use of a digital toolkit and the framing of the design brief in relation to setting learning goals that are suitable for a school context. The design cases indicate that DBL with digital toolkits can play a valuable role in teaching 21 st Century skills, such as problem solving, creativity, and digital literacy to children in schools, if the other components of the framework, such as school’s learning goals, are taken into account.


Introduction
Changes in society and economy have led people to reflect on what education should look like. With the Lisbon strategy created in 2000 and the Europe 2020 strategy, Europe set the goal for becoming ''the most dynamic and competitive knowledgebased economy in the world'' [1]. One of the methods to achieve this is a focus on skills and ''lifelong learning'' by reinforcing the role of education. Countries in Europe have started to translate these ideas into consequences for the curricula provided in schools. A promising approach is using design-based learning (DBL) to address the stimulation of 21 st century skills [2] [3]. Twenty-first century skills are a set of skills consisting of: creativity, critical thinking, problem-solving capabilities, communication, collaboration, researching, innovation (entrepreneurship), digital literacy and reflection [4] [5] [2]. The Netherlands has already started to integrate 21 st century skills and DBL into their curriculum [5] [2], England has also translated (part of) the European strategy into a new curriculum for education that encompasses DBL with design thinking learning goals [6]. DBL is a playful approach to learning: it allows children to seamlessly combine play and learning in a very fluid process. This is similar to how children use their imagination, try out ideas, and think about what they see [4]. While DBL is already being used at technical higher educations and even at a limited amount of special technology focussed high schools in the Netherlands called ''Technasia'', there seems to be little experience with designing DBL approached for a younger target group (primary and secondary school) with 21 st century skills as a learning goal [7] [8].
The goal of this paper is to explore and define all different elements needed for implementing DBL in a playful way, using digital toolkits, to teach 21 st century skills in current (Dutch) primary and secondary schools. We present our insights in the form of a descriptive framework. Furthermore, in our process we explore how DBL toolkits might be used in schools, this will be presented in the form of two case studies which we used for qualitative research.

Related work
The SLO (Dutch foundation of education development) states that the development of 21st century skills should be offered in primary and secondary schools to prepare students for the future [9]. The Netherlands has included this advice in the format of learning goals for the Dutch educational system. The Dutch learning goals have shifted over the past years from developing mostly theoretical knowledge to the development of skills and knowledge. Other countries also acknowledge the need for a change in education stimulating 21st century skills. The United States responds with their STEM programme (Science, Technology, Engineering, Mathematics) while the UK encompass design activities in their curriculum [10]. Although the details of the approaches may be different, the underlying motives of all countries are similar: preparing the new generation for the 21 st century by transforming education.

Learning goals & styles
Previous learning goals for Dutch education (1993) as defined by the SLO had a heavy focus on theoretical skills and knowledge in the form of 103 core learning goals for primary education and 300 core goals for secondary education [11]. In 2006 the SLO decided to leave more room for a school's own interpretation of the learning goals. They created 58 main learning goals for primary education and 58 main learning goals for secondary education that focus more on the learning process and less on pure theoretical knowledge [12].The learning goals now also encompass 21st century skills like ''the ability to research'' and ''reflecting on own and others work''. However, the focus is still mostly on theoretical learning processes and less on domain crossing competencies like 21st century skills. The target group of this project consists of secondary school students from first and second class and primary school students in their final years. These students are between 10-15 years of age. Children of this age are in the formal operational stage according to Piaget's theory of cognitive development [13]. In this stage a person becomes capable of hypothetical and deductive reasoning and able to think about thinking and abstract concepts. Piaget's theory is a theory on which a lot of aspects of current education are based.
As described in the introduction, new skills and knowledge are required to be able contribute in a 21 st century workplace. This has also created new views on learning such as the constructivist perspective on learning [14]. This perspective describes that learning and knowledge are an active construction of creating meaning by the learner. Kolb describes learning as the process whereby knowledge is created while experiences are transformed [15]. This perspective can also be described as learning by reflecting and doing.
The constructivist perspective has an influence on the role of the student, on the design of the curriculum and assessment, and on the role of the teacher [16]. A 21 st century curriculum should allow for active student participation and control, offer ample opportunity for interaction, and provide an authentic context for students' learning. DBL gives the opportunity to integrate these aspects.

Design-based learning
At secondary school, children are often used to executing assignments and meeting the goals set by the teacher. Because of this, children often tend to do their work in a way they think that satisfies their teacher [5]. In kindergarten however, children often learn by going through a specific kind of design-like exploration process: "Imagine, create, play, share, reflect" in which they set their own goals and targets [4]. This iterative process can be compared to DBL. DBL can provide an entirely different way of teaching in which students become co-owner or creator of a research/design assignment. Students learn to think critically and are better connected to what they are doing [5]. DBL furthermore allows teachers to combine the development of theoretical knowledge and 21 st century skills and allows the students to immediately apply what they learn in a social context [2] [3].
Although the effectiveness of a teaching method is often difficult to prove, initial research done on DBL is promising. Kolodner describes a large project in the United States concerning a newly developed DBL method called "Learning by Design" [3]. In this study 240 students participated in a study over multiple years. Kolodner found that students learn to become better critical thinkers and are stimulated to put more effort into their work at school. General (21 st century) competencies like collaboration and negotiation skills will be faster to develop compared to by using traditional learning methods. A big challenge is the fact that the classroom culture has to change and that the teacher has to be very flexible. In addition to this the way a teacher applies DBL heavily influences the achieved learning results, arguably even more than with traditional education styles [3].

Toolkits
DBL is often accompanied by 'tangible' learning. The choice for combining DBL and tangible learning is often made because children learn better while playing and exploring in the physical world [17]. Another benefit is a great amount of engagement; children are less likely to consider a tangible appliance as a traditional learning tool [18]. This is confirmed by Giannakos and Jaccheri with their OurToys program. With OurToys, children build their own digital game/story with physical objects [19]. The program raised awareness for technology, intensified the experience, invited children to explore boundaries and increased collaboration while learning and playing. The traditional Kindergarten approach provides materials for children to play with. Children with different interests and learning styles can use the same materials, each in their personal way. The Kindergarten tools enables children to go through a designlike process and make innovative creations [4]. Resnick states that it is important to transform the traditional tangible Kindergarten tools into different types of tools, media and materials with a regard to age appropriateness and current era [4]. Since the late 1960's researchers argue that the particular properties of the constructive building blocks offered to children limit or enhance what they can build, create and learn [20]. Due to this, the development of electronic, physical toolkits for learning to support creating and teaching the added value of technology has been growing since 1980. Besides toolkits, learning events to teach children about technology and design exist, such a Lego League Junior and STEAM maker festival.
Most of the currently existing toolkits and events are however focused on a very specific set of skills and knowledge and are not embedded in a school context. This can for example clearly be seen with Littlebits [21] and the Arduino/Raspberry Pi platform [22] (figure 1). Both mostly focus on physical digital electronics and/or programming and offer no clear directions for teachers on how to implement learning activities with the toolkits. Just making is not enough to guarantee learning [23]. Lassiter et al. state that it is needed to empower students to control their learning in authentic projects with real-world problems [24].

Method
The approach for this project has two layers, the development of a framework and embedded in this process two design cases that have been conducted. The framework is developed by combining insights from a literature study and our two design case studies. The concepts of the two design cases were iteratively developed by applying a user centred design approach, incorporating input from diverse stakeholders. The design cases contributed to the development of the framework but are in addiction to this a result on its own, showing how DBL with digital toolkits could look in practice [25]. Finally, in the conclusion we will discuss and reflect upon the uncovered requirements and the two final DBL concepts we created.

Framework
To examine what factors influences embedding DBL to teach digital literacy and design thinking in schools, a holistic exploratory research approach is applied. A literature study is done to uncover factors mentioned in previous work, combined with various design cases with stakeholder input (e.g. four teachers, two educational experts, one publisher and two curriculum developers) to develop concrete solutions in real world contexts of primary and secondary education. Four stakeholder workshops were organized to gather the experience of multiple experts in an efficient way: Two workshops with experts from primary education and two with experts from secondary education. In these stakeholder workshops early design concepts (four concepts for PE and four concepts for SE were discussed and used to uncover requirements for implementing DBL in (Dutch) education. Consulted experts are experts from the field of primary and secondary education, having a background as teacher, curriculum developer, curriculum publisher, teacher educator or designer of educational tools. All gathered insights are combined and integrated in multiple different iterations for the framework. The final version of the framework describes a collection of requirements for implementing a toolkit for learning in (Dutch) education. In the following section the framework will be explained in more detail.

Design cases
We will describe two design cases done in the context of Dutch education. Two toolkits for learning were iteratively developed using a user centered design process for two age groups (primary and secondary school, respectively 10-12 year and 12-14 year old children). The end users and stakeholders from the field of education were heavily involved in the design process with observations, user tests, expert meetings, and the stakeholder workshops.
The toolkits were developed to support the development of a subset of the 21 st century skills: creativity, critical thinking, problem solving, collaboration, and reflection. This set of skills can be seen as a 'design' sub-group. To describe the final list of criteria surrounding DBL based toolkits for learning, the main insights of user testing the two design cases in context were integrated into the framework.

Framework structure
The insights gained from literature research and the two research by design cases are combined in the framework to describe the different elements needed to implement DBL for developing 21st century skills in (Dutch) education.  Figure 2 shows the components of the framework. Literature research showed that a playful learning toolkit must be combined with a DBL process. It is important to have a clear design brief that provides a direction and frames the design (DBL) process and uses the toolkit as an instrument for that. Together the toolkit and DBL process should offer the possibility to meet learning goals: set by the curriculum of the (Dutch) government such, 21 st century skills and the development of an awareness for the value of technology in society [11]. The toolkit must be designed to be open ended as to be applicable with different design briefs and different end results [25]. Next to that it must allow children who may have different learning styles and approaches to work with it [4]. In addition, the teacher (role) must be able to guide the process and progress that children are going through and must (be helped to) understand the principles of a DBL process and the toolkit [24].

Framework components
Roles. It is important that teachers must get used to a certain mind-set of children being more self-directed instead of blindly following a method. Therefore it is necessary that a design brief does not blur the creativity of teachers but gives them support without forcing in one direction. In project-based education, the teacher should be able to assess progress, diagnose problems, provide feedback and evaluate overall results [26]. This means on the one hand that the teacher must have a certain knowledge about DBL and the principles of the toolkit in order to explain and guide the progress [24]. On the other hand, teachers should also get familiar or confident that they cannot always predict the outcome of design activities.
Design brief. It should provide the opportunity to apply a design process and should create a clear role for how the whole toolkit will be used. It should be composed in such a way that is possible to connect the design activity to other courses: this way the added value of technology (in other disciplines) can be highlighted. Meaningful contexts and scientific concepts, close to children's own imagination, enhance opportuni-ties for discussion. Reasoning, interpretation and reflection are important for knowledge building [27] [28]. This often implies examples from everyday life and current socio-scientific issues. These themes or topics are by their nature interdisciplinary, and require teacher cooperation [28].
DBL. DBL is a teaching method that incorporates a design process to stimulate learning. The design process is introduced by the design brief. There are multiple design processes that can be used, the chosen design process should allow a connection between activity and learning goals. In addition to this, a DBL approach must offer two levels: the level of applying the process as a tool and learning through the process by for example reflections, and gaining course based knowledge.
Toolkit. The framework shows the importance of the usability of the toolkit, the toolkit should be general enough that children with different learning styles can work with it but it should also offer the possibility to dig deeper in the materials and functions of the toolkit [4]. These layers must make it is possible to create multiple iterations in their design process. The iterations will enforce the design process due to observing, reflecting, discussing and improving. The toolkit should be designed in such a way that it can fit multiple design briefs, allowing for reuse of the teaching method.
Learning goals. A last important insight is that the approach had to support the ability to measure if learning goals are achieved. This has to be facilitated by both briefing and design process. This becomes a clear challenge when you compare developing skills and a certain mind-set with examining answers on a test that have only one possible solution.

Design case 1: Spark! Toolkit for learning
With the Spark! design case, secondary school students (first and second class, 12-14 years) are targeted.

Concept -Spark! toolkit for learning.
With the Spark! concept, students create solutions for societal problems. Throughout the Spark! process, both teacher and students are supported with a briefing that stimulates to achieve depth. Everything needed for a Spark! lesson for two students is provided in one wooden case, which can be used for different (societal) problems. The full process needs at least three hours but should ideally be spread over multiple lessons, depending on teaching style, the skill level of students and required results. The following design process is used in the Spark! concept: 1. Identifying and describing the problem: E.g. the introduction of for example street litter as a societal problem. Spark! supports tangible learning by having students physically prototype and experiment with technology by combining general arts and crafts materials, LED's, conductive glue, a specially modified Arduino and visual programming. The form and contents of the whole Spark! toolkit is intended to make students feel like designers. Technology used in the toolkit is selectively exposed.
Scenario. During a Spark! lesson, the teacher functions as a facilitator of learning to stimulate all duos in his classroom to reach enough depth to safeguard development. Duo's form a team and function like a design agency. The selectively exposed technology allows students to experience the power of technology by prototyping their design solutions and making them work while at the same time not scaring them with too difficult technological aspects. Results of a design duo (e.g. with the design case ''decreasing the amount of street litter'') can range from talking trashcans that should convince people to throw their garbage in the bin to playful litter collection baskets that require people to playfully throw away their trash and sort the garbage at the same time.
Evaluation. Participants: Spark! has been tested during three user tests. Two students (12-13 years old, VWO) participated in the first test. Four students (12-13 years old, VWO) participated in a session that lasted 100 minutes. Fourteen students (aged 12-13 years old, HAVO) participated in 5 lessons of 50 minutes. During the first two user tests the first author of this paper took on the role of teacher. For the third user test all lessons were independently given by a teacher. Design: The Spark! concept as shown in figure 3. Procedure: The first user test had the goal to test the overall usability of the learning toolkit. The second user test examined the amount of influence students had on each other and tested to what extend students could complete the process independently by using improvements done after the first user test. The third user test tested the final design, in which a teacher independently gave the class to 14 students, using the briefing for teachers to help him give the lessons. Analysis: reflection on outcomes based on observations during the user test by both designer and teacher. A short survey for both students and teacher held after the third user test. DBL process: Students were able to complete the whole design process using the provided briefing by going through all provided steps. Students seemed to have fun during the whole design process and were proud of their end results. Students collaborated intensively during the whole process and were able to come up with various design solutions for the proposed societal problem Learning goals: Students were challenged in the area of creativity, critical thinking, problem-solving capabilities and reflection skills (21 st century skills) and were also challenged concerning Spark4Arduino (the visual programming environment) and making electronic circuits (knowledge). It was however difficult to establish how much they exactly learned during the user test. During reflection students mentioned they learned about Snap4Arduino, making an electronic circuit and thinking about and defining problems and general societal problems like street litter. The user tests showed reflection is difficult for students. The second and third user test showed that after reading ''prompt questions'' students were able to formulate more grounded answers to the main question. Prompt are questions like: ''Why do we think this solution works?''. Reflection remained to be difficult for the students however, often needing help from the teacher to reach enough depth. In addition to this, all involved teachers during the process saw opportunities to connect physics and sociology objectives to Spark!.
Design brief and DBL support: Guidance and examples in the briefing helped students independently complete the design process. However, observations showed that students are easily influenced in one direction and that they do sometimes need help from the teacher to achieve enough depth with difficult tasks like reflection. The second user test showed that the introduction course for the technological parts of the toolkit enabled students to use the toolkit for prototyping independently. Using the briefing for teachers, the teacher of user test #3 could give a clear introduction of the lesson for children. The teacher also made his own planning for using Spark! Taking 5 lessons of 50 minutes for the whole process. 5 lessons were however a little bit too short for fully completing the process. The prototyping phase took more time than the teacher expected.
Toolkit property reflections: The students creatively use all contents of the toolkit to make their prototypes. Students also combined the materials and tools from the toolkit with other materials and tools that they sometimes even brought in from outside the classroom. Different materials also allowed students to come up with different design solutions.

5.2
Design case 2: Dolly X4.2 The focus was on Dutch children (11)(12) within their final years of primary school. Concept. Dolly X4.2, is a method developed to introduce DBL and give children the opportunity to make the design process their own in order to develop 21st century skills. The method creates the awareness that technology is everywhere, it shows the added value of technology and it provides depth by exploring technical principles in detail. Furthermore, the method can be connected to other courses. The method is based on growth: groups of children create their own fantasy animal that relates to other created animals in the classroom. The phases of growth: "Pre-birth"; understand what is needed to be born. "Birth"; understand what is needed to survive as individual. "Growing up"; understand what is needed to be part of a population and what is the relation to other animals. The name "DOLLY X4.2" is inspired by the cloned sheep 'Dolly', children are triggered to think like inventors and create their own animal. The intended use, covers at least 15 sessions of 1 hours. The methods elements: ─ A teacher guide describing the different process phases of the method and the reason behind each phase. It contains design activity cards for each phase that explain the activity, show connected learning goals and what kind of materials you need. ─ A toolkit containing transparent objects with different electronic circuits that demonstrate an actionreaction principles (from the categories: colour, vibration, sound and temperature). For example, a sensor that captures a colour and a RGB LED that displays the same measured colour. ─ Worksheets for students that guide children in decision-making steps. Children have to write down why they made a certain when they create a fantasy animal. For the teacher it is possible to review answers in their own time and see what the children based their choices on. ─ An evaluation instrument that helps to measure progress of gained knowledge and skills over time. With this form, the level of reflection can be scored. Level 1 describes that children have put no extra thought in why they wanted a certain characteristic (it's just cool) while level 3 describes a higher level because children have thought about how animals survive.
Scenario concept. The method consists of three process phases. The purpose of phase one is to, by exploratory learning, try out different technical modules. Small activities after each module will be an introduction to the DBL approach. Phase two and three provide an iterative design process and include different activities. The teacher is able to follow a script described in the method or to select provided design activities that fit the curriculum. In total, the three phases can be integrated within one school year.
Evaluation. Participants: Two tests have been executed to examine elements of the developed concept. In each test two groups of four children participated (16 children). Design: The DOLLYX4.2 concept as shown in figure 4.

Procedure:
The main question of the first test was: do children understand that they can use the modules of the toolkit to build their fantasy animal and can they use the technical principles as functions of their animal? The main question of the second test was: do children understand the principles of the modules and can they explore them? Both user tests were conducted at a Dutch primary school. The teacher took a role as supervisor, while in both tests the second author took the role as teacher.
Analysis: reflection on outcomes based on observations during the user test by both designer and teacher.
DBLA: All children were able to create a fantasy animal on paper and formulate different characteristics, such as 'I want my animal to be pink because it is my favourite colour' or 'I want my animal to be invisible because it will be able to hide from predators'. The different reasons showed an already different levels of reflection. The evaluation tool provided the possibility to score these levels. In an interview afterwards the present teacher expressed the added value about the way these skills could be measured because of the rating and time management possibilities. During the process of making design decisions together, children improved their designs constantly. For example, children first had to draw their super animal. Secondly they had to select 5 characteristics and mention why they selected them.
Learning goals: Some children had difficulties with describing the working principles of the modules during the second test and needed some explanation before they realized what they could do with it. After they explored a few modules, the children were given a short design brief in which they had to solve a problem. It revealed that children integrated knowledge gained during previous design activities in their solution. This was observed without mentioning they had to implement this knowledge. For example, children had just explored modules about temperature, vibrations and sound and did implement this knowledge immediately.
Design brief properties: In both tests children were enthusiastic and eager to think of a 'super animal' together. The children were able to read the explanation on paper because the text was lively and written from their perspective (image that YOU were a super smart inventor...). The children were already well informed by their teacher who had made sure the children knew what they could expect. This resulted in much curiosity from the children. This curiosity will probably stay over time because different (design) activities can be introduced.
Toolkit properties: Most children were curious about the inner workings of the modules and liked the fact that they could see the electronics. They were surprised that they could influence what happened (by action, reaction principles) and how they could suddenly come close to technology. For some children it was hard to make the connection between the technical modules and the animal that they created on paper, therefore the actual building of an animal was hard. For example one kid asked: "But does the animal have to hold the battery all the time?" This confusion is possibly due to the fact that the children did not have an introduction about the working principles of the modules during test one (in an ideal situation this would be a phase before the animal creation phase). A little explanation helped them to imagine new ways of using the modules, for example by using a colour sensor and an RGB LED as eyes.

Conclusion and discussion
In this paper we present a framework for teaching (a sub-set of) 21 st Century skills with digital toolkits in a (Dutch) school context. Two design cases and a literature study have been conducted to examine how DBL could offer children a different and playful approach to learn 21 st century skills with digital toolkits. The framework and the two case studies create a basis for others on how to develop an integrated approach that suits a school context. -A toolkit in combination with a DBL approach offers creative, playful learning. The user tests confirmed that children are indeed less likely to see lessons with a tangible toolkit as a traditional learning tool. The added value of technology can be exposed when combining a technology toolkit with design cases and activities that are related to real-world problems. This makes a connection between society and technology. Real world problems furthermore keep the activity close to the own imagination of children. The cases showed that children engaged with the topics in the design briefs, were eager to find design solutions and had fun while doing so.
-It is possible to stimulate 21 st century skills with DBL and digital toolkits. We also found it is possible to combine teaching 21 st century skills and course-based learning goals through DBL. It is important to connect existing course based learning goals to future learning goals (21 st century skills). Without addressing both current and future learning goals, a digital toolkit is not applicable in practice. Connectedness makes sure that the toolkit covers more areas then just technology, this makes it more appealing for schools to implement it. Therefore, when other similar design based learning solutions are created, a designer should carefully focus on what children can learn on a current course-based level and on a future learning goal level (21 st century skills).
-Children can relatively independently complete design processes when provided with a well thought out design process, toolkit and briefing. There are multiple criteria surrounding these aspects to DBL with toolkits. These criteria are defined in the framework mentioned in section 4. A teacher has relatively little time per student. Because of this a good briefing is a very important aspect in the framework. A briefing can however not replace a teacher. A teacher will remain to be the facilitator of learning, actively stimulating and helping children where necessary.
Although our design explorations delivered promising first results, further research is needed to get a better understanding on how an integrated DBL approach can be embedded in a real school context. The challenge in applying the framework is in developing an integrated solution tailored to a specific context: e.g. embedded in a school context, linking to appropriate course learning goal, using a digital toolkit that can be applied by the teacher in a set of learning activities. The framework provides a good starting point for further work. For example on how the properties of digital toolkits can best be linked to different types of course-based learning goals. In addition to this, our user tests showed that although our design solutions are specifically designed for usage in classrooms, the teacher still has to put a lot of effort in getting to know design based learning and applying it in class. This was also described by Kolodner [3]. Twenty-first century skills seem to form the future of education, actual development of these skills is however difficult to measure. Implementing and measuring 21st century skills in a non-isolated way will ask for a big change in the current form of education. In our opinion and in the opinion of most consulted experts, this will however improve the connection between education and (future) society.