Assessing Social Engagement in a Digital Role-Playing Game

. Classcraft is a digital role-playing game dedicated to classroom management. Teachers can create teams and assign an avatar to students, as well as points and ‘powers’ as rewards for desired behaviour. We conducted a study in 4 classrooms from Switzerland. The study aimed at assessing to what extent the game fosters the social component of students’ engagement. The detection of so-cially-engaged behaviours is based on the monitoring of players’ behaviours. We collected and analysed players’ digital traces with kTBS4LA, a platform dedicated to playing analytics. The data collected shows that social engagement varies across time or gender. This variation seems to be linked to specific features of the game and also depends on how the game is played.


Managing the classroom with a role-playing game
Classcraft is a digital role-playing game dedicated to classroom management [1].The objective of Classcraft is to transform the classroom into a role-playing game for the duration of the school year.Teachers can create teams and assign avatars to students.They also assign points and 'powers' as rewards depending on students' behaviour, defined by specific game rules.In order to acquire powers, a player must demonstrate behaviour expected by the school, such as doing homework or being on time for class.The students play as warriors, mages or healers and, depending on their avatar, they can acquire and use specific powers that have impact on real life.For example, a student being late to class may be saved from punishment if one of his teammates uses the power called "Protect_1'.The teacher is the game-master.He deducts points or rewards depending if students' behaviours fit (or not) the classroom rules.Collaborative behaviours are expected from students, and players own individual or collaborative powers.

Classcraft as ludicisation of classroom management
Classcraft aims to convert an ordinary class into a playful situation.Each player, depending on his avatar, plays a specific role and owns specific powers.Players are rewarded or punished by the game master.Each week, a random event provokes positive or negative consequences, such as acquiring or losing powers.This use of game features, in order to convert non-game context into a game, is called gamification [2].However, Classcraft is not limited to the use of game features in a mechanical way.
Classcraft is based on a metaphor that changes the meaning of the action performed by the students.The actions themselves are not altered.The students attend class on time, do their homework or collaborate for learning activities.However, the meanings and the motives of these actions are changed when Classcraft is played.The classroom becomes a battle, where players have to overcome difficulties and to develop specific skills.Sanchez et al. called ludicisation [1] the conversion of a non-game context into a playful situation aiming at fostering gameful experiences and students' engagement.

Objectives of the study: assessing students' social engagement
There is an ambiguity about the concept of players' engagement.Different definitions have been provided [3] and, due to the ambiguity of the related concepts and their context-dependent definitions, assessing players' engagement might be difficult.Our approach is based on Self-Determination Theory (SDT) [4], which states that motivation results from innate psychological needs.Based on SDT, Bouvier et al. [3] consider that players' engagement encompasses four components: the environmental component in relation to autonomy needs; the self-component that relates to autonomy needs; the action component linked with competence and with autonomy needs.The social component is in relation to relatedness.By playing Classcraft, students are expected to help other students and to collaborate with teammates.Thus, Classcraft aims to foster players' social engagement.As a result, the objective of our study aims at assessing the social component of the players' engagement.For this study, we examine the evolution of the student's social engagement during a school year.We also examine gender differences.

3
Methodology of the study

Monitoring the player: playing analytics
In order to assess a player's social engagement, we developed a specific methodology based on playing analytics [5].We detect engaged-behaviours by monitoring players.Players' digital traces are players' actions performed during a digitally-mediated activity.Obsel (observed elements) are automatically collected.Obsels [6] are elementary players' actions (like buying or using powers).Each obsel is characterised by a type of event, a timestamp (beginning and end of the event) and information that is useful to derive meaning (attributes and relations with other obsels).Some obsels, like using collaborative versus individual powers, are inherent to players' social engagement.Indeed, we consider that the use of collaborative powers (powers that have a positive impact on a teammate) brings information on the participation of the player to a collaborative play, where the outcomes of the game depend on the capacity of players to take their teammates into consideration.Thus, we consider that the use of collaborative powers demonstrates that a player is socially engaged with his/her teammates.

Data collection and data processing
The data collected come from 11 classes in Switzerland.For each class, the data consist of 8 JavaScript Object Notation (JSON) files.The main file (logbook.JSON) is a list of obsels collected during a session dedicated to play.The logbook.JSON file is completed with information coming from the other files: gender of the players; different teams involved; and game-levels reached by a given player.These operations are carried out with the data collected from the different classes.However, four sets of data have been selected for this exploratory study.The data analysis is performed with a specific digital platform called kernel Trace Based System for Learning Analytics (kTBS4LA) [7].The data collected are uploaded, and different functionalities dedicated to data processing are available.Some of them were developed for the needs of this study.First, a kTBS4LA export model is created.This model gives a precise description of the digital traces uploaded on the platform.The platform allows for selecting specific obsels from the data collected.The obsels that are considered to be relevant for the study are extracted from the whole dataset.kTBS4LA enables the visualisation of these obsels along a timeline and different colours and shapes can be used for making apparent specific features.In order to obtain consistent results, the same protocol is applied to each dataset.We designed and recorded several scenarios for the visualisation of obsels such as the use of collaborative powers or individual powers according to gender and teams.

Results and discussion
The 4 classes selected for this preliminary study have an average of 16 students, 7 girls and 9 boys, and 3 teams per class.We collected data for 4 to 7 months, depending on the different classes.Table 1 shows that 'the activity' of the different classes (measured by the number of obsels collected and powers used by students) varies a lot.Class 4 is a specific case.The data collected show that the teacher gave all the powers to everyone, right from the beginning of the game.For classes 1, 2 and 4, collaborative powers are increasingly used during the school year.However, it is not the case for class 3.For this class, there is a balance between individual and collaborative powers used by the students.As a result, for 3 classes, students' social engagement increases during the school year.The students are increasingly involved in collaborative play.The game seems to have the expected influence on students: fostering collaborative behaviours.However, the differences observed with class 3 show that the influence of the game is complex, and might depend on different factors.Students' social engagement also probably depends on how the game is played by the students and orchestrated by the teacher.

Class 1
Collaborative  Playing analytics do not provide information regarding this issue, so further classroom observations or/and teachers' and students' interviews are needed.

Variation of social engagement depending on students' gender
The data collected from the 4 classes selected for the preliminary study also show differences between how girls and boys play.While boys use more collaborative than individual powers, they still use a lot of individual powers in comparison to girls.Girls are less active but more efficient, and use few individual powers (see Table 2).Social engagement varies according to the students' gender.Girls are more involved in collaboration than boys.

Conclusion
Based on the data collected from preliminary study, there are arguments to state that the ludicisation of classroom management with Classcraft helps students to develop positive classroom behaviour.The game influences how students collaborate with their teammates and, therefore, fosters students' social engagement.Changing the meaning of the situation experienced by the students helps them to take into account their teammates.Students' social engagement varies along the school year and according to gender.However, the differences observed for different classes also show that the way the game is orchestrated by the teacher influences how the game is played by the students.Playing analytics do not offer any information regarding this issue and further investigations are needed.Thus, we plan to continue to address this issue with a mixed methodology.The project will consist of collaborative work with voluntary teachers for the implementation of the game in different classes.The combination of learning analytics and classroom observations should enable the characterisation of students' social engagement and the identification of the game elements that foster students' collaboration.Therefore, we expect to gain a better understanding of how ludicisation might be applied for classroom management.

Figure 1
Figure1represents collaborative (green) versus individual (red) powers used by the students during the school year (time line).For classes 1, 2 and 4, collaborative powers are increasingly used during the school year.However, it is not the case for class 3.For this class, there is a balance between individual and collaborative powers used by the students.As a result, for 3 classes, students' social engagement increases during the school year.The students are increasingly involved in collaborative play.The game seems to have the expected influence on students: fostering collaborative behaviours.However, the differences observed with class 3 show that the influence of the game is complex, and might depend on different factors.Students' social engagement also probably depends on how the game is played by the students and orchestrated by the teacher.

Figure 1 .
Figure 1.Powers used by the students from the different classes during the school year

Table 1 .
Information about the different classes of the study

Table 2 .
Uses by gender: collaborative powers versus individual powers