ADITHO – A Serious Game for Training and Evaluating Medical Ethics Skills

. This paper presents “A Day In The HOspital”, a Digital Serious Game aiming at providing a technological tool for both evaluating and training ethical skills of medical staff personnel. During the game, the player interprets the role of a physician who has to perform a decision-making process that involves his ethical and medical skills. Usability and sense of Presence have been assessed through a specific post-game Likert-questionnaire. In order to evaluate the potential of the game as a medical training tool, experimental sessions have been conducted with two different groups of participants (“non-medical” and “medical” groups) and game outcomes have been statistically compared. Participants belonging to both the experimental groups report a high level of game Usability and sense of Presence. Finally, the statistically higher game score obtained by the “medical” group demonstrates the usefulness of ADITHO for evaluating and training ethical skills.


Introduction
Virtual Reality (VR) can be defined as a three-dimensional, computer generated environment which can be interactively explored by a user. Nowadays, VR is increasingly being used as a tool for education, training, teaching and dissemination purposes. At the same time, a growing interest in the use of video games for educational scope has been observed. As a part of Information and Communications Technology, video games are considered potentially powerful learning environments [1]. In this context Digital Serious Games (DSG), definable as "video games for purpose other than entertainment" [2], have been developed. In the literature, DSGs are mostly used in the field of education [3,4] and they are classified by several criteria based on the type of educational content, the principle of learning, the targeted users or the game technologies [5,6]. DSGs developed in three-dimensional scenarios could be considered as a subset of VR. In a VR-based DSG, factors such as Immersion and Presence are topics of crucial importance. Immersion is a powerful experience that contributes to the amount of information acquired, skills developed, and subsequent transfer of knowledge to real environments [7,8]. In the same way, the sense of Presence, definable as the subjective feeling of being in a different place from where one physically is [9,10], can increase motivation and provide a more engaging experience [11].
Training based on video games shows several benefits ranging from the safety training to the ability in transfer the acquired knowledge and skills to the operational environment [7,12]. DSGs can also support the development of several competencies such as analytical and spatial skills, strategic skills, recollection capabilities and psychomotor skills [2].
In the last decades, a great deal of research has pointed out the need of highly trained and educational health care professionals to avoid medical errors. The use of DSGs in health provide an efficient tool for increasing interest in training, education and evaluation of medical personnel [13,14]. Ethics in clinical professions is a matter that involves all the health care professionals, whatever their specialization. The importance of ethics in clinical profession is underestimated within the context of the whole medical curriculum. In particular, while students learn the theoretical contents of ethical knowledge, they commonly do not acquire more than a poor ability to act based on this knowledge [15]. The goal of education in ethics and professionalism should be oriented to develop a method to track students' capacity to apply their knowledge and skills in professional clinical scenarios [16]. It has been demonstrated that following an ethical approach is of fundamental importance for creating a shared decision-making process which is the ideal treatment for difficult clinical case scenarios [17,18]. Moreover, in order to make an ethically correct decision, it is important to create both a strong collaboration with other healthcare operators and a relationship with the patient and his family [19,20]. As aforementioned, DSGs have been developed for training or informing regarding topics such as healthcare, social learning and ethics [21]. To the best of our knowledge no DSG regarding the Medical Ethic have been developed and evaluated yet.
Motivated by the above, we have developed a first person VR-based DSG, "A day In The Hospital" (ADITHO), aiming at training and evaluating physician's ethical skills. In the presented game, the user plays the role of a physician who has to make a clinical decision based on the information he collects during the game. By simulating realistic clinical scenarios, ADITHO allows players to deal with complex relationship and to gain new skill on medical ethics. The use of 3D environments allows the user to experience a strong immersion and sense of presence that are factors that commonly imply an enhancement of the learning process [10].
ADITHO has been developed as a part of SONNA (www.sonna.unisi.it), a larger research project aimed at investigating novel technology-enhanced learning methodologies based on the use of social media and 3D Virtual Environments [22]. The implemented game architecture allows to easily extend the game to other different clinical case. The usability of the game has been already evaluated in a preliminary study with users non expert in medical field [23]. In this paper the potentiality of the developed game as a tool for training and evaluating ethical skills is investigated through two experimental sessions conducted with two groups of users which differ in their expertise in medical matters.
The "playability" of ADITHO, in terms of Usability of the system and in the users' perceived quality of the game experience, has been also investigated through a postgame questionnaire.

Material and methods
This section discusses the experiment presenting first the materials comprising the game architecture and the storyboard of the scenario. Then the methods are discussed presenting the experimental procedure, the questionnaires used for evaluating the game experience and the criteria adopted to assess the ethical profile of participants.

Game storyboard
The game takes place in a hospital ward where a young patient suffering from Cystic Fibrosis (CFi) is in treatment. CFi is a genetic disorder frequent in young Caucasian populations and it was selected for this game storyboard because it rises up important decision-making processes to deal with in the ethical field [24].
In the proposed game, the player in the role of a physician has to decide the best strategy for treating the CFi patient using his ethical competences and collecting information by talking with the patient himself, his relatives and with other physicians and caregivers.
ADITHO starts with the choice of the avatar gender and a help guide screen that describes all main commands. At the beginning, the user is located in the hospital hall and a nurse reminds him to visit the CFi patient. Then, a series of events connected with the evolution of the clinical case are presented to the player.
In order to proceed through the game story, players have to interact with other agents of the game (Non-Player Characters, or NPCs). The interactions with other NPCs are based on dialogues in which players select a sentence within a set of available ones (Fig. 1). Players can also perform several other actions that contribute to acquire information, such as reading e-mail, watching films, browsing the web, reading the patient's journal, etc. The game also provides a collaborative feature in order to simulate the real information-sharing process between the medical staff. Different players that are playing in the same moment can meet each other in the coffee room of the virtual hospital.

Game architecture
The game architecture is realized with a separation between the Core Game Engine (CGE) and a set of Game Information (GI). CGE has been implemented using the XVR technology [25], a framework that enables the realization of web-based and/or stand-alone interactive VR applications. GI describe the game in terms of a set of resources (images, sounds, video, 3D objects, etc…) and game mechanics (actions). An action is composed by functionalities and enabling conditions and it represents the relationships between the user's interactions and the resources. Depending on the CGE global states, a set of specific enabling condition could be verified allowing the corresponding action to be activated. Actions' functionalities consist by a set of reactions that could modify the CGE global state or the game scene. The separation between CGE and GI makes the architecture of the game versatile and thus the game could be easily expanded to other clinical cases in order to satisfy different educational needs.
With the purpose of giving users a greater sense of immersion in the environment, the game is also provided with sound feedback.
The game client can connect to a centralized server; this enables users who are simultaneously playing the game to communicate through an integrated text chat in the coffee room. Each user is visualized with an avatar of the appropriate gender ( Fig.  2). When a user enters the coffee room, other players are informed by means of a noninvasive notification (see Fig. 1, bottom left).
All dialogues are described in GI, and they are automatically adapted based on the gender chosen by the player. Dialogues are composed by sentences and/or questions with related answers. Sentences develop a linear dialogue whereas questions may generate an oriented graph structure without loops depending on the user's replies. In Fig. 1 is shown a screenshot of the game in which the player has to choose a sentence among three possible once, whereas in Fig. 3 is depicted a general scheme of a dialogue. Each answer contributes to calculate the ethics score of the user (more details are discussed in Section 2.3).
All users' avatars and NPCs are animated virtual humans realized with Autodesk 3D Studio Max and Motion Builder and integrated in the XVR environment. The text chat is shown on the left whereas the player's username appears when the mouse hover over her/him.  3. The oriented graph of a general dialogue. Sentences produce one exit way only; questions produce two or more exit ways, called answers.

Evaluation methods
This section describes the methods used for evaluating the questionnaire answers (concerning Usability and the perceived sense of Presence) and the game scores.

Usability and sense of presence evaluation.
The game Usability and sense of Presence have been evaluated through two different questionnaires.
In order to evaluate the Usability of the system, the System Usability Scale (SUS) [26] has been adopted. SUS is a 10 items Likert-scale questionnaire used to investigate the subjective assessment of usability by covering effectiveness, efficiency and satisfaction of the experience. Each statement indicates the degree of agreement or disagreement in a 5 points scale. The users' levels of immersion and involvement in the virtual game were evaluated by means of the Presence Questionnaire (PQ) [10]. PQ is composed by 19 items, rated on a 7-points Likert-scale. It uses 4 factors that exert their influence on presence by affecting both involvement and immersion:  Control Factor (CF) about degree, immediacy, anticipation, mode and physical environment modifiability;  Sensory Factor (SF) about modality, environmental richness, multimodal presentation, consistency of multimodal information, the degree of movement perception and active search;  Distraction Factor (DF) about isolation, selective attention and interface awareness);  Realism Factor (RF) about scene realism, information consistent with the objective world, meaningfulness of the experience, separation and anxiety/disorientation [10].
The PQ questionnaire was then customized in order to fit with the purpose of our study. We excluded 7 items of the questionnaire because we considered it not appropriate and not pertaining to the characteristics of the VR experience of ADITHO. We then selected 12 items that allow us to capture: (i) the ability to control the relation of sensors to the environment (Q1-Q5 and Q10-Q11); (ii) the consistency of the all information conveyed by the virtual world (Q4, Q6, Q8, Q12); (iii) the ability to focus on VE experience (Q9) and (iv) the realism of the stimuli perceived during game experience (Q7). All of these 12-items refer to the four factors explained before. Finally, in order to improve the game experience in future application, participants were asked to fill an open-ended question in which they provided feedbacks and suggestions.

Ethical profile evaluation.
In order to evaluate the participants' ethical profile, a numerical score is assigned during the game, depending on users choices, using a combination of two different criteria.
The first criterion is based on the latest Code of Medical Ethics 2014 (CME) [20], a system of moral principles establishing the ethical rules of behavior of physicians. Using CME criterion, each answer choice results in a score in the range [0, +4]. Since ethics looks at the inner attitudes and to the intentionality of the agents [27], the second criterion is founded on the four main principles of biomedical ethics postulated by Tom Beauchamp and James Childress [28]:  the principle of respect for autonomy (A), according to which physicians recognize self-determination of patients and equip them with the means to make reasoned informed choices;  the principle of beneficence (B1), considering the balance of benefits of treatment against the risks and costs;  the principle of non-maleficence (B2), stating that the harm of each treatment, even if minimal should not be disproportionate to the benefits of treatment;  the principle of justice (J), concerning the distribution of scarce health resources, and the decision of who gets what treatment.
Because of their similarities, principles B1 and B2 were combined in a single principle (B). Moreover, in order to evaluate the ethical attitude of the physician it was added another important criterion:  communication (C), concerning the physician's capability of accompanying the patient in his diagnostic-therapeutic path sharing meaningful information and showing cooperative attitudes.
Using the four principles criterion, each answer choice results in a score in the range [-2, +2] for each principle.
Since the CME is based on ethics principles universally shared [29], a series of Spearman rank-order correlations were conducted in order to determine if there were any relationships between the scores assigned to the four principles and the ethical code (A, B, J, C, CME). A two-tailed test of significance indicated that there was a significant positive relationship between the scores assigned to each answers (Spearman's rho(27) > .66, p < .01 for all the tests).

Experimental procedure
An experimental procedure was carried out for investigating the potentiality of the proposed game as a tool for training physicians' ethical skills. Two experimental sessions were performed with two different groups of participants: "non-medical group" and "medical group" (see Section 2.5 for further details). The experiment was conducted in accordance with the WMA Declaration of Helsinki and subjects provided written consent to participate.
All participants had to fill-in a preliminary questionnaire concerning their personal information, their familiarity with computer games, their knowledge of VE and DSG, and their knowledge of the CME.
Before starting the experiment, a demo session was experienced by participants to get familiar with the game environment. During this phase all players were located in the hospital ward where a NPC explains the functions of the game through the assignment of specific tasks. In order to complete the demo session players had to learn how to navigate through the environment and how to talk with other NPCs.
Once all participants had finished the demo session, the experimental game session started. All users of each session have played the game at the same time, using computers connected to a network. During the game users could not verbally speak to each other. At the end of the game a summary screen with the obtained score was shown to players and all participants were required to complete the evaluation questionnaire. Finally each participant has been shown the correspondence between each multiple choice answers and CME plus A, B, J, C scores.

Participants
Twelve volunteers (3 females and 9 males) with mean age of 30.6 (range 25-43), and ten residents from "Azienda Ospedaliero-Universitaria Pisana" hospital in Pisa, Italy, (6 females and 4 males) with mean age of 27.8 (range 26-31) were recruited for the experiment. All people recruited are independent of the project. The groups, respectively the "non-medical group" and the "medical group", did not differ either by age (t-test, p = 0.118) or by sex (Fisher Exact Test, p = 0.192). "Nonmedical group" is composed by people with experience in Virtual Reality and Computer Science. This group evaluated the usability, the sense of presence and immersion in the Virtual Environment developed on the game independently form the specific topic threated. The integration between the answers of the two groups gives a more complete overview in order to evaluate the game.
As aforementioned, before starting the experiment, subjects were asked to rate both their knowledge of the Italian CME and their previous experiences with Video Games and Serious Games in a scale ranging from zero (no previous experience) to five (expert users). As expected, the groups differed by the knowledge of CME (Mann-Whitney U test, p < 0.01) but they did not differ either by previous experience with Video Games (U test, p = 0.42) or by previous experience with Serious Games (U test, p = 0.16).

Results and Discussion
In the following sections, results and discussion of data analysis are reported for both questionnaire answers (SUS and PQ) and for game scores.

Questionnaire Results and Discussion
In order to compare answers between the two experimental groups ("non-medical" and "medical") a series of independent-samples t-test were conducted to compare questionnaire results (in terms of SUS score and the four PQ factors scores). Table 1 reports means and standard deviations of the SUS scores and the PQ factors scores for both the experimental groups and for the two groups merged.
Accordingly to Brooke [26], the SUS score is a percentage indicating the overall level of Usability of the game and it is obtained for each participant by the contribution of each answer score. Very high results were obtained for both groups (average scores of 74.54 ± 15.21% and 73.89 ± 21.96% for the "non-medical" and "medical" groups respectively, see Table 1). No statistical difference was found between the SUS scores of the two groups (t(20) = 0.082, p = 0.935). This means that no particular technical skills should be involved by the participants to play ADITHO. It follows that the game is "playable" for all medical personnel and it could be used for ethics training in clinical setting. This result confirms the achievement of our goal to create a tool easy to use by wide range of users.
Concerning the PQ, we analyzed separately the four different factors (CF, SF, DF, and RF, see Section 2.3 for more details) and respectively assigned a score to each subject [10]. Both groups obtained high scores for all factors (Table 1) and no significant statistical differences were found between the two groups (t(20) < 1.367, p > 0.187, for the four tests). This means that the subjective experience of participants was not affected by the topic addressed in the game, but it could be more correlated with the engagement solicited by the game story.
Results obtained in the PQ were extremely important as they support the idea that ADITHO could be a useful tool for training ethical skills. Indeed, it has been demonstrated that level of Presence positively correlates to the amount of information learnt and to performance obtained by the subjects [10,30].
The four factors (CF, DF, RF, and SF) define the level of immersion in the environment, the impact of outside distractions and the level of the active participation through the perceived control over events in the environment; high values of these factors commonly imply an enhancement of the learning process and performance [10]. Accordingly with this interpretation, 6 out of 10 subjects belonging to the medical group self-reported that the game is "potentially very useful" in particular "to pose questions about situations that inevitably we face in our work". In addition, mostly all subjects reported that "dialogues were very involving", "the game was very interesting" and that "the game could be very useful as training tool for medical candidate".

Game results and discussion
As explained in Section 2.3, the users' game choices resulted each on a set of scores assigned for the CME and for the four main ethical principles: A, B, J and C. Thus, at the end of the experimental session, based on the game answers, each participant obtained an ethical profile consisting in such scores. Scores obtained by the two groups were statistically compared using the non-parametric Mann-Whitney U test for independent samples. It resulted that the "medical group" obtained a statistically higher score for each analyzed variable. Bargraphs in Fig. 4 show the final comparison of game scores (in terms of CME and the four principles), averaged over the participants, for the "non-medical" and "medical" groups. As we expected "medical" obtained a score higher than "non-medical group" for all the scores. This is completely in line with the goal of ADITHO in which ethical competences are required in order to address the clinical case with an ethical approach. In fact, in the game professional competencies are required in order to face the clinical case with a correct ethical procedure. Moreover these results validate the efficiency of ADITHO as a useful tool for evaluating the ethical knowledge in medical personnel.
At the end of the experience each participant has been shown the correspondence between each multiple choice answers and CME plus A, B, J, C. In this way the players have become aware of the impact of their choices, improving their competence by learning from the simulated experience just lived in ADITHO.

Conclusion
In this paper we have presented a DSG called ADITHO together with the investigation of its potentials as a tool for training and evaluating the ethical skills of medical personnel. In a previous preliminary research, the usability of ADITHO has been assessed by performing an experiment with group of non-medical expert users [23]. In this research we have evaluated the game through an experimental test conducted on two groups of subjects: the "medical" and the "non-medical". From the analysis of the SUS and PQ questionnaires answers, no significant differences were found between the two groups. The high overall SUS score obtained by both groups shows that the game is easily "playable" by a wide range of users. Moreover, the high sense of Presence reported by all participants suggested that users not owning particular technological skills was as highly engaged and immersed in game experience as users more experts in the use of digital technologies. It is worth noting that a high level of engagement is usually associated with an improvement in the information retention and memory. In addition, the analysis of participants' ethical profiles (i.e. the game scores) has showed that the "medical" obtained a statistically significant higher score than the "non-medical group". This fact confirms the hypothesis that ADITHO might be used as a tool for evaluating ethical skills of medical personnel. At the same time, these results suggest that ADITHO could be an efficiency-training tool to improve the ethical competencies in physicians.
We envisage carry out future work in two main directions. The first involves the exploration of new game scenarios by implementing additional storyboards dealing with other clinical cases relevantly involving ethics (e.g. living will, religion belief) and by expanding the game to other medical personnel (e.g. nurses).
The second direction focuses on the improvement of game dynamics by improving graphics and audio features and by enhancing the interaction between simultaneous players. The improved interaction will allow to evaluate more deeply the social component of the game and how it influences the sense of Presence. Finally, in order to assess the effectiveness of ADITHO as a training tool, an evaluation by using different kinds of scenario will be carried out.
The final goal of ADITHO is to be included in a larger training protocol within the context of the whole medical curriculum.