ACMMM2014 Serious Games Workshop paper

of 6
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
A natural interface for the Surgical Safety Checklist Training
  A Natural and Immersive Virtual Interface for the SurgicalSafety Checklist Training Andrea Ferracani, Daniele Pezzatini, Alberto Del Bimbo Università degli Studi di Firenze - MICCFirenze, Italy [name.surname] ABSTRACT Serious games have been widely exploited in medicine train-ing and rehabilitations. Although many medical simulatorsexist with the aim to train personal skills of medical oper-ators, only few of them take into account cooperation be-tween team members. After the introduction of the SurgicalSafety Checklist by the World Health Organization (WHO),that has to be carried out by surgical team members, severalstudies have proved that the adoption of this procedure canremarkably reduce the risk of surgical crisis. In this paper weintroduce a natural interface featuring an interactive virtualenvironment that aims to train medical professionals in fol-lowing security procedures proposed by the WHO adopting a‘serious game’ approach. The system presents a realistic andimmersive 3D interface and allows multiple users to interactusing vocal input and hand gestures. Natural interactionsbetween users and the simulator are obtained exploiting theMicrosoft Kinect TM sensor. The game can be seen as a roleplay game in which every trainee has to perform the correctsteps of the checklist accordingly to his/her professional rolein the medical team. Categories and Subject Descriptors H.5.2 [ Information Systems Applications ]: InformationInterfaces and Presentation— Miscellaneous  ; J.3 [ ComputerApplications  ]: Life and Medical Sciences— Health  General Terms Gaming, edutainment, human factors, health Keywords Serious games, medical training, immersive environments,Surgical Safety Checklist Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from SeriousGames’14,  November 07 2014, Orlando, FL, USACopyright 2014 ACM 978-1-4503-3121-0/14/11. $15.00. 1. INTRODUCTION The concept of ‘serious games’ has been used since the1960s whenever referring to solutions adopting gaming witheducational purpose rather than pure players’ entertainment[1]. Among the several fields on which serious games havebeen exploited, medicine is one the most prolific [12] count-ing a large number of applications that feature ImmersiveVirtual Environments (IVEs). Improvements in medicaltraining using gaming and IVEs have been brought out es-pecially in the field of surgical education where IVEs alreadyplay a significant role in training programmes [20][7].The Off-Pump Coronary Artery Bypass (OPCAB) game[8] and the Total Knee Arthroplasty game [9], for exam-ple, focus on the training of decision steps in a virtual op-erating room. Serious games featuring IVEs about differ-ent topics are Pulse! [6], for acute care and critical care,CAVE TM triage training [3] or Burn Center TM for the treat-ment of burn injuries [17]. Though all these medical trainingsimulators focus on individual skills, an important aspect of healthcare to be taken into account is that, for the most, ithas to be provided by teams. Common techniques of teamtraining in hospitals include apprenticeship, role playing andrehearsal and involve high costs due to the required per-sonnel, simulated scenarios that often lack realism, and thelarge amount of time needed. Several serious games fea-turing IVEs for team training have also been developed inthe last years. 3DiTeams [6], CliniSpace TM [18], HumanSim[22], Virtual ED [26] and Virtual ED II [14] are some ex-amples of games in team training for acute and critical carewhose main objective is to identify and reduce the weak-nesses in operational procedures. A virtual environment fortraining combat medics has been developed by Wiederholdand Wiederhold [25] to prevent the eventuality of post trau-matic stress disorder. Medical team training for emergencyfirst response has been developed as systems distributed overthe network by Alverson et al. [2] and Kaufman et al. [16].IVEs have proved to be an effective educational tool. Taskscan be repeated in a safe environment and as often as re-quired. IVEs, in fact, allow to realistically experience a widerange of situations that would be impossible to replicatein the real world due to danger, complexity and impracti-cability. Though IVEs have been traditionally associatedwith high costs due to the necessary hardware, especiallyto provide realtime interactivity (multiple projectors, inputdevices etc.), and have always presented a difficult setup, inthe last years these issues have been partially solved by theavailability of cheap and easily deployable devices such asNintendo’s Wii or Microsoft Kinect TM .  Moreover 3D technologies used in IVEs offer several ad-vantages in medical education training. In this context it isessential to provide realistic representation of the environ-ment and several points of view in order to create in learnersthe correct mental model and to reinforce the memorabilityof a specific procedure. On the basis of the constructivismtheory [11], one of the main feature of educational IVEs isthe possibility of providing highly interactive experiences ca-pable to intellectually engage trainees in carrying out tasksand activities they are responsible for. The opportunity tonavigate educational scenarios as first person controllers al-lows learners to have direct awareness of the interaction con-text and of their responsibilities than in sessions mediatedthrough an un-related element such as a graphical user in-terface or an other symbolic representation. In this regardIVEs present a less cognitive effort in elaborating the con-text and stimulate imagination. This is even more true inscenarios where not only skills are learned in the environ-ment where these will be applied, but also the tasks can becarried out by the trainee acting in a natural way withoutthe mediation of any device (e.g. keyboard, mouse or othercontrollers). It is also pointed out by constructivists thatlearning is enhanced when gaming, group-work activity andcooperation are provided [10]. Interacting with humans ismore interesting and involving than interacting with a com-puter and it implies personal responsibility. Each learneris expected by others to account for his/her actions and tocontribute to the achievement of the team goal. The use of gaming techniques in education is called ‘edutainment’ andit is preferred by trainees to the traditional ones because itincreases students motivation and engagement in the learn-ing context [15].Anyway, despite the perceived advantages and the goodappreciation of serious games featuring IVEs for training,the adoption of such system is still poor in real medical fa-cilities. This is due to the fact that some open issues stillexists in systems’ usability, such as the facility to get lost dueto the possibility of free movement or the difficulty to pro-vide contextual and significant set of options in the environ-ment without switching to traditional device controlled 2Dinterfaces (panels visualizing multiple choices or drop-downmenus). In particular it is an essential negative aspect of allthe mentioned system for team training that, for the mostpart, trainees have to interact with the interface via com-puter monitors or head mounted displays. This reduces theimmersive effect and hinders the desired natural collabora-tiveness of participants in the simulation. In this regard wethink that serious games could benefit a lot adopting naturalinteraction techniques and exploiting new low-cost devicesavailable on the market for navigating and controlling IVEs.In this paper we introduce a serious game set in an immer-sive virtual environment with the aim to train profession-als in surgery to carry out efficiently the Surgical SecurityChecklist (SSC) introduced by the World Health Organiza-tion in 2009. The system features natural interaction tech-niques and it is easily deployable requiring only a standardPC, a projector and a Kinect TM sensor. The proposed sys-tem is so part of the vocational training systems where theneed is to train an activity usual in the everyday job of sur-geons. The paper is organized as follows: Sect. 2 describesin details the SSC guidelines and defines the proposed sim-ulation scenario. In Sect. 3 the architecture and the mainmodules of the system are discussed along with some imple-mentation details. Finally, conclusions are drawn in Sect. 4. 2. SURGICAL CHECKLIST SIMULATION Surgical-care is a central part of health-care throughoutthe world, counting an estimated 234 million operations per-formed every year [24]. Although surgical operations areessential to improve patients health conditions, they maylead to considerable risks and complications. In 2009, theWorld Health Organization (WHO) published a set of guide-lines and best practices in order to reduce surgical compli-cations and to enhance team-work cooperation [13]. TheWHO summarised many of these recommendations in theSurgical Safety Checklist, shown in Fig. 1. Starting formthe proposed guidelines, many hospitals have implementedtheir own version of the SSC in order to better match re-quirements of internal procedures.The SSC identifies three main distinct phases correspond-ing to a specific period during the execution of an operation:“before the induction of anaesthesia”,“before the incision of the skin”, “before the patient leaves the operating room”.In each phase, the surgical team has to complete the listedtasks before it proceeds with the procedure. All the actionsmust be verified by a checklist coordinator who is in chargeto guarantee the correctness of the procedure. The goal is toensure patient safety checking machinery state and patientconditions, verifying that all the staff members are identifi-able and accountable, avoiding errors in patient identity, siteand type of procedure. In this way, risks endangering thewell-being of surgical patients can be efficiently minimized.Gawande et al. [4] conducted several simulations of a sur-gical crisis scenario in order to assess the benefits obtainedby the adoption of the SSC. Results have shown that theexecution of the SSC improves medical team’s performanceand that failure to adhere to best practices during a surgicalcrisis is remarkably reduced. Figure 1: The Surgical Safety Checklist proposed bythe WHO in 2009. The proposed system is a serious game featuring an IVEto train and practice users in the accomplishment of the sur-gical checklist. It adopts natural interaction via gestures andvoice. The system  de facto  acts as the ‘checklist coordinator’of the surgical team.The simulation involves multiple trainees (up to three),each of them associated with his/her role, and guide themthroughout a complete SSC in a surgical operation scenario.  Three actors (surgeon, anesthesiologist and the nurse), ex-pected to complete the checklist, are automatically detectedby the system when approaching the IVE and can interactgesturing and speaking as in the real life. Interactions andchoices are not mediated by haptic devices or other con-trollers but mimic how the real procedure should be. 2.1 Simulation scenario The virtual simulation allows the three health profession-als who are going to execute a surgical operation to com-plete the SSC, with respect of their professional role. TheIVE was designed with the objective to help the trainees tounderstand the correct procedures to be followed.Professionals (i.e. trainees) stand in front of the simu-lation interface (see Fig. 2). The simulator associate userswith a professional role on the basis of their position in thephysical space. In practice, the user standing on left will beassociated with the anesthesiologist, the one in the centrewill be the surgeon and one on the right will be the nurse. Figure 2: Avatar selection: the trainee can choosewhich role to enact standing on the left, on the cen-tre or on the right of the IVE. Once every user is associated with a role, the IVE is shownand the simulation can start. The first environment repre-sents the pre-operating room, where usually the“before theinduction of anaesthesia” part of the SSC takes place, withthe patient and the three professionals’ avatars (see Fig. 3).From this moment, user can take control of the simulationby taking a step ahead. When one of the user has control,the environment is shown from his/her first-person point of view (POV). Hence, the active user can interact via voiceor gesture in order to carry out the specific step of the SSCprocedure.Interactions can be performed both by voice and handgestures. Voice-based interactions are used during the SSCwhen one of the professionals is expected to communicatewith the patient or with another team’s member. For in-stance, one step of the SSC simulation contemplates thenurse confirming to other team members the site of the op-eration, let’s imagine it to be the right arm. Accordingly,the user enacting the nurse should take a step ahead andsay something like “we are going to operate the right arm”or a similar sentence. As soon as the sentence is pronounced, Figure 3: A 3D pre-operating room environmentfrom a first-person POV, showing the patient andother professionals’ avatars. the system verifies if the site of the operation is correct, itupdates the SSC and it gives feedback in order to continuewith the simulation.Hand gestures instead (e.g. hand pointing and push) areused for other types of interaction, such as to touch thepatient, to check the state of the medical equipment or toactivate virtual menus. In the“before the induction of anaes-thesia”part of the procedure the nurse has to indicate withhis/her hand the part of the body to be operated. The IVEdisplays the patient with a view from above lying on the bedin order to allow the trainee a better precision of movementin the 3D space. Contextually an hand pointer is shown,mapped to the real position of the nurse’s hand, which al-lows him/her to select the body part as shown in Fig. 4. Figure 4: The nurse indicates the patient’s bodypart to be operated pointing his/her hand. Furthermore, during the simulation, the active user canperform a  swipe   gesture with his/her hand to activate a vir-tual overlay containing all the information about the pa-tient clinic history and the status of the SSC to be carriedout. The overlay simulates the patient’s medical card, usu-  ally available in the operation room. When all the steps of the checklist are correctly performed, feedback is given totrainees of the good outcome of the simulation and a sum-mary is presented with the time spent in completing theprocedure and the recording of all the errors committed byeach user.The system exploits these data in order to provide thesimulation with some gaming aspects: 1) the team mem-bers share the goal to correctly complete the checklist in theshortest time (the system provides a ranking table of thebest team scores); 2) each trainee competes with the otherteam members and his performance is measured by a scor-ing system which keeps track of all the individual errors andshow the results at the end of the simulation.Although the SSC have been strictly defined and formalisedby the WHO at an high level, the ‘content’ of the simula-tion sessions (i.e. the patient’s anamnesis and clinic card) isfully configurable in the system. This means that instruc-tors can simulate, and trainees experience, all possible op-erations and risks, and therefore that specific scenarios canbe created for teams of medical professionals in every field. 3. THE SYSTEM The proposed simulation system is a natural interfacethat exploits a realistic 3D interface and natural interactionparadigms in order to train users to correctly execute theSSC. Users stand in front of a large-sized screen or projec-tion and can interact without using any wearable or hand-held device. Contrariwise they can use their voice and bodyas interface controller. The system is composed by two mainmodules: ã  The Interaction Detection Module (IDM). ã  The Immersive 3D Interface and the associated GameStatus Controller (GSC).Figure 5 shows how these modules are organized from a log-ical point of view in the simulator.The remainder of this section describes in details the mod-ules. Finally, some technical implementation details are pro-vided. Figure 5: Logical system architecture: the IDManalyses motion and depth data coming from theKinect TM sensor, it detects actions and communi-cates with the GSC that updates the IVE on thebasis of the scenario’s configuration. 3.1 Interaction Detection The interaction between users and the IVE is obtainedtracking movements and identifying users’ actions by ex-ploiting the Microsoft Kinect TM sensor [27]. In particular,the interface can be controlled by trainees using their po-sition in the physical space, hand gestures and voice (seeFig. 6). The three different types of interaction can be ex-ecuted concurrently or are turned off/on depending on thephase of the simulated procedure. For instance, if the sim-ulator is waiting for a gesture input from the active user,the speech recognition module is temporarily switched off.The IDM is responsible of recognizing and notifying the useractions to the GSC module in order to proceed with the sim-ulation. Figure 6: Different types of interaction the systemis able to detect. In details, the IDM is able to detect: Active user.  The Kinect TM sensor is able to identity upto six human figures standing in front of the cam-era, but it can only track two skeletons simultaneously.Since the system is designed for three users, a policyis needed to dynamically define which of them is con-trolling the simulation (i.e. the interface). From thedepth map obtained by the sensor, the IDM detectswhich user is closer to the interface. When a traineeperforms one step ahead, resulting in a reduction of thedistance on the  z-axis  , the module notifies a change of the active user. This detection is active during all thesimulation session. Hand gestures.  Once the active user is identified, skeletontracking is exploited to detect his movements and, inparticular, to track his/her hand position in the space.The hand position is used to map an hand pointer inthe IVE used to interact with interface elements. TheIDM tracks the active hand of the trainee and sends itsspatial coordinates in the 3D space in order to updatethe interface. When the user needs to ‘activate’ somevirtual element on the interface, he/she must performa  push   gesture with the open hand. This is somehowsimilar to a  click   on a mouse-based interface. Handstracking and gesture updates can be switched on/off by the GSC, depending on the phase of the simulation.Furthermore, a  swipe   gesture has been provided thatallows the user to open a virtual 2D overlay on theinterface containing the patient case history and cliniccard. This gesture is performed by moving the rightarm from the right to the left. Speech inputs.  The Kinect features an array of four sepa-rate microphones spread out linearly at the bottom of 
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks