Evaluating User Experience in Games: Concepts and Methods PDF
Preview Evaluating User Experience in Games: Concepts and Methods
Human-Computer Interaction Series Editors-in-chief JohnKarat IBMThomasJ.WatsonResearchCenter(USA) JeanVanderdonckt UniversitécatholiquedeLouvain(Belgium) EditorialBoard GaëlleCalvary,LIG-UniversityofGrenoble1,France JohnCarroll,SchoolofInformationSciences&Technology,PennStateUniversity,USA GilbertCockton,NorthumbriaUniversity,UK LarryConstantine,UniversityofMadeira,Portugal,andConstantine&LockwoodLtd, Rowley,MA,USA StevenFeiner,ColumbiaUniversity,USA PeterForbrig,UniversitätRostock,Germany ElizabethFurtado,UniversityofFortaleza,Brazil HansGellersen,LancasterUniversity,UK RobertJacob,TuftsUniversity,USA HilaryJohnson,UniversityofBath,UK KumiyoNakakoji,UniversityofTokyo,Japan PhilippePalanque,UniversitéPaulSabatier,France OscarPastor,UniversityofValencia,Spain FabioPianesi,BrunoKesslerFoundation(FBK),Italy CostinPribeanu,NationalInstituteforResearch&DevelopmentinInformatics,Romania GerdSzwillus,UniversitätPaderborn,Germany ManfredTscheligi,UniversitätofSalzburg,Austria GerritvanderVeer,UniversityofTwente,TheNetherlands ShuminZhai,IBMAlmadenResearchCenter,USA ThomasZiegert,SAPResearchCECDarmstadt,Germany Human-Computer Interaction is a multidisciplinary field focused on human aspects of the development of computer technology. As computer-based technology becomes increas- ingly pervasive – not just in developed countries, but worldwide – the need to take a human-centered approach in the design and development of this technology becomes ever more important. For roughly 30 years now, researchers and practitioners in computational and behavioral sciences have worked to identify theory and practice that influences the directionofthesetechnologies,andthisdiverseworkmakesupthefieldofhuman-computer interaction.Broadlyspeaking,itincludesthestudyofwhattechnologymightbeabletodo forpeopleandhowpeoplemightinteractwiththetechnology. Inthisseries,wepresentworkwhichadvancesthescienceandtechnologyofdeveloping systemswhicharebotheffectiveandsatisfyingforpeopleinawidevarietyofcontexts.The human-computer interaction series will focus on theoretical perspectives (such as formal approachesdrawnfromavarietyofbehavioralsciences),practicalapproaches(suchasthe techniquesforeffectivelyintegratinguserneedsinsystemdevelopment),andsocialissues (suchasthedeterminantsofutility,usabilityandacceptability). Forfurthervolumes: http://www.springer.com/series/6033 Regina Bernhaupt Editor Evaluating User Experience in Games Concepts and Methods 123 Editor Asst.Prof.ReginaBernhaupt UniversitéPaulSabatier,ToulouseIII InstitutdeRechercheen InformatiquedeToulouse(IRIT) 118RoutedeNarbonne 31062ToulouseCedex9 France [email protected] ISSN1571-5035 ISBN978-1-84882-962-6 e-ISBN978-1-84882-963-3 DOI10.1007/978-1-84882-963-3 SpringerLondonDordrechtHeidelbergNewYork BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary LibraryofCongressControlNumber:2009943127 ©Springer-VerlagLondonLimited2010 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced, storedortransmitted,inanyformorbyanymeans,withthepriorpermissioninwritingofthepublishers, or in the case of reprographic reproduction in accordance with the terms of licenses issued by the CopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesentto thepublishers. Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceofa specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree forgeneraluse. Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions thatmaybemade. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) ForPhilippe Foreword It was a pleasure to provide an introduction to a new volume on user experience evaluationingames.Thescope,depth,anddiversityoftheworkhereisamazing.It atteststothegrowingpopularityofgamesandtheincreasingimportancedeveloping arange oftheories,methods,and scalestoevaluate them.Thisevolution isdriven bythecostandcomplexityofgamesbeingdevelopedtoday.Itisalsodrivenbythe need to broaden the appeal of games. Many of the approaches described here are enabledbynewtoolsandtechniques.Thisbook(alongwithafewothers)represents awatershedingameevaluationandunderstanding. Thefieldofgameevaluationhastruly“comeofage”.ThebroaderfieldofHCI canbegintolooktowardgameevaluationforfresh,critical,andsophisticatedthink- ingaboutdesignevaluationandproductdevelopment.Theycanalsolooktogames forgroundbreakingcasestudiesofevaluationofproducts. I’ll briefly summarize each chapter below and provide some commentary. In conclusion,Iwillmentionafewcommonthemesandoffersomechallenges. Discussion In Chapter 1, User Experience Evaluation in Entertainment, Bernhaupt gives an overview and presents a general framework on methods currently used for user experience evaluation. The methods presented in the following chapters are sum- marizedandthusallowthereadertoquicklyassesstherightsetofmethodsthatwill helptoevaluatethegameunderdevelopment. In Chapter 2, Enabling Social Play: A Framework for Design and Evaluation, Isbisterexaminestheneglectedareaofsocialgames.Shereviewstheliteratureand considers sample games and some of the primary factors affecting the experience of social gaming. These include contextual factors, motivational factors, and the conceptual andtheoreticalgroundwhichshapesuchevaluations.Isbisterpointsto ecological validity being a primary focus when considering any testing method. She stresses the importance of looking at the entire set of measures including attitudinal and behavioral data. Her chapter creates a context for planning and conductingevaluationsofsocialgames. vii viii Foreword Chapter 3, Presence Involvement and Flow in Digital Games (Takatalo, Hakkinen, Kaistinen, and Nyman) is a report on a massive study using multivari- ateanalysistorevealthesubcomponentsofuserexperienceingames.Itreviewsthe historyofclassificationsofuserexperiencegoingbacktotheclassicpsychological categories ofthinking,feeling,andwill.Theauthorscompareandanalyze current concepts:immersion,fun,presence,involvement,andflow.Theyalsoreviewprevi- ousmultivariateanalysisofgamequestionnaires.Fromthisanalysis,theydevelop thePresence-Involvement-Flowframework(PIFF)whichencompassesbothtechni- calgamecomponentsandpsychologicaldeterminantofUX.Usingafactoranalytic study, they evaluate this model. The final questionnaire was assessed by compar- ing two different groups of gamers in two different games. In addition, the profile for games was compared to Metacritic scores and user ratings. The results were promising, with results of PIFF accounting for the important differences between gamesandilluminatingthelearningcurvesofdifferentusersduringthefirsthourof play.PIFFisverybroadinscopeandshowspotentialtoevaluateuserexperienceat differentstagesofdevelopment. InChapter4,AssessingtheCoreElementsoftheGamingExperience,Calvillo- Gamez,Cairns,andCoxassumeanambitiousgoal–creatinganewtheoryofuser experience for games. They adopt a novel process to achieve this end. They have chosen to use a bottom-up approach. They build the theory by analyzing video gamereviewsusingthemethodologyderivedfromgroundedtheory.Theiranalysis iscomplexcontainingtwocoreelements:puppetryandvideogame,eachwiththree elements. They term this theory the Core Elements of Game Experience (CEGE) theory. This theory is integrated into a model and translated into a questionnaire. The questionnaire is then applied to user experience in two versions of a popular game (Tetris). The results are as predicted, i.e., scales show no difference for ele- ments of thegame thatwerenotdifferent(visuals and sound) but aredifferentfor thosethatrelatetocontrol–puppetry.Theambitiousgoalthatwasproposedatthe beginningofthechapterismatchedbyanimpressiveresult–validationofatheory for gaming. Since the theory includes measurement instruments, it represents an advanceoverprevioustheories,inwhichthemeasurementisunspecified. Emily Brown reports on the current practice in the gaming industry about the LifeandToolsofaGameDesigner(Chapter5). InChapter6,InvestigatingExperiencesandAttitudesTowardVideogamesUsing a Semantic Differential Methodology, Lemay and Lessard explore the use of the semanticdifferentialtoevaluatevideogamesasleisureactivities.Theyofferanew toolforresearchonperceptionofgames.Astheypointout,havingastandardsetof questions and metrics for attitudes about games would provide a foundation for more research. This also could provide a foundation for analyzing the attitudes towardpotentialmarketsegmentsforagame.Theirresultsarepromisingandcould helpgamedesignersunderstandtheirintendedaudiences. In Chapter 7, Video Game Development and User Experience, McAllister and White provide a detailed overview of the testing done for three products. The chapter gives a snapshot of how evaluation is currently being done in industry. The methods and procedures are quite broad. The first case study recounts the Foreword ix development of Pure by Black Rock Studio. In that case study, three useful dis- tinctionsintestmethodologyareintroduced.Theseareasfreeflow(playthegame asyounormallywould)vs.narrowspecific(playonlyapartofthegameandplay itrepeatedly)vs.broadspecific(userplaysmoreofthegamebutplaysrepeatedly). Whilethesethreetypesoftestsareusedwidely,namingthemisastepforward.In thesecondcasestudy,thedevelopmentofZoëModeillustratessomebasictruisms oftesting,suchastheneedformultipleiterativetestsnotonlytorefinebutalsoto developacompletelyalternativeandmoreeffectiveapproach–inthiscase,chang- ingfromtextdescriptionstovisualimagestoillustratesuccessfulcalibration.The third and final case study examines Relentless Software’s approach to evaluating Buzz!QuizTV.Focusgrouptestingwasusedtocollectuserratingsofthegame.In addition, telemetry from the game was captured remotely postlaunch (a technique alsousedbyBlackRockStudio).Theauthorsseefuturedevelopmentsascapturing moredatasuchasfacialexpressionsandphysiologicaldata. InChapter8,UserExperienceDesignforInexperiencedGamers:GAP–Game ApproachabilityPrinciples,DesurvireandWibergprovideaninsightfulanalysisand usefulguidancetodesignstudiosthataredevelopinggamesforbroadaudiences.A classic challenge that these studios face is creating a game that is “approachable”. Thatis,thegamemustbeaccessibletonovicesandholdtheirinterestsothatthey willpurchase itand continue to play it.Traditionally, game studios emphasize the “playability”oftheirgames.Playabilityisusuallytakentomeanthegameisgood for experienced players. While traditional methods like the members of the studio playingthegameortestingforbalancebyaqualityassuranceteammightproduce a “playable” game (good game balance, few bugs, elimination of “golden” paths, etc.), “approachability” is often a bigger challenge since it addresses the needs of newornoviceplayers.Desurvireetal.addressthisproblemusingtwo“traditional” usabilitymethods:usabilitytesting(UT)andheuristicevaluation(HE).Theyenrich theHEwithGameApproachabilityPrinciples(GAP).Heuristicevaluationisonly asgoodastheprinciplesthatdriveitandtheexperienceoftheevaluators.Intheir chapter, Desurvire et al. describe a study that tested several games using both HE and UT. They found that using the methods together produced actionable insights either method alone would not have uncovered. Their approach is unique, since mostoftheresearchliteraturehasattemptedtoevaluatewhichmethodisbest.This chapter shows that the combination of UT and HE produces more “value” for the designstudio.Valueisdefinedasthemostactionabledatafortheinvestmentmade. Anystudiohopingtoproduceanapproachablegamewouldbewisetoincorporate bothHEandUTintotheirtoolkit. In Chapter 9, Digital Games, the Aftermath, Poels, IJsselsteijn, de Kort, and VanIerselexplorearelativelyunchartedareaofgaming–theshort-andlong-term effects of the game on the player. While most past studies have focused on single possible effects of gaming, e.g., gaming leads to isolation or gaming desensitizes one fromviolence, Poels looks atmoreglobaleffects.The resultsshowboth tem- poraryandlong-lastingeffectsthatarenotdifferentfromexposuretoothermedia suchasmoviesorbooks.Theseresultsarenotunexpected,buttheirimplicationsare consistentwithasignificantshiftintheconceptualizationofpersonality.Thatis,the rejectionofafixed,trait-likeconceptionofpersonalityinfavorofamoremalleable
The list of books you might like
