Tilburg University Rapid adaptation of video game AI Bakkes, Sander

216 Pages·2017·4.71 MB·English

Tilburg University
Rapid adaptation of video game AI
Bakkes, S.C.J.
Publication date:
2010
Document Version
Publisher's PDF, also known as Version of record
Link to publication in Tilburg University Research Portal
Citation for published version (APA):
Bakkes, S. C. J. (2010). Rapid adaptation of video game AI. [s.n.].
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
Download date: 14. mrt.. 2023
Rapid Adaptation of Video Game AI

Rapid Adaptation of Video Game AI
PROEFSCHRIFT
terverkrijgingvandegraadvandoctor
aandeUniversiteitvanTilburg,
opgezagvanderectormagnificus,
prof.dr.Ph.Eijlander,
inhetopenbaarteverdedigentenoverstaanvaneen
doorhetcollegevoorpromotiesaangewezencommissie
indeaulavandeUniversiteit
opwoensdagmaartom:uur
door
SanderCornelusJohannesBakkes
geborenopjanuariteSwalmen
Promotor: Prof.dr.H.J.vandenHerik
Copromotor: Dr.ir.P.H.M.Spronck
Ledenvandebeoordelingscommissie:
Prof.dr.A.P.J.vandenBosch
Prof.dr.J-J.Ch.Meyer
Prof.dr.ir.G.vanOortmerssen
Prof.dr.E.O.Postma
Dr.D.W.Aha
e research reported in this thesis has been funded by NWO, the Netherlands Or-
ganization for Scientific Research, in the framework of the Rapid Online Learning for
EntertainmentComputing(ROLEC)project,withgrantnumber...
e research reported in this thesis is part of the Interactive Collaborative Information
Systems(ICIS)project,supportedbytheDutchMinistryofEconomicAffairs,withgrant
numberBSIK.
SIKSDissertationSeriesNo.-
e research reported in this thesis has been carried out under the auspices of SIKS, the
DutchResearchSchoolforInformationandKnowledgeSystems.
TiCCPh.D.SeriesNo.
ISBN----
©SanderBakkes
CoverdesignbyJobPereboom,StudioKonstruktief
All rights reserved. No part of this publication may be reproduced, stored in a retrie-
val system, or transmitted, in any form or by any means, electronically, mechanically,
photocopying,recordingorotherwise,withoutpriorpermissionoftheauthor.
Preface
Intheearlys,thevideo-gameindustryhasgrowntosurpasstheHollywoodmoviein-
dustryinrevenues.Inrecentyears,videogaminghasbecomeamass-marketentertainment
industryonaparwithTV,movies,andmusic.Modernvideogameshavebecomeincrea-
singlyrealisticintheirvisualandauditorypresentation.However,theartificialintelligence
(AI)ofvideogameshasnotreachedahighdegreeofrealismyet.Amajordisadvantageof
currentgameAIisitsinabilitytoadaptadequatelytogamecircumstances.edisadvan-
tagecanberesolvedbyendowinggameAIwithadaptivebehaviour,i.e.,theabilitytoadapt
togamecircumstances,generallybymeansoflearningadequategamebehaviourwhilethe
gameisinprogress.
CurrentapproachestoadaptivegameAItypicallyrequirenumeroustrialstolearnef-
fectivebehaviourinonlinegameplay(i.e.,gameadaptationisnotrapid).Inaddition,game
developersareconcernedthatapplyingadaptivegameAImayresultinuncontrollableand
unpredictablebehaviour(i.e.,gameadaptationisnotreliable).esecharacteristicsham-
pertheincorporationofadaptivegameAIinactual,commerciallyreleasedvideogames.e
generalgoalofourresearchistoinvestigatetowhatextentitispossibletoestablishgame
AIcapableofadaptingrapidlyandreliablytogamecircumstances.Ourfindings,thatare
discussedinthisthesis,maybeusedbygamedevelopersforincorporationinactualvideo
games.
isthesiswouldnothavebeenpossiblewithouttheassistanceandencouragementofa
largegroupofpeople.Iwasmostfortunatethatmysupervisor,JaapvandenHerik,allowed
me to pursue my goals and I owe many thanks to his support and commitment. I highly
appreciatehiscommentsandadviseonmyscientificwritings,aswellashiseffortsinarran-
gingadditionalfundingforfinalisingtheresearch.Aspecialgratitudegoesouttomydaily
advisor,PieterSpronck,forsupportingmesoverydedicatedly,forhisresearchinspiration,
andforthecharmhebringstoacademia.
Moreover,IwouldliketothankmycolleaguesatTilburgUniversityandMaastrichtUni-
versity.esupportivestaffofbothuniversitiesdeservemyappreciationfortheirhelpin
manyrespects.IwouldespeciallyliketomentionPeterGeurtzfortechnicalassistancein
thepast,andJokeHellemons,LeenJacobs,andDennisTantradjajafortheirongoingefforts
in different areas of support. e research atmosphere that I experienced within the uni-
versitywallsisonethatIhappilylookbackupon.Iconsidermyselfluckytohaveworkedin
aninternationalenvironment,andsecretlyenjoyedthedailydosageofslightlyunorthodox
behaviourandthoughtsthatitbroughtabout.Forthat,Iacknowledgegratefullyalldirect
colleagues,androommatesovertheyears.IwouldespeciallyliketothankStevendeJong,
GuillaumeChaslot,AlessandroRossi,SanderSpek,AndreasRüdel,SophiaKatrenko,and
AnneBoldersfortheirongoingassistance,hospitality,andfriendship.
During my research, I had the pleasure to supervise and cooperate with several stu-
dents. I explicitly thank Laurens van der Blom, Philip Kerbusch, Bart Mehlkop, and Fre-
derik Schadd, whose research efforts and their results have been integrated in the thesis.
Inaddition,IthankTomNowellandAlexanderSeizingerfortheirgenerousassistancein
developingAIfortheopen-sourceSgameenvironment.
ii Preface
Outwiththeacademia,Ifoundthesupportofindividualsandorganisationsforexpres-
singmyselfintheformofphotographyandDJ’ing.HereIemphasisethecollectiveofphoto-
graphers‘Nxtlvl’,andtheDJ’sof‘Loki’.Iacknowledgeinparticularthepersonsbehindsocial
gatheringsinvarious(improvised)settings,andinthe‘Ipanema’,‘Zondag’,and‘Landbouw-
belang’,forbringingtogetherthemanypeoplethatIwillmisswhenobligationswillchange
mylifeafterthethesisdefence.
Inconclusiontotheseacknowledgements,Iwouldliketothankfriendsfrommyhome-
townSwalmenandmybrotherMilan,withwhomitwasapleasuretogrowup.Ichbedank
mienleefomaveur’tmichmitgaevevanhäörwieslaeveslessedieouchopvandaagnaggeljig
zin;enmieneljers,veurhäörónveurwaardelikkesjteunenvertroewe.Finalmente,gostaria
deexpressaraminhafelicidadeporreceberoamordaCândida.
SanderBakkes
March,
Contents
Preface i
Contents iii
 Introduction 1
. Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
. GameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
.. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
.. Gamedevelopers’approachtogameAI . . . . . . . . . . . . . . . . 4
.. AcademicapproachtogameAI . . . . . . . . . . . . . . . . . . . . . 5
. Problemstatementandresearchquestions . . . . . . . . . . . . . . . . . . . 6
. Researchmethodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
. esisstructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
 Background 11
. Videogames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
.. History. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
.. Gamegenres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
.. Gamestudies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
.. Stateoftheindustry . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
. ResearchintogameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
.. EntertainmentandgameAI . . . . . . . . . . . . . . . . . . . . . . . 16
.. Non-adaptivegameAI . . . . . . . . . . . . . . . . . . . . . . . . . . 17
.. AdaptivegameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
. Chaptersummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
 IncrementaladaptivegameAI 21
. WhatisincrementaladaptivegameAI? . . . . . . . . . . . . . . . . . . . . . 22
. IncrementaladaptivegameAIwithTEAM . . . . . . . . . . . . . . . . . . . 23
.. TeamAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
.. eTEAMadaptationmechanism . . . . . . . . . . . . . . . . . . . 24
.. TEAMvs.QuakeIIICTFteamAI . . . . . . . . . . . . . . . . . . . 27
.. DiscussionofTEAM . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
. IncrementaladaptivegameAIwithTEAM . . . . . . . . . . . . . . . . . . 32
.. eTEAMadaptationmechanism . . . . . . . . . . . . . . . . . . 32
.. TEAMvs.QUAKEIIICTFteamAI . . . . . . . . . . . . . . . . . . 33
.. DiscussionofTEAM . . . . . . . . . . . . . . . . . . . . . . . . . . 36
. Practicalapplicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
iv Contents
 Case-basedadaptivegameAI 41
. Whatiscase-basedadaptivegameAI? . . . . . . . . . . . . . . . . . . . . . 41
.. Case-basedreasoning . . . . . . . . . . . . . . . . . . . . . . . . . . 42
.. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
.. Contributionsandlimitationsoftheapproach . . . . . . . . . . . . . 44
. Case-basedadaptivegameAIformapadaptation . . . . . . . . . . . . . . . 45
.. Approachtomapadaptation . . . . . . . . . . . . . . . . . . . . . . 45
.. Experimentwithmapadaptation . . . . . . . . . . . . . . . . . . . . 47
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 55
. Case-basedadaptivegameAIinasimulatedgame . . . . . . . . . . . . . . . 55
.. Simulatedvideogame . . . . . . . . . . . . . . . . . . . . . . . . . . 56
.. Gatheringdomainknowledge . . . . . . . . . . . . . . . . . . . . . . 57
.. Exploitingdomainknowledge . . . . . . . . . . . . . . . . . . . . . . 59
.. Experimentswithexploitingdomainknowledge . . . . . . . . . . . 62
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 63
. Towardscase-basedadaptationinvideogames. . . . . . . . . . . . . . . . . 64
. Chaptersummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
 eevaluationfunction 67
. Evaluationfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
.. Evaluationfunctionsinclassicgames . . . . . . . . . . . . . . . . . . 68
.. Evaluationfunctionsinvideogames . . . . . . . . . . . . . . . . . . 69
.. Establishinganevaluationfunction . . . . . . . . . . . . . . . . . . . 71
.. Exploitinggameknowledge . . . . . . . . . . . . . . . . . . . . . . . 73
. AnevaluationfunctionforSPRING . . . . . . . . . . . . . . . . . . . . . . . 73
.. egameenvironment . . . . . . . . . . . . . . . . . . . . . . . . . . 73
.. Towardsacasebaseofgameknowledge . . . . . . . . . . . . . . . . 76
.. Ourevaluationfunction . . . . . . . . . . . . . . . . . . . . . . . . . 77
.. Phaseofthegame. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
.. Materialstrength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
.. Commandersafety . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
. Validatingtheevaluationfunction . . . . . . . . . . . . . . . . . . . . . . . . 79
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
.. Twomeasuresforperformanceassessment . . . . . . . . . . . . . . 82
.. Measuredperformance . . . . . . . . . . . . . . . . . . . . . . . . . 82
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 86
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
 eadaptationmechanism 89
. Adaptationmechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
.. Rapidadaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
.. Reliableadaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
. AnadaptationmechanismforSPRING . . . . . . . . . . . . . . . . . . . . . 93
.. Generaladaptationprocedure . . . . . . . . . . . . . . . . . . . . . . 93
Contents v
.. Gameindexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
.. Clusteringofobservations . . . . . . . . . . . . . . . . . . . . . . . . 94
.. InitialisationofgameAI . . . . . . . . . . . . . . . . . . . . . . . . . 95
.. Similaritymatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
.. Onlinestrategyselection . . . . . . . . . . . . . . . . . . . . . . . . . 96
. Validatingtheadaptationmechanism . . . . . . . . . . . . . . . . . . . . . . 97
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
.. Performanceassessment . . . . . . . . . . . . . . . . . . . . . . . . . 101
.. Exp:Resultsofgameadaptation . . . . . . . . . . . . . . . . . . . . 101
.. Exp:Resultsofdifficultyscaling . . . . . . . . . . . . . . . . . . . . 104
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 105
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
 Opponentmodelling 109
. Opponentmodelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
.. Opponentmodellinginclassicgames. . . . . . . . . . . . . . . . . . 110
.. Opponentmodellinginvideogames . . . . . . . . . . . . . . . . . . 111
. EstablishingopponentmodelsinSPRING . . . . . . . . . . . . . . . . . . . 113
.. Approachtoestablishingopponentmodels . . . . . . . . . . . . . . 113
.. Implementationoftheapproach . . . . . . . . . . . . . . . . . . . . 114
.. Experimentswiththeimplementation . . . . . . . . . . . . . . . . . 116
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 122
. ExploitingopponentmodelsinSPRING . . . . . . . . . . . . . . . . . . . . 123
.. Approachtoexploitingopponentmodels . . . . . . . . . . . . . . . 123
.. Anexperimentwithexploitingopponentmodels . . . . . . . . . . . 125
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 127
. Chapterconclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
 Integratingthethreecomponents 129
. Howtointegratethethreecomponents . . . . . . . . . . . . . . . . . . . . . 129
. Incorporatingopponentmodelling . . . . . . . . . . . . . . . . . . . . . . . 130
.. Establishingopponentmodels . . . . . . . . . . . . . . . . . . . . . . 131
.. Exploitingopponentmodels . . . . . . . . . . . . . . . . . . . . . . . 132
. Experimentswithcase-basedadaptivegameAI . . . . . . . . . . . . . . . . 134
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
.. Performanceassessment . . . . . . . . . . . . . . . . . . . . . . . . . 134
.. Generatedopponentmodels. . . . . . . . . . . . . . . . . . . . . . . 135
.. Resultsofgameadaptation . . . . . . . . . . . . . . . . . . . . . . . 135
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 138
. Practicalapplicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
.. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
.. Dealingwithimperfectinformation . . . . . . . . . . . . . . . . . . 140
.. Generalisationtodifferentgames . . . . . . . . . . . . . . . . . . . . 141
.. Acceptancebygamedevelopers . . . . . . . . . . . . . . . . . . . . . 142