anytime deliberation for computer game agents

217 Pages·2009·0.95 MB·English

ANYTIME DELIBERATION FOR COMPUTER GAME
AGENTS
by
NICHOLAS ANDREW HAWES
A thesis submitted to
The University of Birmingham
for the degree of
Doctor of Philosophy
School ofComputerScience
TheUniversityofBirmingham
Birmingham,B15 2TT
November2003

Abstract
This thesis presents an approach to generating intelligent behaviour for agents in computer game-like
worlds. Designing and implementing such agents is a difficult task because they are required to act in
real-time and respond immediately to unpredictable changes in their environment. Such requirements
havetraditionally causedproblemsforAItechniques.
To enable agents to generate intelligent behaviour in real-time, complex worlds, research has been
carried out into two areas of agent construction. The first of these areas is the method used by the
agent toplan future behaviour. Toallow anagent tomakeefficient useofits processing time, aplanner
is presented that behaves as an anytime algorithm. This anytime planner is a hierarchical task network
plannerwhichallowsaplanningagenttointerruptitsplanningprocessatanytimeandtrade-offplanning
timeagainstplanquality.
The second area of agent construction that has been researched is the design of agent architectures.
This has resulted in an agent architecture with the functionality to support an anytime planner in a dy-
namic, complex world. A proof-of-concept implementation of this design is presented which plays
UnrealTournament anddisplays behaviour thatvariesintelligently asitisplacedunderpressure.

Acknowledgements
First I must express my sincere gratitude to mysupervisor, Aaron Sloman. Without his encouragement
and knowledge, and the intellectual freedom he allowed me, I would not have been able to tackle this
research.
Perhapsthegreatestinfluenceovertheday-to-day directionofmyresearchwasprovidedbymyfirst
external supervisor, Ian Wright. Although I lost his official supervision just as I was starting to make
progress, he continued to motivate and guide me with an equal mix of inspiration and fear. For his
unfaltering inputandfriendship Iamtrulygrateful.
NoneofthiswouldhavebeenpossiblewithoutthesupportandfundingofSonyComputerEntertain-
mentEurope, and the workof DaveRanyard. Also, manythanks mustgo toManfred Kerberand Mark
Ryanfortheirinsightful commentsasmembersofmythesisgroup.
Iwouldalsoliketothankmyfamilyandfriendswhohavesupported mefromtheverybeginning of
my education until now, especially those that were there for me during the writing-up period. The list
is too long for everyone to be named individually, but if I’ve spoken to you in the last four years, then
you’vehelpedmethroughthis.
I don’t think I would have even managed to survive the first year of my PhD without the always
welcome distractions provided by my like-minded office mates. For the rowdy nights out and racquet
sports,thankstoyouall.
Iwouldliketotakethisopportunity tothank myA-LevelPhysicsteachers, Mr(John) Atkinson and
Mr(Andy)Stoddart,whomademeworkwhenIreallyneededpushing.
Finally, I offer all of my love and thanks to Rosie for moving to Birmingham, putting up with me,
andsupporting methrough everything.

Contents
I Preamble 1
1 Introduction 3
1.1 WhatisaComputerGame? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 AnIntroduction toAIResearchforGames . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 GeneralacademicAIresearch . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.2 Commercialgamedevelopment . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 Agent-specific AIresearchforgames . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 AnExampleGameScenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 SummaryofMainContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 OverviewofThesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 DetailedProblemStatement 12
2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.1 Simulationvs. Entertainment . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 ExpandedScenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 PlanningProblemsInDetail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 Problemsfromthescenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.2 Problemsingeneral. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.3 Planningindynamicworlds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 AgentProblemsInDetail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.1 Problemsfromthescenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.2 Problemsingeneral. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.5 TheArchitectural Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 DetailedProblemStatement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
II Planning ForComputer GameWorlds 22
3 ReviewofPlanningLiterature 23
3.1 ABriefHistoryofPlanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 PlanningApproaches SuitedtoGameWorlds . . . . . . . . . . . . . . . . . . . . . . . 27
ii
3.2.1 Fastplanners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Reactiveapproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.3 Continualplanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2.4 TheAutodriveproject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4 IntroductiontoAnytimePlanning 37
4.1 WhatisanAnytimeAlgorithm? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 WhyuseanAnytimePlanningAlgorithm? . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.1 Agentbenefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.2 Developerbenefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3 PossibleProblemswithAnytimePlanning . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.4 AReviewofAnytimeAlgorithm Literature . . . . . . . . . . . . . . . . . . . . . . . . 42
4.4.1 Anytimealgorithm frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4.2 Anytimeplanners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.5 GeneralObservations onAnytimePlanning . . . . . . . . . . . . . . . . . . . . . . . . 50
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5 TheA-UMCPAnytimePlanner 53
5.1 TheInterruptibility ofPlanners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 AQualityMeasureforAbstractPlans . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.3 AlgorithmsfortheHeuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.4 TheA-UMCPSearchFramework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6 AnalysisofA-UMCP 67
6.1 Applicability oftheHeuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.2 GeneralBehaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.3 PerformanceAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.4 EffectivenessoftheAlgorithmExtensions . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.5 TheHeuristicasaCritic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.6 PerformanceAffectingDomainFeatures . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.7 ComparisontoExistingAnytimePlanners . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
III A HybridAgent ForComputer GameWorlds 81
7 ReviewofAgentLiterature 83
7.1 ReactiveAgents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.2 HybridAgents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
iii
7.2.1 TheCogAffarchitecture schema . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.2.2 TouringMachines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.2.3 TheGuardianagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.2.4 TheCypressagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.2.5 Anarchitecture fromtheOzproject . . . . . . . . . . . . . . . . . . . . . . . . 93
7.2.6 TheSOMASSsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
7.3 PastComputerGameAgents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.3.1 TheSoarbotproject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.3.2 TheExcaliburproject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8 DevelopmentofaComputerGameAgent 98
8.1 Iteration One: TheBasicAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.1.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
8.1.2 Exampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
8.1.3 Analysisofexampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . 111
8.2 Iteration Two: ExecutionInterrupts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
8.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
8.2.2 Exampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
8.2.3 Analysisofexampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . 117
8.3 Iteration Three: SimplePlannerInterrupts . . . . . . . . . . . . . . . . . . . . . . . . . 118
8.3.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
8.3.2 Exampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
8.3.3 Analysisofexampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . 121
8.4 Iteration Four: AnytimePlanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
8.4.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8.4.2 Exampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
8.4.3 Analysisofexampleimplementation . . . . . . . . . . . . . . . . . . . . . . . . 130
8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
9 FurtherExperimentsandAnalysis 132
9.1 Information Processing View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
9.2 PlanInterpretation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
9.3 PerformanceProfileResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.4 AgentBehaviour Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
9.4.1 Study1: Scoringunderpressure . . . . . . . . . . . . . . . . . . . . . . . . . . 139
9.4.2 Study2: Intercepting theenemy . . . . . . . . . . . . . . . . . . . . . . . . . . 142
9.4.3 Study3: Fullgame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
9.5 TheIterativeApproachtoDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
iv
9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
IV Conclusion 147
10 Contributions,ConclusionsandFutureWork 149
10.1 Contributions toKnowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.1.1 Principalcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.1.2 Secondarycontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
10.2 CriticalEvaluation ofthePlannerandArchitecture . . . . . . . . . . . . . . . . . . . . 151
10.3 ComparisonsToExistingWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
10.4 WiderApplications OfTheResearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
10.5 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Appendices 161
A TheCaptureTheFlagPlanningDomain 163
B TheBlocksWorldPlanningDomainUMCPNotation 172
C AnExampleoftheHSPHeuristic 176
D AnExampleA-UMCPPlanningProcess 180
E ProblemDefinitions 184
E.1 BlocksWorldProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
E.2 CaptureTheFlagProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
E.3 UMTranslogProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Bibliography 193