Building Robust Real-Time Game AI

265 Pages·2016·5.08 MB·English

Building Robust Real-Time Game AI:
Simplifying & Automating Integral
Process Steps in Multi-Platform Design
submitted by
Swen Gaudl
for the degree of Doctor of Philosophy
of the
University of Bath
Department of Computer Sciences
May 2016
COPYRIGHT
Attention is drawn to the fact that copyright of this thesis rests with its author. This
copy of the thesis has been supplied on the condition that anyone who consults it is
understood to recognise that its copyright rests with its author and that no quotation
from the thesis and no information derived from it may be published without the prior
written consent of the author.
This thesis may be made available for consultation within the University Library and
may be photocopied or lent to other libraries for the purposes of consultation.
Signature of Author.................................................................
Swen Gaudl
Acknowledgements
After nearly four years of work and continuous learning, reading, programming and
experimenting and iterating this cycle, the thesis is finally complete. Driven by my
desire to understand and advance the development of agents for highly demanding
domainssuchasdigitalgames,Iexploredmanyscientificfields. Iventuredintorobotics,
cognitive science and biology to understand modelling approaches for sophisticated
agents. Now, this journey comes to an end, and the cumulated knowledge I gained is
collected within this thesis.
During those four years, I could concentrate full-time on my research due to the
support I received from the University of Bath and the Department of Computer Sci-
ence. Due to my University scholarship, I could focus and dive deeper into research
than I initially had hoped. I enjoyed teaching and the interaction with graduate and
undergraduate students. I was able to visit the University of California, Santa Cruz,
because of the University travel grant which I am thankful for. I would also like to
thank my supervisor Joanna J. Bryson for giving me the opportunity to undertake my
studies in the first place and also for initiating my research visit.
I would like to thank Michael Mateas and Noah Wardrip-Fruin for integrating me
into their research group for that short period of time and for spending the time to dis-
cussmyresearch. IwanttothankAprilGrow,JosephC.Osbornfortheircollaboration
and time. Additionally, I want to thank James Ryan and his wife Nina for taking care
of me in Santa Cruz; James on top of that for our research discussions and his feedback
on my research. I wish I could have stayed longer. They and all other researchers I
met at UCSC allowed me to expand my understanding on games and games research
a lot.
1
I would like to thank Leon Watts, a lot, for our continuous conversations on research
over coffee and tea and for listening and discussing my research ideas in great detail.
I want to thank Rob Wortham and Andreas Theodorou for their feedback and our
conversations on action selection and the philosophy of computation and complexity.
A special thanks to Paul Rauwolf and Dominic Mitchell for our long conversations
and our discussions on research rigour and how to survive a PhD.
I am also grateful for the tremendous amount of work Eliza Shaw and Michael
Wright undertook, helping me to enhance the understanding and readability of my
thesis. I do feel that my writing skills got better because of you and your feedback.
I am forever indebted to my partner Denise Lengyel, especially for the last eight
months when I spent most of my time writing, eating and brooding over this work
ignoring most other things. Without our conversations and your support and patience,
this would not have been possible. I would like to thank my family for not giving up
on me when I was too busy to reply or answer calls and for their moral support and
encouragement.
Lastly, I want to thank the numerous people I did not manage to name and those I
met during my PhD at conferences and workshops which gave feedback to my research
and thus took part in shaping it.
2
Summary
Digital games are part of our culture and have gained significant attention over the
last decade. The growing capabilities of home computers, gaming consoles and mobile
phones allow current games to visualise 3D virtual worlds, photo-realistic characters
andtheinclusionofcomplexphysicalsimulations. Thegrowingcomputationalpowerof
thosedevicesenablestheusageofcomplexalgorithmswhilevisualisingdata. Therefore,
opportunities arise for developers of interactive products such as digital games which
introducenew, challengingandexcitingelementstothenextgenerationofhighlyinter-
activesoftwaresystems. Twoofthosechallenges, whichcurrentsystemsdonotaddress
adequately, aredesignsupportforcreatingInteractiveVirtualAgents(IVAs)andmore
believable non-player characters for immersive game-play. We start in this thesis by
addressing the agent design support first and then extend the research, addressing the
second challenge. The main contributions of this thesis are:
• The posh-sharp sytem is a framework for the development of game agents. The
platformismodular,extendable,offersmulti-platformsupportandadvancedsoft-
waredevelopmentfeaturessuchasbehaviour inspection andbehaviour versioning.
The framework additionally integrates an advanced information exchange mech-
anism supporting loose behaviour coupling.
• The Agile behaviour design methodology integrates agile software development
and agent design. To guide users, the approach presents a work-flow for agent
design and guiding heuristics for their development.
• The action selection augmentation ERGo introduces a “white-box” solution to
altering existing agent frameworks, making their agents less deterministic. It
augments selected behaviours with a bio-mimetic memory to track and adjust
their activation over time.
Withthenewapproachtoagentdesign,thedevelopmentofdeeperagentbehaviour
for digital adversaries and advanced tools supporting their design is given. Such mech-
anisms should enable developers to build robust non-player characters that act more
human-like in an efficient and robust manner. Within this thesis, different strategies
are identified to support the design of agents in a more robust manner and to guide
developers. These discussed mechanisms are then evolved to develop and design IVAs.
Because humans are still the best measurement for human-likeness, the evolutionary
cycle involves feedback given by human players.
3
Related Publications
• Grow, A., Gaudl, S. E., Gomes, P. F., Mateas, M., and Wardrip-Fruin, N. (2014).
A methodology for requirements analysis of ai architecture authoring tools. In
Foundations of Digital Games 2014. Society for the Advancement of the Science
of Digital Games
• Gaudl, S. E., Davies, S., and Bryson, J. J. (2013). Behaviour oriented design for
real-time-strategy games – an approach on iterative development for starcraft ai.
In Proceedings of the Foundations of Digital Games, pages 198–205. Society for
the Advancement of Science of Digital Games
• Gaudl, S. E. and Bryson, J. J. (2014). Extended ramp goal module: Low-
cost behaviour arbitration for real-time controllers based on biological models
of dopamine cells. In Computational Intelligence and Games (CIG), 2014 IEEE
Conference on, pages 1–8. IEEE
• Gaudl, S. E., Osborn, J. C., and Bryson, J. J. (2015). Learning from play: Fa-
cilitating character design through genetic programming and human mimicry. In
Progress in Artificial Intelligence, pages 292–297. Springer International Publish-
ing
4
Contents
1 Introduction 9
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 How to Build Games and Agents Therein? . . . . . . . . . . . . . . . . . 11
1.2.1 Game Development . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.2 Developing Game Agents . . . . . . . . . . . . . . . . . . . . . . 15
1.3 Identifying Critical Points in Game Development . . . . . . . . . . . . . 18
1.4 Motivation for Providing a New Approach to Agent Design . . . . . . . 21
1.5 A New Integrated Process of Agent Design . . . . . . . . . . . . . . . . 23
1.5.1 Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.5.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.6 Overview of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2 Background 29
2.1 Fundamental Game AI Techniques . . . . . . . . . . . . . . . . . . . . . 31
2.1.1 Decision Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.2 Spatially Centred Approaches . . . . . . . . . . . . . . . . . . . . 50
2.1.3 Evolutionary & Learning Approaches . . . . . . . . . . . . . . . 62
2.1.4 Summary of Approaches to Game AI . . . . . . . . . . . . . . . . 72
2.2 Agents and Agent Design . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.2.1 Agent Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.2.2 Behaviour-Based Artificial Intelligence . . . . . . . . . . . . . . . 76
2.2.3 Goal-Driven Autonomy (GDA) . . . . . . . . . . . . . . . . . . . 78
2.2.4 Goal Oriented Planning (GOAP) . . . . . . . . . . . . . . . . . . 81
2.2.5 Heavy Cognitive Architectures . . . . . . . . . . . . . . . . . . . 84
2.2.6 ICARUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
2.2.7 MIT cX agent architecture . . . . . . . . . . . . . . . . . . . . . 90
2.2.8 A Behavior Language (ABL) . . . . . . . . . . . . . . . . . . . . 95
2.2.9 Behavior-Oriented-Design (BOD) . . . . . . . . . . . . . . . . . . 98
5
2.2.10 Generative Agent Design . . . . . . . . . . . . . . . . . . . . . . 104
2.2.11 Summarising Agent Design Approaches . . . . . . . . . . . . . . 105
2.3 Game AI Design Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
2.3.1 Pogamut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
2.3.2 ABODE and POSH . . . . . . . . . . . . . . . . . . . . . . . . . 110
2.3.3 Visual BT Editors . . . . . . . . . . . . . . . . . . . . . . . . . . 111
2.4 Summarising the State of the Art . . . . . . . . . . . . . . . . . . . . . . 115
3 Requirements for Agent Tools 120
3.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.3 The System-Specific Step . . . . . . . . . . . . . . . . . . . . . . . . . . 124
3.4 Interview Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.5 The Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.6 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.6.1 Case Study 1: BOD using POSH . . . . . . . . . . . . . . . . . . 126
3.6.2 Case Study 2: FAtiMA . . . . . . . . . . . . . . . . . . . . . . . 130
3.6.3 Case Study 3: ABL . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.7 Authoring Support Strategies . . . . . . . . . . . . . . . . . . . . . . . . 136
3.8 Summarising the System-Specific Step . . . . . . . . . . . . . . . . . . . 138
4 Integrating Human Knowledge into Game AI 140
4.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.3 StarCraft AI Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.5 Case Study: BOD applied to Real-Time Strategy Games . . . . . . . . . 147
4.5.1 StarCraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.5.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.5.3 Iterative Development . . . . . . . . . . . . . . . . . . . . . . . . 152
4.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.5.5 Summarising the Results . . . . . . . . . . . . . . . . . . . . . . 157
4.6 Advanced Planning for StarCraft . . . . . . . . . . . . . . . . . . . . . . 158
4.6.1 Encoding User Knowledge . . . . . . . . . . . . . . . . . . . . . . 159
4.6.2 Extending beyond individual strategies . . . . . . . . . . . . . . 160
4.7 Concluding Real-Time Strategy AI Contributions . . . . . . . . . . . . . 164
6
5 Advancing Tool Supported Action Selection 165
5.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.2 Agile Behaviour Design for Games . . . . . . . . . . . . . . . . . . . . . 166
5.2.1 Handling Complexity . . . . . . . . . . . . . . . . . . . . . . . . 171
5.3 POSH-SHARP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
5.3.1 POSH-SHARP Modules . . . . . . . . . . . . . . . . . . . . . . . 173
5.3.2 Behaviour Inspection & Primitive Versioning . . . . . . . . . . . 174
5.3.3 Memory & Encapsulation . . . . . . . . . . . . . . . . . . . . . . 176
5.3.4 Monitoring Execution . . . . . . . . . . . . . . . . . . . . . . . . 178
5.4 Concluding Advanced Authoring Support . . . . . . . . . . . . . . . . . 180
6 Augmenting Action Selection Mechanisms 182
6.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.3 The Extended Ramp Goal Model (ERGo) . . . . . . . . . . . . . . . . . 187
6.3.1 Approach: Biomimetic Models . . . . . . . . . . . . . . . . . . . 187
6.3.2 Basic activation mechanism . . . . . . . . . . . . . . . . . . . . . 188
6.3.3 Duration of activation . . . . . . . . . . . . . . . . . . . . . . . . 191
6.3.4 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6.3.5 Summary of Augmenting Behaviour Arbitration . . . . . . . . . 194
6.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
6.6 Concluding An Augmentation for Behaviour Arbitration . . . . . . . . . 203
7 Evolutionary Mechanisms for Agent Design Support 205
7.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
7.3 Background & Related Work . . . . . . . . . . . . . . . . . . . . . . . . 206
7.3.1 Agent Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.3.2 Generative Approaches . . . . . . . . . . . . . . . . . . . . . . . 208
7.3.3 Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . 209
7.4 Setting and Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.5 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.6 Results & Future Work using GP . . . . . . . . . . . . . . . . . . . . . . 214
7.7 Evolutionary Mechanisms Summary . . . . . . . . . . . . . . . . . . . . 216
8 Discussion & Future Work 217
8.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
7
9 Conclusion 226
A Behaviour-Oriented Design 229
B StarCraft 232
C Requirements 234
C.1 A Behaviour Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
D Augmenting Action Selection Mechanisms 237
Glossary 240
Acronyms 245
8
Chapter 1
Introduction
In this thesis, we seek to understand and advance the current state of the art in ar-
tificial intelligence (AI) design and development approaches for controlling embodied
agents. To achieve this goal, a comprehensive literature survey was conducted which
is backed up by expert interviews. Based on the findings, a new architecture and
methodology for agent design are proposed—Agile Behaviour Design, discussed in
Chapter 5. To support the design methodology, a new agent framework is proposed to
allow for experimentation and testing of the new methodology. This new agent frame-
work is posh-sharp which integrates novel mechanisms for inspecting and versioning
of behaviours and supported by a new arbitration mechanism—ERGo, discussed in
Chapter 6—which responds significantly better in noisy environments than traditional
approaches.
1.1 Motivation
To pursue the goal of a more robust and supportive agent design environment, while
not arriving at a too abstract answer to render it unusable, a special focus is put on
behaviour-based AI (BBAI) [Maes, 1993; Brooks, 1986] for development and design
of character artificial intelligence. As embodied agents interact in an environment,
digital games are chosen as an interesting and challenging spatial environment for the
character to interact in.
Existing character design tools, methodologies and architectures were analysed to
understand their design process and differences in approaching agent design. Based
on this analysis, the most fitting methodology was selected and enhancements were
proposed to it. Additionally, an entirely new combination of tools is presented which
results in a usable toolbox for authors of digital games in combination with the given
9