Computational Models of Motivation for Game-Playing Agents

217 Pages·2016·6.58 MB·English

by  Kathryn E. Merrick (auth.)

Kathryn E. Merrick
Computational
Models of
Motivation for
Game-Playing
Agents
Computational Models of Motivation
for Game-Playing Agents
Kathryn E. Merrick
Computational Models
of Motivation
for Game-Playing Agents
123
Kathryn E.Merrick
Schoolof Engineering andInformation
Technology
University of NewSouthWales
Australian Defence Force Academy
Canberra,ACT
Australia
ISBN978-3-319-33457-8 ISBN978-3-319-33459-2 (eBook)
DOI 10.1007/978-3-319-33459-2
LibraryofCongressControlNumber:2016947761
©SpringerInternationalPublishingAG2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAGSwitzerland
Preface
In recent years, two quests have been emerging to focus artificial intelligence
research for games. The first, and newest, is concerned with developing our
understanding of game players. The second is concerned with building more
believableandexciting virtualworldsinwhichtoplaygames.Inthisbookweuse
information gathered by game data mining researchers about players to inform the
design of novel self-motivated game-playing agents to control non-player charac-
ters. We demonstrate how self-motivated agents can increase the diversity of
non-player characters by permitting agents to exhibit unique decision-making
characteristics, which lead to interesting, emergent patterns when they interact.
More and more studies are emerging of the factors that motivate people to play
online games and the cultures that emerge among humans in virtual worlds.
However, the diversity we see in humans is not yet present in the
computer-controlled characters that support online virtual worlds. Techniques for
embedding human-like motives in game-playing agents have not yet been widely
explored.Asthecomplexityandfunctionalityofmultiuservirtualworldsincreases,
computer-controlled characters are becoming an increasingly challenging applica-
tion for artificial intelligence techniques. Players are demanding more believable
and intelligent non-player characters to enhance their gaming experience.
This book presents a new artificial intelligence technique—computational
motivation—and shows how it can be used to increase the diversity of non-player
charactersbyadaptingsometraditionalarchitecturesforcomputer-controlledgame
characters. Theoretical issues are addressed for representing and embedding com-
putational models of motivation in rule-based agents, crowds, learning agents and
evolutionary algorithms. Practical issues are addressed for defining games,
mini-games or in-game scenarios for virtual worlds in which computer-controlled,
motivated agents can participate alongside human players.
Large-scalevirtualworldsmayhavehundredsorthousandsofvirtualcharacters,
and the question arises as to how these characters can be embedded with diverse,
individual personalities. Computational motivation provides a novel answer to this
question. Our starting point is the ‘influential trio’: achievement, affiliation and
v
vi Preface
power motivation. Incentive-based theories of achievement, affiliation and power
motivation are the basis of competence-seeking behaviour, relationship-building,
leadership and resource-controlling behaviour in humans. We show how these
motives can be modelled and embedded in artificial agents.
Theaimofthisbookistoprovidegameprogrammers,andthosewithaninterest
inartificialintelligence,withtheknowledgerequiredtodevelopdiverse,believable
game-playing agents for virtual worlds. Computational motivation is an exciting,
emerging research topic in the field of artificial intelligence. The development of
motivated agents is at the cutting edge of artificial intelligence and cognitive
modellingresearch.Thisopensthewaybothfornewtypesofartificialagents,new
typesofcomputergamesandin-worldmini-gamesorscenarios.Thisbookprovides
anin-depthlookatnewcomputationalmodelsofmotivationandoffersinsightsinto
the strengths, limitations and future development of motivated agents for gaming
applications.
Part I—Game Playing in Virtual Worlds by Humans
and Agents
Chapter 1—From Player Types to Motivation
Chapter2—ComputationalModelsofAchievement,AffiliationandPower
Motivation
Chapter 3—Game-Playing Agents and Non-Player Characters
The first part of this book studies the relationship between game play and moti-
vation in humans and proposes ways in which this can be represented and
embeddedinartificialagents.Chapter 1studiesthemotivationalcharacteristics we
seeinhumansplayinggames,whichwemightexpecttoseeinadiversesocietyof
computer-controlled game-playing agents. After examining player types that have
been identified through subjective and objective studies of human game players,
Chap.1turnstocomplementaryliteraturefrommotivationpsychologyandreviews
the theories that may contribute to these characteristics in humans. It specifically
focusesonthree incentive-based theories ofmotivation for achievement, affiliation
and power motivation.
Chapter 2 introduces computational models of motivation to embed these
human-inspired motives in artificial agents at different levels offidelity. A flexible
mathematical model is introduced that permits achievement, affiliation and power
motives to be expressed in terms of approach and avoidance components, which
canbeadjustedtocreatedifferent motivationvariants.Alternativesareprovidedto
model individual motives, or profiles of several motives.
Chapter 3 describes how motivation can be embedded in four architectures for
game-playing agents that can be used to control non-player characters: rule-based
agents, crowds, learning agents and evolutionary algorithms. Motivated rule-based
agents are suitable for decision making by individual agents, while motivated
Preface vii
learningagents are suitable for competitive or strategicdecisionmakingwhen two
or more agents interact. The motivated crowd and evolutionary algorithms are
suitableforcontrollinggroupsofagents.Thesearchitecturesarethetopicsofstudy
in Part II, Part III and Part IV of the book.
Part II—Comparing Human and Artificial Motives
Chapter 4—Achievement Motivation
Chapter 5—Profiles of Achievement, Affiliation and Power Motivation
Part II of this book describes how the models introduced in Part I can be used to
createanumberofspecificmotivationsubtypesthathavepreviouslybeenobserved
in humans. These are then demonstrated by reproducing three canonical human
experiments with artificial agents. In Chap. 4, four subtypes of achievement
motivationaredescribed.Theseareembeddedinagentsplayingthering-tossgame.
We demonstrate the similarities that can be observed between motivated
game-playing agents and humans playing the same game.
In Chap. 5 three profiles of achievement, affiliation and power motivation are
introduced for agents playing two different games: roulette and the prisoners’
dilemmagame.Weagaindemonstratethesimilaritiesthatcanbeobservedbetween
motivatedgame-playingagentsandhumansplayingthesamegames.Weshowthat
wecancreateadiversegroupofagentsthatcompeteorcooperateindifferentways
when playing games.
Part III—Game Scenarios for Motivated Agents
Chapter 6—Enemies
Chapter 7—Pets and Partner Characters
Chapter 8—Support Characters
InPartIIIin-gamescenariosaredescribedthatareappropriatefordifferentkindsof
motivated agents, including motivated rule-based agents, crowds and learning
agents. These scenarios are presented inthree chapters,corresponding to scenarios
for three common non-player character types: enemies, partner characters and
support characters.
Chapter6presentsanapplicationofmotivatedrule-basedagentsasenemiesand
two scenarios in which motivated learning agents are used as enemies. Chapter 7
considers two further scenarios for motivated learning agents as pets and partner
characters.Chapter8explorestheuseofmotivatedcrowdsandmotivatedlearning
agents in support characters. We show that agents with different motives exhibit
different strategies for playing games. This results in behavioural diversity among
viii Preface
characters. In addition, when agents with different motives interact, different
outcomes emerge.
Includedinthispartaretheoreticalanalyses,empiricalinvestigationsandsample
game applications. We show how theoretical and empirical results can be used to
predict the behaviour of agents in practice. We use these motivated agents in two
ways: first as non-player characters and secondly to model the different types of
responses possible to non-player characters, either by individuals or groups of
human players. The applications in this part of the book demonstrate how each
abstract agent architecture from the previous part can be connected to the concrete
actions available to a game character.
Part IV—Evolution and the Future of Motivated Agents
Chapter 9—Evolution of Motivated Agents
Chapter 10—Conclusion and Future
Part IV looks to the future of motivated game-playing agents, first with a specific
focusontheconditionsunderwhichagentswithdifferentmotivesmightevolveina
virtual world and then more broadly. Chapter 9 considers the evolution of moti-
vationinasocietyofgame-playingagents.Agentsarestudiedinmultiplayersocial
dilemma games and in a custom implementation of a classic single-player shooter
game. This chapter demonstrates how the composition of a society of motivated
agentscanchangeovertimeinresponsetosubjectiveorobjectivedefinitionsofthe
fitness of an agent.
Chapter 10 summarises the algorithms, components and combinations of com-
ponents that have been studied in the book, and discusses how these components
might fit into other motivated agent architectures. The strengths and limitations of
motivated agents are considered and used as a basis for discussion of the future
directions for motivated agents and multiuser computer games. Advances in
computational models of motivation, motivated agent models and their application
to different types of games are considered.
Canberra, ACT, Australia Kathryn E. Merrick
Contents
Part I Game Playing in Virtual Worlds by Humans and Agents
1 From Player Types to Motivation .. .... .... .... .... ..... .... 3
1.1 Virtual Worlds and Online Games .. .... .... .... ..... .... 3
1.2 Explorer, Achiever, Socialiser, Aggressor: Human Player
Types ... .... .... ..... .... .... .... .... .... ..... .... 4
1.3 Incentive-Based Theories of Motivation.. .... .... ..... .... 9
1.3.1 Incentive .. ..... .... .... .... .... .... ..... .... 9
1.3.2 Achievement Motivation... .... .... .... ..... .... 10
1.3.3 Affiliation Motivation . .... .... .... .... ..... .... 13
1.3.4 Power Motivation .... .... .... .... .... ..... .... 14
1.3.5 Dominant Motives and Motive Profiles.... ..... .... 15
1.3.6 Motivation and Zeitgeist ... .... .... .... ..... .... 16
1.4 Conclusion ... .... ..... .... .... .... .... .... ..... .... 17
References. .... .... .... ..... .... .... .... .... .... ..... .... 18
2 Computational Models of Achievement, Affiliation
and Power Motivation... ..... .... .... .... .... .... ..... .... 21
2.1 Towards Computational Motivation . .... .... .... ..... .... 21
2.2 Modelling Incentive-Based Motives Using
Approach-Avoidance Theory .. .... .... .... .... ..... .... 22
2.2.1 Modelling Achievement Motivation .. .... ..... .... 26
2.2.2 Modelling Affiliation Motivation. .... .... ..... .... 31
2.2.3 Modelling Power Motivation.... .... .... ..... .... 33
2.3 Motive Profiles for Artificial Agents. .... .... .... ..... .... 34
2.3.1 Modelling Profiles of Achievement, Affiliation
and Power. ..... .... .... .... .... .... ..... .... 35
2.3.2 Modelling the Dominant Motive Only. .... ..... .... 36
2.3.3 Optimally Motivating Incentive.. .... .... ..... .... 36
2.4 Using Motive Profiles for Goal Selection. .... .... ..... .... 39
2.4.1 Winner-Takes-All .... .... .... .... .... ..... .... 40
2.4.2 Probabilistic Goal Selection. .... .... .... ..... .... 41
ix
x Contents
2.5 Summary .... .... ..... .... .... .... .... .... ..... .... 41
References. .... .... .... ..... .... .... .... .... .... ..... .... 42
3 Game-Playing Agents and Non-player Characters . .... ..... .... 45
3.1 Artificial Intelligence in Non-player Characters .... ..... .... 45
3.2 Rule-Based Agents. ..... .... .... .... .... .... ..... .... 46
3.2.1 Game Scenarios for Motivated Rule-Based Agents.... 47
3.2.2 Motivated Rule-Based Agents... .... .... ..... .... 48
3.3 Crowds.. .... .... ..... .... .... .... .... .... ..... .... 49
3.3.1 Motivated Crowds.... .... .... .... .... ..... .... 51
3.4 Learning Agents... ..... .... .... .... .... .... ..... .... 52
3.4.1 Social Dilemma Games.... .... .... .... ..... .... 53
3.4.2 Strategies in Game Theory . .... .... .... ..... .... 55
3.4.3 Motivated Learning Agents. .... .... .... ..... .... 56
3.5 Evolution .... .... ..... .... .... .... .... .... ..... .... 59
3.5.1 Multiplayer Social Dilemma Games .. .... ..... .... 59
3.5.2 Evolution in Multiplayer Social Dilemma Games . .... 60
3.5.3 Evolution of Motivated Agents.. .... .... ..... .... 60
3.6 Summary .... .... ..... .... .... .... .... .... ..... .... 64
References. .... .... .... ..... .... .... .... .... .... ..... .... 64
Part II Comparing Human and Artificial Motives
4 Achievement Motivation. ..... .... .... .... .... .... ..... .... 69
4.1 Scenarios and Mini-games for Motivated Agents... ..... .... 69
4.2 The Ring-Toss Game .... .... .... .... .... .... ..... .... 70
4.3 Modelling Achievement-Motivated Rule-Based Agents ... .... 72
4.4 AchAgents Playing the Ring-Toss Game . .... .... ..... .... 75
4.4.1 Comparison to Humans.... .... .... .... ..... .... 75
4.4.2 Comparison to Atkinson’s Risk-Taking Model ... .... 77
4.4.3 Comparison to a Model of Achievement Motivation
with Learning ... .... .... .... .... .... ..... .... 79
4.5 Conclusion ... .... ..... .... .... .... .... .... ..... .... 80
References. .... .... .... ..... .... .... .... .... .... ..... .... 82
5 Profiles of Achievement, Affiliation and Power Motivation ... .... 83
5.1 Evolving Motivated Agents to Play Multiple Games ..... .... 83
5.2 Roulette . .... .... ..... .... .... .... .... .... ..... .... 84
5.2.1 Motive Profiles for Roulette .... .... .... ..... .... 85
5.2.2 Motivated Rule-Based Agents Playing Roulette .. .... 87
5.2.3 Comparison to Humans.... .... .... .... ..... .... 89
5.3 The Prisoners’ Dilemma Game. .... .... .... .... ..... .... 90
5.3.1 Motivated Rule-Based Agents and the Prisoners’
Dilemma .. ..... .... .... .... .... .... ..... .... 92
5.3.2 Comparison to Humans.... .... .... .... ..... .... 92
5.4 Conclusion ... .... ..... .... .... .... .... .... ..... .... 94
References. .... .... .... ..... .... .... .... .... .... ..... .... 95