Modelling Natural Action Selection

569 Pages·2011·6.35 MB·English

by  Anil K. Seth

ModellingNaturalActionSelection
Actionselectionisafundamentalprobleminbiologyandecology.Itrequiresdetermining
availablealternatives,executingthosemostappropriate,andresolvingconflictsamong
competinggoalsandpossibilities.
Using advanced computational modelling, this book explores cutting-edge research
into action selection in nature from a wide range of disciplines, from neuroscience
tobehaviouralecology,andeventopoliticalscience.Itdeliversnewinsightsintoboth
detailedandsystems-levelattributesofnaturalintelligenceanddemonstratesadvancesin
methodologicalpractice.Contributionsfromleadingresearcherscoverissuesincluding
whetherbiologicalactionselectionisoptimal,neuralsubstratesforactionselectionin
thevertebratebrain,perceptualselectionindecisionmaking,andinteractionsbetween
groupandindividualactionselection.
Thismajorintegratedreviewofactionselectioninnaturecontainsabalanceofreview
andoriginalresearchmaterial,consolidatingcurrentknowledgeintoavaluablereference
forresearchers,whileillustratingpotentialpathsforfuturestudies.
AnilK.SethisaReaderintheSchoolofInformaticsattheUniversityofSussex,anEPSRC
Leadership Fellow, and Co-Director of the Sackler Centre for Consciousness Science.
His research crosses the fields of computational neuroscience, consciousness science,
andneurorobotics.Inadditiontocontributingnewinsightsintothemechanismsofaction
selection,hehasdevelopednewwaystolinkbrainactivitytoconsciousexperienceand
heiswellknownforhisresearchonthestatisticalanalysisofcausality.
Tony J. Prescott is Professor of Cognitive Neuroscience at the University of Sheffield
whereheteachescoursesoncomputationalneuroscienceandbiomimeticrobotics.His
research lies within the biological and brain sciences, and concerns understanding the
evolution, development, and function of natural intelligence. He is particularly well
knownforhisworkonmodellingtheneuralsubstratesforactionselection,andbuilding
robotmodelsofanimalandhumanbehaviour.
JoannaJ.BrysonisaReaderintheDepartmentofComputerScienceattheUniversity
ofBath.Sheconductsinterdisciplinaryresearchontheorigins,structure,andconstruc-
tionofhumanandanimal-likeintelligence,andisinvolvedintopicsrangingfromthe
evolution of altruistic communication to the ethical role of robots in our society. She
is recognised for her work in systems AI and the design of action selection. Her most
recentworkiscentredontheapplicationofmodellingtounderstandingtheevolutionof
humansocialstructuresandculturallyderivedbehaviourmoregenerally.

Modelling Natural Action
Selection
Editedby
ANIL K. SETH
UniversityofSussex
TONY J. PRESCOTT
UniversityofSheffield
JOANNA J. BRYSON
UniversityofBath
CAMBRIDGEUNIVERSITYPRESS
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,Sa˜oPaulo,Delhi,Tokyo,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107000490
(cid:2)C CambridgeUniversityPress2012
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2012
PrintedintheUnitedKingdomattheUniversityPress,Cambridge
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloguinginPublicationdata
Modellingnaturalactionselection/editedbyAnilK.Seth,TonyJ.Prescott,JoannaJ.Bryson.
p. cm.
Includesbibliographicalreferencesandindex.
ISBN978-1-107-00049-0
1.Cognitiveneuroscience. 2.Neurobiology–Computersimulation. 3.Decisionmaking–Computer
simulation. I.Seth,AnilK. II.Prescott,TonyJ. III.Bryson,JoannaJ. IV.Title.
QP360.5.M63 2011
612.8(cid:3)233–dc23 2011027497
ISBN978-1-107-00049-0Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredto
inthispublication,anddoesnotguaranteethatanycontentonsuch
websitesis,orwillremain,accurateorappropriate.
Contents
Foreword pageviii
PeterM.Todd
Preface xi
Listofcontributors xii
1 Generalintroduction 1
AnilK.Seth,JoannaJ.Bryson,andTonyJ.Prescott
PartI Rationalandoptimaldecisionmaking
2 IntroductiontoPartI:rationalandoptimaldecisionmaking 7
AnilK.Seth,TonyJ.Prescott,andJoannaJ.Bryson
3 Doweexpectnaturalselectiontoproducerationalbehaviour? 12
AlasdairI.Houston,JohnM.McNamara,andMarkD.Steer
4 Optimisedagent-basedmodellingofactionselection 37
AnilK.Seth
5 Compromisestrategiesforactionselection 61
FrederickL.Crabbe
6 Extendingabiologicallyinspiredmodelofchoice:multi-alternatives,
nonlinearity,andvalue-basedmultidimensionalchoice 91
RafalBogacz,MariusUsher,JiaxiangZhang,andJamesL.McClelland
7 Bayesianapproachestomodellingactionselection 120
MaxBerniker,KunlinWei,andKonradKo¨rding
8 Post-retrievalinhibitioninsequentialmemorysearch 144
EddyJ.Davelaar
PartII Computationalneurosciencemodels
9 IntroductiontoPartII:computationalneurosciencemodels 169
TonyJ.Prescott,JoannaJ.Bryson,andAnilK.Seth
vi Contents
10 Actionselectionandrefinementinsubcorticalloopsthroughbasalganglia
andcerebellum 176
JamesC.Houk
11 Corticalmechanismsofactionselection:theaffordancecompetition
hypothesis 208
PaulCisek
12 Towardanexecutivewithoutahomunculus:computationalmodelsofthe
prefrontalcortex/basalgangliasystem 239
ThomasE.Hazy,MichaelJ.Frank,andRandallC.O’Reilly
13 Hierarchicallyorganisedbehaviouranditsneuralfoundations:
areinforcement-learningperspective 264
MatthewM.Botvinick,YaelNiv,andAndrewG.Barto
14 Themedialreticularformation:abrainstemsubstrateforsimpleaction
selection? 300
MarkD.Humphries,KevinN.Gurney,andTonyJ.Prescott
15 Understandingdecision-makingdeficitsinneurologicalconditions:insights
frommodelsofnaturalactionselection 330
MichaelJ.Frank,AnoukScheres,andScottJ.Sherman
16 Biologicallyconstrainedactionselectionimprovescognitivecontrolinamodel
oftheStrooptask 363
TomStaffordandKevinN.Gurney
17 Mechanismsofchoiceintheprimatebrain:aquicklookatpositive
feedback 390
JonathanM.Chambers,KevinN.Gurney,MarkD.Humphries,andTonyJ.Prescott
PartIII Actionselectioninsocialcontexts
18 IntroductiontoPartIII:actionselectioninsocialcontexts 421
JoannaJ.Bryson,TonyJ.Prescott,andAnilK.Seth
19 Agent-basedmodelsasscientificmethodology:acasestudyanalysingthe
DomWorldtheoryofprimatesocialstructureandfemaledominance 427
JoannaJ.Bryson,YasushiAndo,andHagenLehmann
20 Anagent-basedmodelofgroupdecisionmakinginbaboons 454
RussellA.Hill,BrianS.Logan,WilliamI.Sellers,andJulianZappala
21 Endogenousbirthanddeathofpoliticalpartiesindynamicparty
competition 477
MichaelLaver,ErnestSergenti,andMichelSchilperoord
Contents vii
22 Onoptimaldecisionmakinginbrainsandsocialinsectcolonies 500
JamesA.R.Marshall,RafalBogacz,AnnaDornhaus,RobertPlanque´,TimKovacs,andNigelR.Franks
23 State-dependentforagingrulesforsocialanimalsinselfishherds 523
SeanA.Rands,RichardA.Pettifor,J.MarcusRowcliffe,andGuyCowlishaw
Index 538
Colourplatesaretobefoundbetweenpp.240and241.
Foreword
Theseapenleadsasimplelife.Afterfloatingonthedeep-seacurrentsasajuvenile,it
settlesdownontoacomfortablepatchofsandandbeginsitslargelyimmobileadultlife,
growingintoafeatheryshapeandswayinginthewaterwhileensnaringwhateveredible
morselspassitsway.Ithardlymovesonitsown;itjustpassivelyfilterstheworldthat
goesby.Nochoicesneedbemade,fortherearenoactionstotake.Assuch,theseapen
willnotfeatureprominentlyinthisbook.
For other more active (and more behaviourally interesting) species, life presents a
streamofdecisionpoints,atwhichactionsmustbechosen:stayormoveon,ingestor
passby,approachoravoid,waitorstrike,court/acceptordecline.Theseareallessentially
formsoftheexploitation/explorationtrade-offthatorganismsmustbalancethroughout
their lives, whenever resources are distributed in space or time and the individual can
activelyseekthem.Thisistherealmofnaturalactionselection.Howdoorganismsdo
it?
Action selection mechanisms are decision mechanisms. Like the study of decision
mechanisms used for making inferences – a common topic in research on (human)
judgement and decision making – the study of action selection mechanisms aims to
uncover what the mechanisms are that people and other animals use, how they work,
and when and where they work well or do not work – that is, the conditions under
whichtheydoordonotproduceadaptive (orrational)behaviour. Researchonhuman
inference has revealed that there are multiple decision mechanisms that people can
use in particular situations – the mind’s adaptive toolbox – and often several of these
can produce similar outcomes. The chapters in this book reveal the range of possible
action selection mechanisms that can be used as well. Some of the simple heuristics
thathavebeenidentifiedforhumaninferenceshowthatdecisionscanbemadewithout
fullycomparingdifferentalternativesonmultipleincommensuratedimensions(e.g.,by
ignoringconflictingdimensionsandjustdecidingonthebasisofonefactor);someof
themodelsinthisbookshowsimilarshortcutsorrulesofthumbthatcanworkinaction
selection.Onedifferencebetweenstandarddecision-makingresearchandthatonnatural
action selection is that, in the former, the possible alternatives that can be selected in
aninferencetaskaregenerallyknownorspecifiedbytheresearcher,e.g.,whichcityis
larger,DetroitorMilwaukee?Thepossibleactionsthatananimalmighttakeinnature
may not always be explicitly identified and, without knowing precisely the range of
possibilities,studyingthemechanismsthatselectamongthembecomesmoredifficult.
Foreword ix
In the human inference literature, the study of the contents of the adaptive toolbox
has led to a further question beyond asking how people choose between alternatives
in an inferential task: how do people choose what mechanism to use to make their
inferences in a particular task? That is, how do people choose which tool to use from
theadaptivetoolbox?Thisisanopenandchallengingquestion,withdifferentpossible
answers;sometimestheenvironmentwilldeterminethechoice,dependingonwhattypes
of information are available; other times, individuals may learn through experience or
can be taught what decision mechanisms are good to employ in particular settings. If
therearemultiplecompetingactionselectionmechanismsinagivensituation–notjust
multiplecompetingactionsthemselves–thenthiskindofmeta-selectionquestionwill
alsobeimportantinthestudyofnaturalactionselection.
Specifyingtheactionselectionmechanismsthatinhabitthemind’sadaptivetoolbox
meansspecifyinghowtheywork–buthowcanthisbedetermined?Thisisalsoamain
themeofthisbook:exploringthemethodsthatarebeingemployedatthecuttingedgeof
researchforunderstandingtheoperationofactionselectionmechanisms.Experiments
and field observations are necessary for generating data on action selection in action,
butthedatamustbeunderstoodintermsofatheory.Herethetheoriesareinstantiated
asmodels,whichbecausetheyoftenoperateatmultiplelevels–cognitive,neural,and
social – are complex enough that they are best implemented as computer simulations.
Ideally,multiplecompetingmodelsthenspecifyfurtherdatatobecollectedtodistinguish
betweenthem,andthenewdataconstrainthemodels(andthetheoreticalunderstanding)
further,inanongoingfeedbackloop.Inthisbookweseemultiplestagesofthatprocess,
andthecurrentstateofunderstandingtowhichithasled.
This modelling approach can thus be characterised as ‘understanding by building’.
But once the action selection models have been built and understood, we can also use
themodelsasawaytoexplorehowtochangethingsinimportantapplicationareas.The
applications,likethemodels,canbeatdifferentlevelsofdescription.Attheneurallevel,
important health implications emerge from understanding clinical conditions such as
Parkinson’sdiseaseandobsessive–compulsivedisorderasdisruptionsofactionselection.
And by better knowing how natural action selection works at this level, we can gain
betterunderstandingofwhatisgoingonwhenthingsgowrong–tweakingthemodels
toachieve‘understandingbybreaking’–andinsightsintopossiblewaysofaddressing
thoseproblems.
Attheindividualandsociallevel,wemaywanttohelppeoplechoosebettercourses
of actions for themselves and others – avoiding the third piece of chocolate cake, or
promotingtheelectionofinnovativeleaders.Againbymodellingandunderstandingthe
processes of action selection at these levels of description, we can develop, and test,
ideasforhowtochangetheactionselectionprocessindesireddirections,beforetrying
themoutintherealworld.Thesemethodscouldinvolvegivingpeopleandgroupsnew
waysofselectingtheiractions–newtoolsforchoosingwhatandwhentoeat,ornew
voting mechanisms, for instance – or could rely on changing the environmental cues
thattheyreceive,to‘nudge’peopleintomakingdifferentselections.Andmodellingcan
help us determine which approach may be more effective, changing environments or
changingmindsornorms.