the Role of Emotion in Problem Solving

9 Pages·0.721 MB·English

by  Belavkin R.V.

THE ROLE OF EMOTION IN PROBLEM SOLVING
Roman V Belavkin
School ofComputerScience andInformationTechnology
UniversityofNottingham,Nottingham,NG81BB,UK
[email protected]
Abstract
Performanceanddatafromsomecognitivemodelssuggestedthatemotions,experiencedduringproblemsolving,should
betakenintoaccount. Moreover, itisproposedthatthecognitivescienceapproachusingboththeoreticalandexperi-
mentaldatamayleadtoabetterunderstandingofthephenomena. AcloserinvestigationofACT-Rcognitivearchitecture
(Anderson1993) revealedsomepropertiesanalogous tophenomena knownfromtheactivationtheoryofemotion. A
modeloftheclassicalYerkes-Dodsonexperimentwasbuilttotestthepredictions. Thestudyexplainedsuchpsycho-
logicalphenomenaasarousal,motivationandconfidencewithinthemathematicalnotation. Theinfluenceofchanges
inthesemotivationalstates,controlledbyemotion,oninformationprocessinghasbeeninvestigatedanditisshownthat
thedynamicscorrespondstothewell-knownoptimisationmethods,suchasbest-firstsearchandsimulatedannealing.
1 Introduction
componentofintelligence,asalsosuggestedbyGoleman
(1995).But,asnotedin(Sloman1999),thisconclusionis
Recent progress in cognitive modelling has allowed the alittlepremature,astheroleofthedamagedbrainareasis
testing of quitea broad rangeof humanpsychologyand stillnotsufficientlywellunderstoodduetothecomplex-
cognitiontheories.Alotofworkinexperimentalpsychol- ityofthehumanbrain.
ogy has been reproduced and reconsidered by cognitive Perhaps, in order to support these experimental ob-
scientists within theories such as SOAR (Newell 1990) servations, oneneedssome theoryexplainingthesephe-
or ACT (Anderson 1993). This work has produced new nomenaas a mathematicalmodel. We believe that such
insights into our understanding of some phenomena of atheorycanbecreatedwithintheframeofcognitivesci-
humanmemory,learning,perceptionandreasoning. Yet ence. If the role of emotion in cognition can be under-
therehavebeenfew(ifany)attemptstounderstandemo- stood within this theory then, perhaps, we shall also be
tionandaffectwithinthisframework. abletodecidewhatroleitplaysinintelligence.
Thesubjectofemotionhasbotheredmanypsycholo- It is becoming evidentthat cognitive modelsused to
gists,philosophersandneurobiologistssinceWilliamJa- testdifferentpsychologicaltheoriesshouldtakeemotion
mes first attempted to defineemotion. Many theoriesof into account (Belavkin, Ritter & Elliman 1999). Many
emotion, sometimes quite contradictory, have appeared of these models simulate subjects solving various puz-
sincethen(see(Plutchik1994,LeDoux1996)forreviews). zlesandproblems,somemodelsconsiderchildren,asin
Recentlythesubjectofemotionhasattractedtheattention (Jones, Ritter & Wood 2000), whose emotions are eas-
ofthecomputerscienceandartificialintelligencecommu- ily observable. We know from our own experience and
nities,andhasemergedintoanewareaofresearch,some- observationthatemotionaccompaniesanyproblemsolv-
times referred as affective computing (see (Picard 1997) ing process, but these computer simulations say nothing
forreview). aboutit. Itseemsthatincognitivesciencethissubjecthas
Althoughthereis no doubtemotionis a very impor- notyetbeenstudieddeeplyenough.
tant component of human and animal psychology, one Thisisnotto claimthatcognitionandemotionwere
of the most intriguing and interesting question remains notconsideredtogetheratall,buttherewasnotmanyat-
unanswered: Isaffectandemotionthenecessarycompo- tempts to study this subject with cognitive architectures
nent of intelligence? And if it is, how should it be in- suchasSOARorACT-R. Althoughthereisalreadyanum-
cludedintoAItheory? berofcognitiveappraisalmodels,suchas(Ortony,Clore
It is known from the whole history of experimental & Collins 1988) or (Roseman, Antoniou & Jose 1996),
psychology that emotion is closely related to cognitive thatallowustoconcludewhichemotionasubjectshould
processes such as learning, decision making and mem- feelundergivencircumstances,thesesymbolicrepresen-
ory. Recently there have been claims, based on some tations do notexplainwhathappensto the thinkingpro-
experimental evidence, that damage to emotion respon- cessitselfasaresultoftheseemotions.Whatisthediffer-
sibleareasofthebrainimpairsthesecognitiveprocesses encebetweenassemblingthetowerofHanoiinanangry
(Damasio 1994), and thereforeemotion is the necessary orahappymood?
2 Modelling the cognition
Although cognitive science is moving towards a unified
theoryofcognition,thereareseveralschoolsandarchitec-
turesimplementingdifferentinterpretations. Manyideas
in this work were developed after observation of some
cognitivemodels,implementedinACT-R(Anderson1993),
andaftercloserinvestigationofthe ACT-R conflictreso- Figure 1: The Tower of Nottingham and two of 21
lutionmechanism.
woodenblocksusedtobuildtheTowerofNottingham
2.1 ACT-R architecture andmodels
young problem solvers, Jones tried several architectural
ThemaindistinguishingfeatureoftheACT-Rarchitecture changes using parameters, such as number of chunks in
is its sub-symbolic processing capabilities. The mecha- workingmemory,retrievalthreshold,foveaandparafovea
nism, called rational analysis, acts underneath the pro- sizes(perception)andothers.
duction system and it can be seen as a stochastic opti- It is not necessary to describe all the results of that
misation mechanism. Inbrief, allthesymbolsin ACT-R workhere,buttherewasoneparticularadjustmenttothe
(declarative knowledgeunits and productionrules) have model,thataloneproducedexcellentresults(othersingle
someactivationvaluesandassociationsbetweenthemwith parameters could not produce such a good correlation).
differentstrengths. Valuesof theseactivationsandasso- It was themodelwith increased noisein conflictresolu-
ciationsaffectmanyprocessesin theproductionsystem, tion. Thecorrespondingparameter:egn(expectedgain
such as retrieval of knowledge facts, conflict resolution noise),whensettoavalueof6.0,producedaparticularly
(the choice of one production rule from several satisfy- goodmatchforthedatasuchastimeneededtocomplete
ingthecurrentcondition)andthesevaluesarestatistically each layer or the number of constructionsassembled on
learned,maydecayintimeandbeaffectedbyglobalpa- eachlayer(Figure2. From(Jonesetal.2000)).
rameters.
Such a mechanism can be controlled through many
parameters, such as noise variance, goal value, retrieval
andutilitythresholds,etc. Theseparametersmaydramat-
icallyalterthebehaviouroftheproductionsystemandare
usedbycognitivemodellerstoadjusttheperformanceof
theirmodelsandtestsometheories.
AlthoughACT-Rwassometimescriticisedforhaving
toomanyparameters(andhenceagoodcorrelationwith
the data), there is a great deal of experimentalevidence Figure2: Timetakentoassembleeachlayerandnumber
for including them in the theory. There is even a neu- ofconstructionsassembledoneachlayeroftheTowerby
ralimplementation ACT-RN (Lebiere&Anderson1993) childrenandEGN6model.
(“N” for neural), and the latest version of ACT-R pos-
The fact that children seem to be more “noisy” or,
sesses most of its connectionistpropertieswhile still re-
tainingthehighlevelofabstractionallowingforencoding speaking in ACT terms, less rational than adults lead to
some interesting speculations and questions. For exam-
andsolvingcomplexcognitiveproblems.
ple, it may indicate that emotions play a greater role in
children’slearningandproblemsolving. Indeed,joyand
2.2 The TowerofNottingham
frustration are more observable in children. Could we
Aswasmentionedearlier,manyideasforthisworkwere possiblymodelyoungerchildrenbyincreasingthenoise
inspiredbytheresultsoftheTowerofNottinghammodel evenfurther?Orcouldtheseorbetterresultsbeproduced
(Jonesetal.2000),whichwasusedtostudycognitivede- bysomeotherparametersinconflictresolution?
velopment. Theideaof thisworkwas to createamodel
resemblingthebehaviourofadultsubjectsassemblingthe
3 Decision making in ACT-R
TowerofNottinghampuzzle(Figure1)andthentoachieve
a match with the data from seven years old children by
Thequestionsmentionedinprevioussectionwerestudied
modifyingtheoriginaladultsmodel.
by the author during the experiments with the Tower of
The modelwas implementedin ACT-R and used the
Nottingham model and a closer investigation of ACT-R
Nottingham “Eye and Hand” perception-action module
conflict resolution mechanism formed the basis for this
(Baxter & Ritter 1996) to interact with the task simula-
study.Wedonotneedtoexplainherethewholespectraof
tion. It achieved a fair match with the data from adult
mechanismsandparametersin ACT-Randweshallrefer
subjects at default parameters settings. Then, to match
to(Anderson&Lebiere1998)forfurtherinformation,but
it is important to give a brief introductionto the ACT-R thechoicewillbecomemorerandomandlessdependent
conflict resolution mechanism and its notation, since it onPsandCsofrulesinconflictset.
willbeusedintherestofthepaper. Thosefamiliarwith As mentioned above, there is a great deal of exper-
ACT-Rmayskipthenextsection. imental evidence confirming the plausibility of ACT-R’s
choicemechanism.Forexample,thedatafrom(Friedman,
3.1 Conflictresolutioninbrief Burke, Cole, Keller, Millward & Estes 1964)showsthat
althoughsubjects choose accordingto the probabilityof
InACTconflictresolution(aprocessofselectingonerule success (or reinforcement), the proportion of choices of
outofseveralmatchingthecondition)isrealisedthrough an alternative with maximum success probability (P =
its rationalanalysis model, which uses the subsymbolic 1.0)neverreaches1.0.Similarly,subjectsstillsometimes
information. Everyrule, in additionto its symbolicrep- chooseeventhemostunfortunatealternative(with0suc-
resentation,hasalsosocalledexpectedgainE andarule cessprobability).So,thereisalwayssomedegreeofran-
with the highest gain wins the competition in a conflict domness (noise) in their choice. Other works on choice
set. probability,suchas(Myers,Fort,Katz&Suydam1963),
ExpectedgainE iscalculatedbythefollowingequa- showed that the choicedependsmore on the probability
tion: ofsuccessforhigherrewards(goalvalue).
E =PG−C+ξ(τ), (1)
whereP is expectedprobabilityof achievingthegoalif 3.2 Asymptoticproperties ofrationality
the rule fires, G is the value of the current goal in time
Only one rule can be selected to fire on each cycle and
units,C isexpectedcostofthatruleintimeunits(itrep-
withnproductionrulesintheconflictset,theprobability
resentshowlongwillittaketoachievethegoalifthatrule
of selecting a particular i-th one is given by Boltzmann
fires),andξ(τ)isarandomvariablerepresentingnoisyor
equation:
non-deterministicpartofACT-Rconflictresolutionmech-
eEi/τ
anism. Thelevelofthisnoisecanbecontrolledthrougha p = (cid:1) , (2)
globalvariablecalledexpectedgainnoisevarian√ceσand i nj=1eEj/τ
weshallrefertoitasthenoisetemperatureτ = σ/π. where E is the evaluation of i-th rule, which is P G−
i i
Although there is a lot of experimental evidence in C . We are interested in how the choice probability p
i i
favourofACTrationalanalysismodel,unfortunately,the dependsonP andC forextremelyhighorlowvaluesof
i i
ACT-R book (Anderson & Lebiere 1998) does not give goalvalueGandnoisetemperatureτ.
anyjustificationorreferencetoasourceofaboveformula Forsimplicity,letusconsiderthecaseoftwoproduc-
(perhaps,assumingthateveryreaderisfamiliarwithBell- tionrulesandletP andP betheirexpectedprobabilities
1 2
man’s dynamic programming theory). In this paper the andC andC theircosts.WemayalsotakeP =1−P .
1 2 2 1
probabilisticinterpretationofexpectedgainwillbeintro- Thentheprobabilityp ofchoosingthefirstoftworules
1
duced and we shall see howequation(1) can bederived willbecalculatedas
fromit. Firstletustakeacloserlookatthisformula.
As we can see the expected probability P and ex- e(P1G−C1)/τ
p = .
pectedcostCarepropertiesoftheproductionrule.These 1 e(P1G−C1)/τ +e((1−P1)G−C2)/τ
properties can be learned statistically and there is also
Nowweshalldescribetheasymptotesofthischoiceprob-
a mechanism to “forget” some of this information with
ability and resulting behaviour of the system at extreme
time, called probabilitydecay. So, if therearetwo rules
valuesofGandτ. Thesepropertieshavebeenshownand
matchingthecurrentgoalandACT-Rmodellearnedfrom
thentestedonamodelin(Belavkin1999).
previousexperiencethat applyingthe first rule will lead
tothegoalin5minutes(C = 5)withexpectedprobabil- i) τ → 0 (nonoise). Inthiscase thesystementirely
ity P = .5, while for the second rule C = 10, P = .8
reliesonstatisticalinformationandmaybetoode-
respectively,thenforG=20(defaultvalue)theexpected
terministic.Wespeculatethatthebehaviourmodels
gains of these rules will be E = .5·20− 5 = 5 and
1 insomecasesthebehaviourthathasbeendescribed
E = .8·20−10 = 6. Inthisexamplethesecondrule
2 byDamasioconcerningsomeofhispatients, such
willbeselectedforE >E .
2 1 as repetitiveerrors, inability to choosefrom equal
Butnotonlystatisticalinformationaboutexpectedprob-
opportunities. Thefirstisduetoexcessivereliance
abilities andcostsof aruleaffecttheconflictresolution.
on the past experience, which may become obso-
ThegoalvalueGisapropertyofthecurrentgoalandit
leteinachangingenvironmentbecauseittakestoo
isglobalparameteraswellasthenoisetemperatureτ. In
longtolearnnewstatisticstooverridetheoldones.
the aboveexampleonecan easily checkthatfor alower
Thesecondtypeofbehaviouroccurssimplydueto
goalvalues,suchasG=14,thefirstrulewillhavehigher
the possibility of equal expected gains for several
expectedgainanditwillbeselected,despiteitslowerex-
rules.
pectedprobability. Also,ifthenoisevarianceistoohigh
ii) τ →∞(highnoise).Itiseasytoshowthat ofaproblemsolverandthevaluesofGandτ represent-
ingtheactivationorthe“energy”ofacognitiveprocess,
1
p → , ∀P ,C whichiscalledarousalinactivationtheoryofemotion.If
1 i i
2 thisistrue,thenthelawsofactivationtheoryofemotion,
Inthiscasethechoicebecomescompletelyrandom suchasYerkes-DodsonInverted-Ucurverelatingarousal
(or irrational)since it doesnotdependon thepast to performance, should also apply to ACT-R cognitive
experience at all. Note, that sometimes such be- models.
haviourmaybeuseful.
4.1 The Yerkes-Dodson experiment
iii) G→0(lowmotivation).Then
In order to test this idea, a model replicating the classi-
p → e−C1 . cal “dancing mouse” experimentby Yerkes and Dodson
1 e−C1 +e−C2 (Yerkes & Dodson 1908) was built (Belavkin & Ritter
Inthiscasethechoiceiscompletelydeterminedby 2000)usingACT-R.
Intheoriginalexperimentamousewasplacedintoa
the the costs C of rules and not by the expected
discriminationboxwithtwoexits:onemarkedbyawhite
probabilitiesP. Thesystemistryingtoputaslittle
cardandanotherbyablackone.Amousewastrainedini-
effortsintothetaskaspossibleanddoesnot“care”
tiallytoexittheboxthroughanydoor,butaftertwodays
abouttheprobabilityofsuccessfuloutcome.
itwasonlyallowedtoexitthroughthewhitedoorandif
iv) G → ∞ (too high motivation). In this case the itdidamistake,itwassubjectedtoaslightelectricshock
resulting p will depend on the value of expected at the black door. The order of the doors was changed
1
probabilityP , which can bebetween 0 and 1. In randomlyandresearchersmeasuredthenumberofwrong
1
theextremecasesweshallhave: choicesamousemadeeachdayuntilaperfecthabithad
beenformed(i.e.,whennoerrorswereproducedforthree
p1 →1 for P1 =1 consequetivedays).
p →0 for P =0. Such experiments with certain variations were per-
1 1
formed by many psychologists studying discrimination
Thiscaseisoppositetothepreviousone:thechoice learning or perception in the early 20-th century. The
doesnotdependonthecostsC,butispurelydeter- remarkablefeatureofYerkesandDodsonworkwasthat
minedbytheexpectedprobabilitiesP. Thesystem they looked at the speed of learning with respect to the
doesnotpayattentiontotheeffort(time)itspends, strengthofstimuli. Theychangedthelightingofthebox
andachievesthegoalwhateverthecost. sothatitwaseasierorhardertodiscriminatebetweenthe
whiteandtheblackdoors,andtheyincreasedthestrength
Note, that if we increase both G and τ keeping the
oftheelectricshockfromweaktomediumandstrong.
ratio G/τ constant, then expected probabilities become
The main result of this experimentwas that the best
moreimportantthencosts. Indeed,letτ → ∞andG →
(fastest)learningoccurredunderthemediumstimulus(Fig-
∞. In this case, like in iii), asymptotes of p are deter-
1 ure3. From(Yerkes&Dodson1908)). Performancefor
minedbythevalueofP :
1 thestrongstimuluswasbetterthenfortheweakone,butit
e wasworsethanmedium,especiallywhenvisualdiscrim-
p → for P =1
1 1 inationwasnotperfect.
e+1
1
p → for P =0.
1 e+1 1 4.2 Task simulationand“dancer” model
So, p for P = 1 is e times bigger than for P = 0 ThetasksimulationwasimplementedusingCommonLisp
1 1 1
(e>1). andGarnetgraphicslibrary.Theuserinterfaceconsistsof
Similarly, for both G → 0 and τ → 0 the costs be- threemainwindows(Figure4): discriminationboxwith
comemoreimportant. ItmeansthatthevaluesofGand three roomsand two doors, a controlpanelwith aslider
τ areimportantandnotonlytheirratioG/τ. forsettingthecontrastbetweenthetwodoorsandacon-
trol for setting the virtual “voltage”, and the third is a
window for displaying the number of errors a mouse is
4 On activation theory of emotion
makingandotherdata. Thecontrastbetweenthedoorsis
relatedtotheG/τ ratioin ACT-R whilethevoltagecon-
Asymptoticpropertiesofchoiceprobabilityshowthatthe
trolindirectlysetsthegoalvalueG.
ratio G/τ determines how much the choice depends on
Thedancingmouseisrepresentedbyaredarrowhead
the learnedstatisticalinformation,whilethevaluesofG
objectanditsactionsarecontrolledbyacognitivemodel,
and τ determine whether the choice is made from costs
implemented with ACT-R. It uses a simple perception-
ofprobabilityperspective. Basedonthisobservationwe
actioncycleandtwo-dimensionalworldrepresentationmodel.
maythinkoftheG/τ ratioasanindicatorofconfidence
sented by a specialchunk-typechoice. When themodel
ispresentedwithachoicechunkasagoal,ithasinitally
onlytwoactionsencodedintwosimpleproductionrules
liketheonebellowtochoosethefirstobject:
IFthegoalisachoiceoffirstorsecond
THENfocusonfirst
andasimilarruleforchoosingthesecondobject.Hereno
featuresof thefirst andsecond objectsareused to make
thedecision.
With no electric shock behind the doors the mouse
choosesdoorsrandomlyanddueto thestatistical proba-
bilitylearningitmayeventuallyformaslightpreference
for one of the doors. When the shock is introduced, on
choosing the wrong door (black door)the mouserecalls
the last choiceit has made and learns to choose another
alternativepayingnowattentiontothefeaturesoftheob-
Figure 3: Learning curves for weak “W”, medium “M” jects it is choosing. This learning uses the production
and strong “S” stimuli undermedium visual discrimina- compilationmechanismanditmayaddanewproduction
tion.Abscissaerepresenttheaveragenumberoferrorsfor rulelike:
fourmiceproducedintentestsforeachseries. Ordinates
representthenumberofseries. IFthegoalisachoiceoffirstorsecond
ANDfirstisablackdoor
THENfocusonsecond
So, if next time the mouse is presented with the same
choiceagain, itwill alreadyhavethreeproductionrules.
Note,thatintheaboverulethelearningoccursbasedon
onedimension—colour. Anotherdimensionisdoorpo-
sition(leftorright)andamousecanlearntochoosebased
onlyonthepositionofthedoor. Two-dimensionallearn-
ing, when both colours and positions are taken into ac-
count,isalsopossible.Theprobabilityofusingthecolour
informationinlearningdependsonthecontrastbetween
thedoors.
Figure4: User interfaceof the “dancing”mouseexperi-
The modelwas tested under severalscenarios. With
ment simulation. Left: the discrimination box with two
thecontrastcontrolsettoaparticularvalue,theG/τ ratio
doors. Right: asimplecontrolpanelwithasliderforset-
remainedconstant,whilethevaluesofGandτ werecon-
tingacontrastbetweenthedoors(andtheG/τ ratio)and
trolledbythe“voltage”goingfromsmalltohighvalues.
voltagecontrolrelatedtoGvalue.
In the first experiment only the speed of probability
learningwasstudiedsinceGandτ parametersaffectonly
theconflictresolutionandnottheproductioncompilation.
It has visual sensors sending information about the ob-
So,thedoorsremainedinthesameorder(statisticallearn-
jectrightaheadandskinsensorssendingtheinformation
ingwouldnotworkiftheorderofthedoorswasrandom)
aboutthe strength of externalstimulus. The currentand
andinformationabouteventualsuccessesorfailureswas
goal states are represented by objects of a special type
used to learn the expected probabilitiesof the two com-
self, that can belocated at some pointor insidean envi-
petingrules.
ronment(aroom)object. Themousemodelusesmeans-
Aspredictedbytheasymptoticproperties,probability
endsanalysisconcepttoachieveagoalbytakingactions
learning is badly affected by high noise (low G/τ), but
(turningandmoving)toreducethedifferencebetweenthe
the performanceincreased with the “voltage”. The later
currentandthegoalstate.
isduetothefactthatprobabilitiesP becomemoreimpor-
The model uses ACT-R’s two main learning mecha-
tantathighGvalues(evenforthesameG/τ). Thecurve
nisms: probabilitylearning(statisticallearningaboutthe
fornumberoferrorsduringstatisticallearninghasashape
usageofaparticularproductionrule)andproductioncom-
ofexponentialdecay(Figure5). Thespeedoferrorsde-
pilationmechanism(Anderson& Lebiere1998)to form
cayincreaseswith lowernoisetemperatureτ andhigher
newproductionrules.
goalvalue G. At extremelyhigh valuesof G and τ and
The main room the mouse is initially placed in has
whentheratioG/τ isnotveryhigh(significantnoise)the
two exits, and the choice of oneof two objects is repre-
Figure 5: Model performancein the first experimenton Figure6:Modeldataforthesecondexperimentwithran-
thespeedofstatisticallearning.Theerrorcurveshaveex- domdoorsorderandproductionruleslearning.Theerror
ponentialdecayshapewiththefastestdecayformedium curvesdecayfasterwhenthenewlylearnedrulesbecome
(“M”)stimulation.SeeFigure3formoredescription. stronger(abreak-point). See Figure3 formoredescrip-
tion.
performancebecomesmorevaried(higherstandarddevi-
5 Dynamics of motivation and ran-
ation)andtheaverageatleastdoesnotbecomeanybetter.
In the second experimentthe model was tested with domness during problem solving
theproductioncompilationmechanismandarandomor-
dering of the doors. The resulting curves have slightly Inthissectiontheprobabilisticinterpretationofexpected
different shapes with a distinguished “break point” and gainE will beintroducedand ACT-R conflictresolution
slope. This is due to the fact that statistical probability willbeconsideredasaprobabilitymaximisationproblem.
learning for the initial two rules does not produce use- WeshalldiscussthedynamicsoftheGandτ parameters
ful results since the doors are changed randomly. The and its analogy with emotions experiencedduring prob-
breakpointcorrespondstothemomentwhenthelearned lemsolving.
newrules,includinginformationaboutthefeaturesofthe
doors, start being used. Initially, the newly formed pro-
5.1 Parameters game
duction rules have a small strength, but their chance to
winthecompetitioninconflictresolutionincreasesevery Thedancing-mousemodelshowedthattheYerkes-Dodson
time theruleis relearned. Themoreoftenaruleis used Inverted-Ucurveeffectcanbeobservedoncognitivemod-
the more information about its eventual success or fail- elsandthatwecouldmodeldifferentarousalandmotiva-
ureisbeingcollectedandthemoreimportantbecomesits tionalstatesusingthegoalvalueandnoiseparametersin
probabilityP.Thiswayamouselearnstoavoidtheblack ACT-R. Butweknowthatarousal,motivationandconfi-
door. dence may changeduring the problemsolving when we
Nevertheless, the effect of G and τ on conflict reso- experienceemotionssuchasfrustrationorjoy.
lution is still important since all the rules satisfying the Itwas proposedearlierin (Belavkinet al. 1999)that
choicegoalareintheconflictset. Thecurvesforadiffer- thegoalvalue G andnoisetemperatureτ shouldnotre-
entstrengthofstimuliareshownonFigure6. mainconstant,whichwasmoreobviousaftersomeexper-
Thehigherdegradationofperformancecouldbebet- imentswiththeTowerofNottinghammodel. Anattempt
terobservedifweassumedthattheratioofG/τ decayed oftheauthorstomodel3–4yearsoldchildrenbehaviour
athigh“voltage”values. Recallthatthisratiocanbere- bysimplefurtherincreaseofnoisedidnotleadtothede-
latedtoconfidence,whichobviouslydidnotgrowunder sired result: themodelcouldnotsolvetheproblemany-
the fear of a strong electrical shock. In addition, there more and sometimes it ran for several simulated hours,
mightbeotherfactorssuchasmemorylossorperception whichobviouslywasveryfarfromthebehaviourof3–4
damage affecting the perfomance under strong stimula- yearsoldchildren.Areasonablequestiontoaskwaswhy
tion. themodeldidnotabandonthetask?
As was mentioned earlier, the parameter indicating
how long it is planned to spend on the task is the goal
valueGanditsdefaultvalueis20minutes. Butin ACT-
R by default its value remains constant throughout the whichcorrespondstotheformofequation(1)fortheex-
task,1 whichmeansthatafterunsuccessfultwohoursthe pectedgain.
modelis still readyto spendanother20 minutessolving We can see now that G and C in (1) represent the
a problem. Obviously, this is not reflecting reality. The “time”componentoftheaprioriprobabilityP(x,t | Y)
goalvalueshouldbeseenasthemaximumamountofre- and choosing a rule by maximum expected gain means
sources (not necessarily time) that the problemsolver is choosing by the maximum a priori probability of rules
readytoallocateatthecurrentcycle,butitmaynot,and andtheircostsforaparticulargoalstate. TheratioG/C
perhaps,shouldnotremainconstant. isproportionaltothemarginalprobabilityP(C ≤G|Y)
and if it is not 0, then by increasing G we increase the
5.2 Expected gainand probability chancetofindthesolution.
Letusconsideraproblemsolver,thathasasetofrandom
5.3 Climbingthe probabilityhill
decisionsx = {X ,...,X } and it is supposedto find a
0 n
solutionatoneofthetimemomentst={T ,...,G}. We The values of the a priori probabilities P for different
0
use Ghereforcompatibilitywith ACT-R notationand it rulesandtimemomentscanberepresentedona3Dgraph.
isthe“dead-line”bywhichweplantogettheanswer. For a better illustration, we may consider a probability
If a problem is solvable in principle, then it means densitysurfaceϕ (x,t)oncontinuousdecisions-timeplane
Y
thatinitiallyaproblemsolverhasenoughknowledgeand foragoalstateY (Figure7).Thepoints(x,t)onthissur-
means to interact with the task to be able reachthe goal facecorrespondto expectedprobabilitiesP and costs C
stateandgiventhesameproblemagainitshouldbeable ofproductionrulesinACT-R. Thisinformationislearned
torepeatthisprocess. Inotherwords,ifagoalisachiev- and updated during problem solving. Finding the max-
able, thenit meansthatthereexistsat leastonedecision imum on this probability surface is a hill-climbing task
X ∈ x, such that applying it to the problem at the ini- withGdeterminingtheangle,ordirectionofthesearch.
tialtimemomentwillleadeventuallytothegoalstateat A solution for the direction (valueof G) is given by the
a moment C ∈ t (again, we use C according to ACT-R maximum-gradientmethod,andideallyitshouldchange
notationforthecost). towardsthe maximumincline. But the maximumgradi-
Letusdenotebyy somestatesintheenvironmentor ent method does not necessarily give a unique solution
workingmemory(problemspace)andletY ∈ y besuch andtheinitialdirection.
a state, that satisfies the goal criteria (in other words, Y
is thegoalstate). Wecan thinkofY asanevaluationof
somefunctionatthe(X,C)point:
Y =f(X,C,ξ ,...,ξ ),
1 n
where ξ are some random variables (unknown parame-
i
ters).
Sincethefunctionf maybenotknownandtheremay
besomeunknownparameters,wemayconsideraproba-
bilityP(X,C |Y)ofthedecisionXandtimemomentC
forthedesiredoutcomeY.2 Thisprobabilityisapriori,
andP (X,C |Y)= 1isaposterioriprobabilityifthe
out
goal state Y has been achieved. If decisionsx and time
momentstareconditionallyindependent,then
P (X,C |Y) = P (X |Y)P (C |Y)
out out out
= P (X |Y)=P (C |Y)=1, Figure7:3Drepresentationofhill-climbinganalogy.The
out out
surface ϕ is the probability density of decisions x and
wherePout(X | Y)andPout(C | Y)aremarginalprob- timemomentstfor agoalstate Y. Theline (P,C) rep-
abilities. SinceG≥ C forC ∈ {T0,...,G}wecanwrite resents currentinformationaboutPs and Cs of thepro-
thefollowinginequality: duction rules. The line t = Gx representsthe direction
orareaofsearchforthegivenGvalue.Threelinesrepre-
G
P (X |Y) −1≥0. sent differentcases: e0 whenG = constantthroughout
out
C thetask;e+whenGvariestowardsthemaximumincline;
or e−whenGvariesfromtheincline.
P (X |Y)G−C ≥0,
out
ItisnothardtonoticethatlowGcorrespondstobreadth-
1Herewedonotconsidersubgoalsandtheirvalues.
first search. Indeed, accordingto asymptotic properties,
2HereP isnottheexpectedprobabilityofaruleinACT-R,because
itconsidersbothrulesandtimemoments low G gives most priority to rules with low costs (less
time needed), and it means that more production rules The relation of positive and negative emotions with
can be tried. On the other hand, high G correspondsto highorlowvalueofG/τ ratiocorrelatesverywellwith
depth-firstsearchsinceitgivesprioritytoruleswithhigh another observation that subjects tend to overestimate a
expected probabilities P not considering their cost very successandunderestimateafailureinagoodmood,while
much (problem solver may spend more time executing beingmorescepticalinabadmood(Johnson&Tversky
onerulewithhighercost). So,problemsolvingwithlow 1983) (Nygren, Isen, Taylor & Dulin 1996). Indeed, at
motivationcorrespondstobreadth-firstsearch,whilehigh highvaluesofGandlowτ theexpectedgain(1)depends
motivation to depth-first search. A search method com- entirely on the statistically learned expected probability
bining these two strategies is known as best-first search P, which may not necessarily reflect the real situation.
(from breadth to depth), and it suggests that G (motiva- ThereversesituationoccursatlowGandhighτ whenthe
tion) shouldgraduallyincreaseduringtheproblemsolv- learnedchoicedoesnotdependonP, althoughits value
ingapproachingagoal. mayperhapsbe1.
5.4 Annealinganalogy
6 Conclusion and open questions
Onthecontrary,noisetemperatureτ shouldbehighinthe
In this work we tried to apply moderncognitivescience
initialstateofproblemsolvingmakingtheprocessmore
andmodellingmethodstothesubjectrelatingintelligence
random.Thiscorrespondswelltothesituation,whensub-
andemotion. Thesubjectishighlyspeculativeandthere
jects start solving a task, such as Tower of Nottingham,
are still many questions with no answers. For example,
when they are just playing with blocks and trying some
howtoevaluateasuccessorfailureduringproblemsolv-
simple constructions. When more information becomes
ing and how to implementit in cognitivearchitectureto
known about the task, subjects become more confident,
make the emotional changes in information processing
whichcorrespondstohigherG/τ ratioandlowernoise.
automatic?Ifmotivationcontrolplaystheroleofresource
Now,lookingattheBoltzmannequation(2),wemay
management,whatshouldbetheuniformmeasureforthe
noticethatsuch heuristicsforcontrollingG andτ isex-
resourcesofanartificialintelligencewithemotions?
actly the same as optimisation by simulated annealing
Thesequestionshopefullywillbeansweredinthefore-
(Kirkpatrick, Gelatt & Vecchi 1983) (with E playing a
seeablefuture,butnowletustrytosummarisetheresults
roleofnegativeenergy).
ofthiswork.Threemainconclusioncanbe:
5.5 Emotionandheuristics 1. Itispossible, andwehaveshownhow,tomodela
rangeofpsychologicalphenomena,relatedtoemo-
Thisobservationsuggeststhatanemotionalproblemsolver
tionusingcognitivearchitecturessuchasACT-R.
ingeneralfollowsaverypowerfuloptimisationmethods:
2. Ithasbeenshownthatemotionandaffectshouldbe
• Positiveemotions,experiencedonsuccessesduring
takenintoaccountbycognitivemodellerssincethe
problem solving, are accompanied by increase of
motivational and emotional changes during prob-
themotivation(goalvalueG)andconfidence(G/τ
lemsolvingmayproducenoticeableeffectsinper-
ratio).Thisprocesscorrespondstocoolingthesys-
formance.
teminthesimulatedannealingandagoalstatecor-
respondstocrystallisation. 3. It has been shown that in general emotion makes
a positive contribution to problem solving, since
• Negative emotions correspond to a decrease in G
it implements powerful heuristic methods already
and G/τ (heating the system up). Negative emo-
knownin AIandmathematics, andhenceitisim-
tions occurring during problem solving can play
portantforintelligence.
a positive role in overcoming possible problems.
Ina“hill-climbing”illustrationtheseproblemsare
knownas localmaximum, plateauandridge. The Acknowledgements
strategiesusedtoovercometheseproblemsarethe
changeofdirectionofthesearch(Gdecrease)and IwouldliketothankDavidElliman,DavidWood,Frank
random jump (noise temperature τ increase). In RitterandBrianLoganforhelpfuldiscussionsandsuper-
simulated annealing it corresponds to melting the vision of this work. This work is sponsored by EPSRC
systemfromaglassstate. CreditandORSAwardScheme.
Theamountofoverallactivationbeforeexperiencing
success or failure may determinethe strength of the ex-
periencedemotion. Forexample,frustrationcorresponds
tonegativeemotionwithlowactivation,whileanxietyto
negativewithhighactivation.
References
the CognitiveScienceSociety’, LawrenceErlbaum
Associates,Hillsdale,NJ,pp.635–640.
Anderson,J.R.(1993),RulesoftheMind,LawrenceErl-
baumAssociates,Hillsdale,NJ. LeDoux, J. E. (1996), The Emotional Brain, Simon &
Schuster,NewYork.
Anderson,J. R. & Lebiere,C. (1998),TheAtomicCom-
ponent of Thought, Lawrence Erlbaum Associates, Myers, J. L., Fort, J. G., Katz, L. & Suydam, M. M.
Mahwah,NJ,London. (1963),‘Differentialmonetarygainsandlossesand
eventprobabilityinatwo-choicesituation’,Journal
Baxter,G.D.&Ritter,F.E.(1996),Designingabstractvi- ofExperimentalPsychology77,453–359.
sualperceptualandmotoractioncapabilitiesforuse
by cognitive models, Technical Report 36, ESRC Newell, A. (1990), Unified Theories of Cognition, Har-
CREDIT,Psychology,U.ofNottingham. vardUniversityPress,Cambridge,Massachusetts.
Belavkin, R. V. (1999),Annualreport: studyingtherole Nygren,T.E.,Isen,A.M.,Taylor,P.J.&Dulin,J.(1996),
of emotions in problem solving with cognitive ar- ‘Theinfluenceofpositiveaffectonthedecisionrule
chitectures, Technical report, School of Computer inrisksituations’,OrganizationalBehaviorandHu-
ScienceandInformationTechnology,Universityof manDecisionProcesses66,59–72.
Nottingham.
Ortony,A., Clore, G.L. &Collins, A. (1988),TheCog-
Belavkin, R. V. & Ritter, F. E. (2000), Adding a theory nitiveStructureofEmotions,CambridgeUniversity
of motivation to ACT-R, in ‘Seventh annual ACT- Press,Cambridge,MA.
R workshop proceedings’, Carnegie Mellon Uni-
Picard, R. W. (1997), Affective Computing, MIT Press,
versity, Department of Psychology, Pittsburg, PA
Cambridge,Massachusetts.London,England.
15213,pp.133–139.
Plutchik,R.(1994),ThePsychologyandBiologyofEmo-
Belavkin,R.V.,Ritter,F.E.&Elliman,D.G.(1999),To-
tion,1edn,HarperCollinsCollegePublishers,New
wards including simple emotionsin a cognitive ar-
York.
chitectureinordertofitchildren’sbehaviourbetter,
in N. Mahwah, ed., ‘Proceedingsof the1999Con-
Roseman, I. J., Antoniou, A. A. & Jose, P. E. (1996),
ferenceoftheCognitiveScienceSociety’,Lawrence
‘Appraisal determinantsof emotions: Constructing
Erlbaum,p.784. amoreaccurateandcomprehensivetheory’,Cogni-
tionandEmotion10(3),241–277.
Damasio,A.R.(1994),Descartes’Error: Emotion,Rea-
son, and the Human Brain, Gosset/Putnam Press,
Sloman, A. (1999), ‘Review of affective comuting’, AI
NewYork,NY.
Magazinepp.127–133.
Friedman,M.P.,Burke,C.J.,Cole,M.,Keller,L.,Mill-
Yerkes, R. M. & Dodson, J. D. (1908), ‘The relation of
ward, R. B. & Estes, W. K. (1964), Two-choice
strengthofstimulustorapidityofhabitformation’,
behaviour under extended training with shifting
Journal of Comparative and Neurology and Psy-
probabilities of reinforcement, in R. C. Atkinson,
chology18,459–482.
ed.,‘Studiesinmathematicalpsychology’,Stanford
UniversityPress,Stanford,CA,pp.250–316.
Goleman, D. (1995), Emotional Intelligence, Bantam
Books,NewYork.
Johnson,E.&Tversky,A.(1983),‘Affect,generalization,
and the perception of risk’, Journal of Personality
andSocialPsychology45,20–31.
Jones, G., Ritter, F. E. & Wood, D. J. (2000), ‘Using a
cognitive architecture to examine what develops’,
PsychologicalScience11(2),1–8.
Kirkpatrick, S., Gelatt, C. D. & Vecchi, J. M. P.
(1983),‘Optimizationbysimulatedannealing’,Sci-
ence220(4598),671–680.
Lebiere, C. & Anderson, J. R. (1993), A connectionist
implementationoftheACT-Rproductionsystem,in
‘ProceedingsoftheFifteenthAnnualConferenceof