Visualizing Cognitive Moves for Assessing Information Perception Biases in Decision Making

0.43 MB·

by  Antony W. Iorio

Visualizing Cognitive Moves for Assessing
Information Perception Biases in Decision Making
Antony W. Iorio
University of New South Wales, ADFA, Canberra ACT 2600, Australia. [email protected]
Hussein A. Abbass
University of New South Wales, ADFA, Canberra ACT 2600, Australia. [email protected]
Svetoslav Gaidow
Defence Science and Technology Organisation, Edinburgh SA 5111, Australia. [email protected]
4
Axel Bender
1
0 Defence Science and Technology Organisation, Edinburgh SA 5111, Australia. [email protected]
2

n
a
J Abstract—In decision making a key source of uncertainty is input information to an internal representation of context.
people’s perception of information which is influenced by their To facilitate the transmission of complex high-dimensional
8
attitudes toward risk. Both, perception of information and risk
2 information, e.g. in a military battle space, one can use a
attitude, affect the interpretation of information and hence the
CommonOperatingPicture(COP).Here,myriadsofpiecesof
choiceofsuitablecoursesofactioninavarietyofcontextsranging
C] from project planningtomilitary operations. Visualization asso- sensor data are fused and get representedin a richer language
ciated with thedynamics of cognitive states of peopleprocessing than that used to describe the raw information.This language
H
informationandmakingdecisionisthereforenotonlyimportant canbeavisualrepresentationofthesituation,e.g.amapwith
. for analysis but has also significant practical applications, in
s a heat diagram of activities; a graphical network with arcs
c particular in the military command and control domain. In representing recordedcommunicationamong nodes (entities);
[ this paper, we focus on a major concept that affect human
cognitioninthiscontext:reliabilityofinformation.Weintroduce or a higher-order logic or a system of equations that derive

1 Cognitive Move Diagrams (CMD)—asimplevisualization tool— fromthe fusedsensory informationthe trajectoriesof a group
v to represent and evaluate the impact of this concept on decision of unmanned aerial vehicles. In short, a COP compresses the
3 making. We demonstrate through both a hypothetical example
sensoryinformationtoarepresentationofrichersemanticsthat
9 and a subject matter expert based experiment that CMD are
can easily be communicatedto a decision maker. CMD are in
1 effective in visualizing, detecting and qualifying human biases.
7 effect COP of the dynamics of mental contents in a group.
. I. INTRODUCTION Theyfacilitatetheunderstandingofhowsmallchanges—such
1
0 Thefieldofcognitionisconcernedwithmentalcontentsand as introducing a new concept to a scenario/situation—affect
4 theprocessesbywhichthesecontentsaremanipulated.Achal- consentordissentofagroupwithrespecttotheinterpretation
1 lengefacingthecognitivesciencesisthatmostmentalcontents of the situation. In some sense, CMD is a graphical repre-
:
v and associated mental processes are invisible. Attempts are sentation of the dynamics of the mental content of a group
i thusbeingmadetoilluminatethisblackboxofhumanmental as a function of physical and/or psychologicalstimuli/affects.
X
processes by way of statistical inferencing supported by data However,as will be illustrated in the remainder of this paper,
r
a from neuro-physiologicalmonitoring. CMD can have a much wider application than that.
Visualization is normally used to represent neuro-physiolo- As mentionedabove,researchersneed to measure the inac-
gicaldatasuchassignalsofbrainactivity;datageneratedfrom cessible mental processes of interest through some proxies or
statistical analysis such as a scatter diagram with a line fit of a set of dependent/outcome variables of an experiment or a
some stimuli categoriesand the selected action;or participant natural situation. Let us define the vector C~t =[c~t1,...,c~tN]
data such as a graph of how a group of children organizes a ofoutcomevariables(vectors)aboutwhichwecollectedsome
setofconcepts.However,visualizationhasa lotmoreto offer measurements V~t at time t. These variables are proxies for
to the cognitive sciences. Simple visualization tools can pro- the concepts of interest. c~ti, i = 1,...,N is the vector
vide powerful, self-explanatory messages during an analysis of outcome i and has dimension M, with M the number of
phase. In this paper, our aim is to visually represent dynamic agents/humans.Inotherwords,foreachoutcomevariable,we
cognitiveelements, contents, and/or processes of a group. We havereadingsforallM agents.V~ isamatrixofmeasurements
hence develop a type of diagrammatic representation that we and has a dimension of M × K, with K the number of
call Cognitive Move Diagrams (CMD). independent variables used to manipulate the experimental
setup. Manipulation occurs through varying the experimental
II. COGNITIVE MOVEDIAGRAMS conditions(i.e.scenarios)atdifferenttime stepst=1,...,T,
Situational awareness is a cognitive process that char- withT representingthenumberoftimesthescenarioschange
acterizes how, in a conscious manner, an agent translates in the whole experiment.The independentvariables represent
thestimuliusedtodrivecognitivedynamicsintheM different centage of words recalled and the percentage of successful
agents. associations of words with concepts.
We now introduce the following definitions:
• Agent Cognitive State (ACS) The cognitive state ~stj A. Outcome Variables and Measurements:
for agent j at time t is defined by the outcome variable
measurement vector~st =[ct ,...,ct ]. C~0 =[[0.20 0.25 0.45][0.50 0.50 0.70]]
j 1j Nj
• Agent Cognitive Move (ACM) When the measurement C~1 =[[0.70 0.75 0.70][0.70 0.70 0.75]]
vector for agent j changes because of a manipulation of C~2 =[[0.25 0.50 0.75][1.00 0.60 0.55]]
the scenario at time step t, the transition from the old V~0 =[0 1 2]; V~1 =[0 1 2]; V~2 =[0 1 2]
ACS, ~st, to the new ACS, ~st+1, is an ACM ∆~st.
j j j
• GroupCognitiveState(GCS)ThecognitivestateS~t for
j B. Agent Cognitive States:
a group of agents j = 1,...,M at time t is defined by
a functional mapping from ACS to a matrix of similar ~s0 =[0.20 0.50]; ~s0 =[0.25 0.50]; ~s0 =[0.45 0.70]
1 2 3
dimension, GCS : [~stj → S~jt]. An example of this ~s11 =[0.70 0.70]; ~s12 =[0.75 0.70]; ~s13 =[0.70 0.75]
mappingcansimplybeafunctionthatclusterstheagents ~s2 =[0.25 1.00]; ~s2 =[0.50 0.60]; ~s2 =[0.75 0.55]
1 2 3
togetherand replacesthe value of each outcomevariable
with a cluster variable (e.g. centroid). Therefore, agents
falling in the same cluster at time t will have identical C. Agent Cognitive Moves:
values; i.e., St = St if and only if both j and l fall
ij il
in the same cluster; that is, similar(j,l) = yes, where
similaristhesimilarityfunctionusedfortheclustering.
Agent 1 = [0.20 0.50]→[0.70 0.70]and
• Group Cognitive Move (GCM) GCM∆S~t : S~t →
St+1; thus a GCM ∆S~t is the transition from the GCS [0.70 0.70]→[0.25 1.00]
at one time step to the next time step. Agent 2 = [0.25 0.50]→[0.75 0.70]and
• A Cognitive Move Diagram (CMD) is a representation [0.75 0.70]→[0.50 0.60]
of the state transition diagram of GCM using some
Agent 3 = [0.45 0.70]→[0.70 0.75]and
encoding scheme.
[0.70 0.75]→[0.75 0.55]
III. EXAMPLE
D. Group Cognitive Moves:
Weillustratethepreviousdefinitionsinahypotheticalexam-
ple.ThisexampledeviatesfromourprimaryuseofCMD, the Assume we used some clustering algorithm for each
visualizationofhumanbiasesrelatedtoinformationreliability weaponsnoiseexposuretimeinterval.Thealgorithmfort=0
andtemporaldependenceofdecision,soastodemonstratethat assignedthefirsttwoagentstothesameclusterwithacentroid
thevisualizationcanbegeneralizedandofbenefittothewider of [0.23 0.50] and the third agent to one cluster described by
cognitive sciences literature. its original vector. For t = 1 all agents were in one and the
In a military context, imagine we are interested to under- same cluster with a centroid of [0.72 0.72]. For the last time
stand the impact of weapons noise on memory. Let’s assume interval,eachagentoccupiedadifferentcluster.Therefore,the
M =3 individualsparticipated in a within-groupexperiment. GCS is as follows:
Eachexperimentinvolvedmemorytestingthatwasperformed S0 =[0.23 0.50]; S0 =[0.23 0.50]; S0 =[0.45 0.70]
1 2 3
on three days, each separated by a week. For each individual, S1 =[0.72 0.72]; S1 =[0.72 0.72]; S1 =[0.72 0.72]
1 2 3
the sequencein which the experimentalconditionsvariedwas S2 =[0.25 1.00]; S2 =[0.50 0.60]; S2 =[0.75 0.55]
1 2 3
chosen randomly. Twenty words and their meanings were
The clustering algorithmsimply revealedthat froma group
presented to the subjects in a random order. Two hours later,
(similarity measure) perspective all agents were in approxi-
they were first asked to recall as many words as they could.
mately the same state when exposed to gun fire noise for an
They were then again presented with the same twenty words
hour. The GCM is as follows:
and were asked to cluster them according to meaning. The
conditions on the test days varied such that on one of the
threedays,theindividualswerenotexposedtoweaponsnoise
Agent 1 = [0.23 0.50]→[0.72 0.72]and
withinthesetwohours,ontheothertwodays,thesubjectwas
exposedtoaone-hourandtwo-hourweaponsnoiseduringthe [0.72 0.72]→[0.25 1.00]
two hours between being told the words and being tested. Agent 2 = [0.23 0.50]→[0.72 0.72]and
In this experiment, there is K = 1 ordinal independent
[0.72 0.72]→[0.50 0.60]
variable—the length of time during which a subject was
Agent 3 = [0.45 0.70]→[0.72 0.72]and
exposed to the sound of a gun fire, i.e. t ∈ {0,1,2} (hours).
N = 2 dependent variables were ratio measured: the per- [0.72 0.72]→[0.75 0.55]
Fig.1. CMDforthememoryexample usingEncodingScheme1. Fig.2. CMDforthememoryexampleusingEncodingScheme2.
IV. ENCODING SCHEMES
We present three possible ways to represent GCM in a
CMD.Thefirstencodingschemereliesonassociatingaunique
symbol to each cluster. In our hypothetical example, there
are seven possible unique clusters. Thus, we associate an
alphabetic letter to each cluster according to the following
rule: a cluster that contains only one of the three individuals
is marked by letter ‘a’, ‘b’, and ‘c’ for individual 1, 2, and
3, respectively. A cluster of two individuals is labelled ‘d’,
‘e’, or ‘f’ when containing individuals (1,2), (1,3), or (2,3),
respectively. When all individuals fall into a single cluster, it
is assigned the letter ‘g’. The resulting CMD is depicted in
Fig.3. CMDforthememoryexampleusingEncodingScheme3.
Figure 1.
Encoding Scheme 1 has the advantage that participants
are uniquely associated with clusters. In its visualization, the V. SCALABILITY ISSUES
arrows that link clusters for different noise exposure times
The visualization of GCM in CMD is restricted to two or
offerbotha glimpseofthedynamicsoftheGCS andthestart
threedimensions.Thisisnotnecessarilyalimitationinexper-
and end states. For instance, one can immediately see that
iments with small numbersof outcome variables. However, it
noise exposure over a period of one hour unified the group
canbecomesignificantinourdomainofapplication—military
memory ability. One can also make out clearly that the first
informationprocessing—wherea primaryobjectiveis to have
two individuals, despite starting with and maintaining similar
a generic situational awareness visualization method that de-
memory ability, diverged dramatically when exposed to two
picts deviations from expected cognitive moves in a group of
hours of noise.
soldiers. As such, often tens of factors need to be analyzed
The disadvantage of Encoding Scheme 1 is that it does
simultaneouslyanddisplayedtoprojecta group’sbehavior.In
not scale well as the number of individuals increases. When this case, our recommendation is to group these factors and,
numbers are large, it is better to label a cluster with the
instead of visualizing the raw outcomes, use dimensionality
number of individuals contained in it. While in this second
reduction techniques such as principal component analysis.
encoding scheme information about individuals gets lost, this The visualization can then be done, for instance, for the first
informationis typically not very useful anyway when cohorts
two principal components only.
are big. Figure 2 shows a CMD with EncodingScheme 2. As
with Encoding Scheme 1 the cognitive changes of the group VI. CASE STUDY
of subjects exposed to different experimental conditions can In military command and control (C2), information relia-
be grasped quickly. bility can have a sizeable impact on the courses of action
IfthenumberofGCMisverylarge,thetransitionlineswill taken. Not knowing how reliable information is, equates to
clutter the diagram. Our third encoding scheme in Figure 3 uncertainty.Inthecontextofdecisionmakingsuchuncertainty
uses colored boxes to represent clusters. Color intensity rep- may affect the decision maker’s objectives. How much a
resentsclustersize:thedarkerthecolor,thegreaterthenumber decision is affected is likely to depend on his or her attitude
of cluster members. The boundary line of a box encodes the to risk (the effect of uncertainty on objectives).
independent experimental variable: the thicker the line, the Obviously,there exists a need to quantify how risk attitude
larger is the independent variable. affectsdecisionmaking.Theliteratureisrepletewithmethods,
models,andstudiesforassessingriskattitudeinhumans.Risk
attitude has been assessed using games, such as the Angling
RiskTask[1],theBalloonAnalogRiskTask[2],theColumbia
Card Task (CCT) [3], the CupsTask [4], and the Devil’sTask
(also known as Slovic’s Risk Task) [5]. However, these are
very specific games that situate players in specific, artificial
environments. Risk attitude though depends on context, in
particularinmilitaryC2.Similarly,therearemanymeasuresof
riskattitude,suchastheAttitudestoRisk Taking[6],Domain
SpecificRiskAttitude(DoSpeRT)[7]andtheRiskPropensity
Scale also known as the Risk Taking Index [8], etc. These
measuresaretypicallyutilizedingeneralriskassessmentsand
are rarely suited to military decision environments.
In a case study we posed the questions how much some-
one’s decision changes when they are told that the infor- Fig.4. CMDusingEncodingScheme1fortheresourceallocation problem
presented tothreemilitary subjectmatterexperts.
mation presented to them was reliable? If the concept of
information reliability did not cross their mind, how does
their behavior alter when they suddenly become aware of
general notions of individual and group cognitive states, and
it? How much does the framing of information reliability
transitionsbetweenthesestates.Thistechniqueforvisualizing
affect their decision making? We phrased these questions in
cognitive similarities and differences is robust for use in a
the context of the strategic planning of truck allocations to
rangeofexperimentswithvaryingmanipulationvariables.We
future operations. Decision makers (“agents”) were required
have presented an illustration of CMD applied to a hypothet-
to make their allocation decisions three times where the
ical case in which participants’ associative memory might be
only factor that changed from experiment to experiment was
affected by gun fire noise. We also presented results from a
the framing of the information about the reliability of truck
real-world case study involving human participants. Here, the
demands. Specifically, in the first experiment scenarios were
effects of both the framing of information reliability and a
described without making any reference to reliability of the
decision’stimehorizonwerestudiedinthecontextofstrategic
provideddemandintervalestimates; in the second experiment
defence procurement decision making. The simple hypothet-
information about maximum and minimum demands across
ical example and the real-world case study demonstrate how
the various competing scenarios was described qualitatively
decisionmakerswhoreachconclusionsindependentlyofeach
(e.g. “very high”); in the third experiment, the reliability of
other, move through a cognitive space. We illustrated how
informationwas quantified. The providedreliability estimates
similarities and differences in cognitive perception can be
constitute the manipulation variable discussed in Section I.
visualized in a CMD applying various encoding schemes.
The experiment was set up such that resources provided to
participants were sufficient to meet the expected value of the REFERENCES
demand intervals. The AGS of the decision makers reflects
[1] T. J. Pleskac, “Decision making and learning while taking sequential
theirdeviationfromexpectedvalueinthecontextofreliability risks,” Journal of Experimental Psychology: Learning, Memory, and
information. Cognition, vol.34,no.1,pp.167–185,Jan2008.
[2] C. W. Lejuez, J. P. Read, C. W. Kahler, J. B. Richards, S. E. Ramsey,
Figure 4 shows a CMD for three participants of the case G.L.Stuart,D.R.Strong,andR.A.Brown,“Evaluationofabehavioral
study experiments with the outcome measurement variables measureofrisk-taking:Theballoonanaloguerisktask(BART),”Journal
ofExperimental Psychology: Applied,vol.8,no.2,pp.75–84,2002.
being the normalized truck allocations that participants made
[3] B.Figner,R.J.Mackinlay,F.Wilkening,andE.U.Weber,“Affectiveand
in a given scenario and the deviation of their allocation from
deliberative processes in risky choice: Age differences in risk taking in
the expected value. The application of clustering mappings thecolumbiacardtask,”JournalofExperimentalPsychology:Learning,
Memory,andCognition, vol.35,no.3,pp.709–730, 2009.
resulted in clusters of single individuals only, labelled ‘a’,
[4] I. P. Levin and S. S. Hart, “Risk preferences in young children: Early
‘b’ and ‘c’ in Figure 4. From the CMD we can see at
evidence of individual differences in reaction to potential gains and
a glimpse that, firstly, the introduction of the concept of losses,”JournalofBehavioralDecisionMaking,vol.16,no.5,pp.397–
413,2003.
Information Reliability had an impact on all the decisions
[5] P. Slovic, “Risk-taking in children: Age and sex differences,” Child
madebytheparticipant.Secondly,thegraphindicatesthatthe Development, vol.37,no.1,pp.169–176, 1966.
groupof participantsdoes nothave a shared understandingof [6] R. Grol, M. Whitfield, J. D. Maeseneer, and H. Mokkink, “Attitudes to
information reliability. For instance, the transition from State risk-takinginmedicaldecision-makingamongbritish,dutchandbelgian
general-practitioners,” British Journal of General Practice, vol. 40, no.
‘a’ to another unique State ‘a’ is not shared by any other 333,pp.134–136, 1990.
participant. [7] A.-R.BlaisandE.U.Weber,“Adomain-specificrisk-taking(DOSPERT)
scaleforadultpopulations,”JudgmentandDecisionMaking,vol.1,no.1,
VII. CONCLUSION pp.33–47,2006.
[8] N.Nicholson, E.Soane,M.Fenton-O’Creevy, andP.Willman, “Person-
In this paper we have introduced and developed Cogni- ality and domain-specific risktaking,” Journal ofRiskResearch, vol. 8,
tive Move Diagrams (CMD), adapted from some simple and no.2,pp.157–176, 2005.