Valence, arousal and dominance in the EEG during game play Boris Reuderink*, Christian Mühl ...

18 Pages·2012·0.77 MB·English

Int.J.AutonomousandAdaptiveCommunicationsSystems,Vol.6,No.1,2013 45
Valence, arousal and dominance in the EEG during
game play
Boris Reuderink*, Christian Mühl and
Mannes Poel
HumanMediaInteraction,
FacultyofEEMCS,
UniversityofTwente,
P.O.Box217,
Enschede7500AE,TheNetherlands
E-mail:[email protected]
E-mail:[email protected]
E-mail:[email protected]
∗
Correspondingauthor
Abstract: In this paper, we describe our investigation of traces of naturally
occurringemotionsinelectricalbrainsignals,thatcanbeusedtobuildinterfaces
that respond to our emotional state. This study confirms a number of known
affectivecorrelatesinarealistic, uncontrolledenvironmentfortheemotionsof
valence(orpleasure),arousalanddominance:(1)asignificantdecreaseinfrontal
powerinthethetarangeisfoundforincreasinglypositivevalence,(2)asignificant
frontalincreaseinpowerinthealpharangeisassociatedwithincreasingemotional
arousal,(3)asignificantrightposteriorpowerincreaseinthedeltarangecorrelates
with increasing arousal and (4) asymmetry in power in the lower alpha bands
correlateswithself-reportedvalence.Furthermore,asymmetryinthehigheralpha
bandscorrelateswithself-reporteddominance.Theselasttwoeffectsprovidea
simplemeasureforsubjectivefeelingsofpleasureandfeelingsofcontrol.
Keywords: affective computing; affective signal processing; emotion
recognition;EEG;electroencephalogram;BCI;brain–computerinterfaces;alpha
asymmetry; frustration; valence; arousal; dominance; adaptivecommunication
systems.
Reference to this paper should be made as follows: Reuderink, B., Mühl, C.
andPoel,M.(2013)‘Valence,arousalanddominanceintheEEGduringgame
play’,Int.J.AutonomousandAdaptiveCommunicationsSystems,Vol.6,No.1,
pp.45–62.
Biographicalnotes:BorisReuderinkreceivedhisMaster’sdegreein2007,after
spendingtimeondifferentmachinelearningproblems,includingrecognitionof
hand-writtentextonenvelopesandthedetectionoflaughterinaudio–visualdata.
Brainsandintelligence–hislifelonginterest–canbecombinedhisPhDposition
at the University of Twente, in which he focuses on making brain–computer
interfacesfunctioninreal-worldsettingsforhealthyusers.
ChristianMühlreceivedhisMaster’sdegreeinCognitiveSciencesin2007atthe
UniversityofOsnabrück,Germany.Hiseducationwasfocusedonneuroscientific
methods, specifically electroencephalography. Since then he is working as a
research assistant in the Human Media Interaction Group at the University of
Copyright ©2013InderscienceEnterprisesLtd.
46 B.Reuderink,C.MühlandM.Poel
Twente,TheNetherlands.InhisPhDThesis,hesearchesforneurophysiological
correlatesofaffectivestatesinthecontextofbrain–computerinteraction.
MannesPoelisanAssistantProfessorintheHumanMediaInteractionGroupat
theUniversityofTwente.Hismainresearchinvolvesappliedmachinelearning
forvision-baseddetectionandinterpretationofhumanbehaviourandtheanalysis
andclassificationofEEG-basedbrainsignals.
1 Introduction
Abrain–computerinterface(BCI)offersadirectlinkbetweenthesignalsproducedbythe
user’sbrainandacomputer,andisaimedtraditionallyatpatientssufferingfromamytrophic
lateral sclerosis (ALS). But for healthy users, brain signals can also provide information
thatcanbeusedtoaugmenttraditionalhuman–computerinteraction(HCI).Onesourceof
information that a BCI may provide is an estimate of subjective emotions while they are
beingexperienced.
The enrichment of HCI by incorporating information about the emotional context of
theinteractionortheuser’sbehaviourisstudiedinthefieldofaffectivecomputing(Picard,
1997).Affectiveinformationenablestheapplicationtorespondmoreadequatelytotheuser,
e.g.byofferinghelpinsituationsofextremefrustration,stressormisunderstanding.Several
modalitiesareabletoconveyinformationabouttheuserstatewithoutdirectlyintrudinginto
theuser’sconsciousnessortaskathand,ase.g.theobservationofovertuserbehaviour(e.g.
facialexpressionsorprosody)(Zengetal.,2007).Anothersourceofaffectiveinformation
that is less dependent on overt behaviour is the group of physiological responses, such
ascardiovascularresponses(Frazieretal.,2004;Langetal.,1993),electrodermalactivity
(Codispotietal.,2001;Langetal.,1993)andmuscleactivity(Cacioppoetal.,1986;Magnée
etal.,2007).
Aspecificcaseofphysiologicalactivityisproducedbyneuronsintheneocortex.Similar
totheaffectiveresponsesrecordedfromtheotherpartsofthebody,thebrainactivityhas
beenstronglyassociatedwithperceptive,experientialandexpressiveemotionalprocesses
(Demareeetal.,2005).Morespecifically,cognitivetheoriesofemotionsclaimthebrainto
betheprimarysourceofaffectiveresponses,asitisprocessingtheverystimuli,memories
or thoughts that lead to an affective reaction in brain, body and behaviour (Sander et al.,
2005).Accordingtothosetheories,theuseofbrainsignalswouldenableafasterandmore
direct recognition of emotional states, and be sensitive to more subtle emotions that do
notcrossthethresholdofbodilyexpression.Asfirstdiscriminativeneuronalresponsesto
emotionalstimulationcanalreadybeobservedwithintensofmilliseconds(Aftanasetal.,
2002; Keil et al., 2001), brain activity responds faster to emotional stimulation than any
otherphysiologicalmeasure.Thus,sensorsmeasuringneuronalactivityfromthebrainoffer
arelativelynon-intrusivemethodoffastandcontinuousobservationoftheuser’saffective
statei.e.notdependentonovertbehaviour.
Severalstudieshaveshowntheprincipalcapabilitytoyieldestimatesofuserstateswith
acceptable precision via physiological (Benovoy et al., 2008; Chanel et al., 2006, 2008;
KimandAndré,2008;Kreibigetal.,2007;Liuetal.,2009;Mandryketal.,2006;Picard
et al., 2001; Rainville et al., 2006) and neurophysiological signals (Chanel et al., 2006,
Valence,arousalanddominanceintheEEGduringgameplay 47
2009;Khosrowabadietal.,2009;Koetal.,2009;Linetal.,2009;Murugappanetal.,2008;
Rizonetal.,2008;Takahashi,2004).Correlatesofaffecthavebeenstudiedviaamultitude
ofexogenousandendogenousaffectelicitationparadigms.Affectinductionprotocolshave
madeuse, amongothers, ofpictures(Aftanasetal.,2001;Husteretal.,2009;Keiletal.,
2001; Müller et al., 1999), sentences (Marosi et al., 2002), music (Sammler et al., 2007;
SchmidtandTrainor,2001),videos(Aftanasetal.,2006;Krauseetal.,2000)andrecallof
emotionalevents(Chaneletal.,2009).Theseneuralcorrelatescanbeusedinthedesignof
anautomaticclassifierofaffect. Buttoensurethegeneralisationofthefindingsfromthe
controlledcontextofthelaboratorytoareal-worldHCIcontext(ecologicalvalidity), the
affectivestateisbestelicitedinawayresemblingthecontextofapplication.
The goal of the present study is to find affective neurophysiological correlates of
frustration, valence, arousalanddominanceinanecologicallyvalidcontext, inthiscase,
game play. To evoke emotions, we introduced frustration by ignoring keyboard input
periodically. Using a non-responsive controller is a well-known method in the fields of
affectivecomputingandHCI(DienerandOertel,2006;Kleinetal.,2002;Scheireretal.,
2002).Priorstudieshaveshownthatitisinprinciplepossibletoidentifyneurophysiological
correlatesofaffectivestateselicitedbygamingevents(SalminenandRavaja,2007,2008)
andwithHCIingeneral(SpiridonandFairclough,2009).
Section 2 will introduce the reader to the relevant types of brain activity, results from
theaffectiveneurosciencesandconsequently,specifichypothesesderivedfromtheseprior
studies.
2 Emotionsandthebrain
Emotions can be studied within the framework of the circumplex model (Russell, 1980).
Thismodeldefinesatwo-dimensionalemotionalspace,spannedbyacontinuousvalence
(pleasure)andanarousaldimension.Inthisspace,allcategoricalemotionallabelscanbe
definedintermsofvalenceandarousal.Forexample,fearisassociatedwithastateofhigh
arousalandnegativevalence,whereasexcitementischaracterisedbyahigharousalanda
positivevalence.Athirddimension,nameddominance,canbeaddedtothisspacetosignify
thesubjectivefeelingofcontrol(dominantvs.submissive).Dominancecanbeunderstood
as a social or cognitive interpretation of an affective event (Russell, 1980), indexing the
“interactive relationship that exists between the perceiver and the perceived [object or
situation]’’(BradleyandLang,1994).
Bymanipulatingtheaffectofusers,differencesinthisemotionalspacecanbemapped
to specific regions of the brain. For example, processes related to emotion recognition
are associated with the right hemisphere (right hemisphere theory), whereas positive and
negativeemotionalstateshavebeenattributedtoregionsintheleftandrightfrontalcortices,
respectively (the valence theory) (Silberman and Weingartner, 1986; Tucker, 1981). An
alternativetothisvalencetheoryofemotionistheapproach/withdrawaltheory(Davidson,
1992), in which the different hemispheres are differently activated depending on the
motivationaldirectionoftheemotionalstate:lefthemisphericfrontalactivityisassociated
withapproach,whereasrighthemisphericactivityisassociatedwithwithdrawal.Asmost
approach-related emotions are associated with a positive feeling and most withdrawal-
relatedemotionswithanegativefeeling, thereisaconsiderableoverlapofboththeories.
Thedefiningdifferenceliesinthehemisphericalactivationcausedbytheemotionofanger;
whileitisanegativeemotion,itisassociatedwithanapproachbehaviour.
48 B.Reuderink,C.MühlandM.Poel
Interestingly,thedominancedimensionenablesadifferentiationbetweenapproachand
withdrawalemotionsthatcannotbedifferentiatedonthebasisofvalenceandarousalalone.
Forexample,angerisaccompaniedbyhighdominance(senseofcontrol)ratings,whereas
fear scores low on dominance (Demaree et al., 2005). This is especially important in the
current study, as the induced frustration (low dominance) is close or might lead to anger
(high dominance), which are both characterised by a negative valence and high arousal.
Whileforthevalenceandarousaldimension,responsesintheelectroencephalogram(EEG)
toaffectivestimulationhavebeenreported,fordominance,wecouldonlyfindHerazand
Frasson (2007). Heraz and Frasson used broad-band power features in the delta, theta,
alphaandbetabands,andcorrelatedcontinuousmeasurementswithself-reportedvalence,
arousal and dominance. Weak correlations with dominance were found, but it is unclear
whetherthereportedcorrelationsaresignificant,asnotallobservationswereindependent.
Inthefollowing,wewillshortlyreviewliteratureontheassociationofemotionalprocesses
andoscillatorybrainactivity.
Thepreviouslyintroducedhemisphericvalencetheoryholdsthatpositiveemotionsare
processed in the left frontal cortex, whereas negative emotions are processed in the right
frontal cortex. As cortical activation is held to be inversely related to alpha (8–12Hz)
activity(Pfurtscheller,1999),theprocessingofpositiveemotionalinformationisreflected
in a decrease of alpha power of the left frontal hemisphere, the processing of negative
emotional information is reflected by the decrease of alpha power over the right frontal
hemisphere.Thispatternwasshowninstudiesusingawidevarietyofemotioninduction
protocols (Allen et al., 2004; Davidson, 1992; Huster et al., 2009; Schmidt and Trainor,
2001).Thefrontalalphaasymmetryisthemostfrequentlyfoundcorrelateofvalence.
Manyothercorrelatesforvalenceareobservedinalessrobustmanner.Forexample,the
frontalasymmetrywasalsofoundinthethetarange(4–7Hz)inresponsetopositiveand
negativepicturecontent(Aftanasetal.,2001)andsentences(Marosietal.,2002).Müller
etal.(1999),alsopresentingpictures,didnotfindhemisphericinteractionswithvalencein
thealphafrequencyrange, butinthegammarangeatthetemporallobes. Similarly, Keil
etal.(2001)foundresponsesinthegammarangeforaversivepictures.Medialandfronto-
medialeffectsofemotionwerefoundinseveralstudies.Anincreaseinfronto-medialtheta
powerwasobservedforpositivemusicstimuli(Sammleretal.,2007)andduringmeditation
(AftanasandGolocheikine,2001),whichwasinterpretedasemotionalprocessingclosely
associated with attentional processes. Krause et al. (2000) found a power increase in the
low-thetabandovermidlineelectrodesinresponsetoangerinducingmovies, supporting
thenotionofanattention-relatedeffect.Thesameregionwasshowntobemoreactivein
terms of delta (0–3Hz) oscillations when aversive stimuli were presented (Klados et al.,
2009).
Arousalactivatesneuralstructuresingeneral,andthereforeshouldbeassociatedwitha
globaldecreaseofpowerinthealphaband(Barryetal.,2007,2009).Morelocalisedeffects
inthealphabandarefoundinSchmidtandTrainor(2001)andAftanasandGolocheikine
(2001),buttheeffectsseemtocontradicteachother;SchmidtandTrainorfoundacorrelation
between the activation of frontal cortical regions and the perceived arousal of musical
excerpts,whileAftanasetal.foundadeactivation(measuredasanincreaseinhigh-alpha
power) in the frontal regions associated with increasing arousal. Other EEG phenomena
observed at posterior sites with increasing arousal include the increase of power in low-
frequencybands,asinthedelta(Kladosetal.,2009)andthetaband(Aftanasetal.,2002),
thedecreaseofalphapower,andanincreaseinuppergammapower(Aftanasetal.,2004).
Valence,arousalanddominanceintheEEGduringgameplay 49
3 Researchquestions
BasedontheliteraturedescribedinSection2,weformulatedhypothesesontheinteraction
of affective states and the EEG to be confirmed in an ecologically valid context for an
adaptiveinterface.
First, we describe the hypotheses for the valence dimension. In the delta band, we
havethehypothesisHvδ:fronto-medialdeltapowerincreaseswithincreasingvalence.For
the theta band, we define three distinct hypotheses: Hvθ1: with increasing valence, left
hemispherical theta power increases, Hvθ2: with increasing valence, right hemispherical
thetapowerdecreasesandHvθ3:withincreasingvalence,posteriorthetapowerincreases.
Inthealphaband,wedefinethefollowinghypotheses:Hvα1:withincreasingvalence,left
hemisphericalalphapowerdecreases, Hvα2: withincreasingvalence, righthemispherical
alpha power increases. Finally, for the gamma band: Hvγ1: with increasing valence, left
temporalgammapowerdecreasesandHvγ2:withincreasingvalence,righttemporalgamma
powerincreases.
For arousal, we formulate the hypotheses Haδ: posterior delta power increases with
increasing arousal, Haθ1: posterior theta power increases with increasing arousal, Haα1:
globalalphaactivitydecreaseswithincreasingarousal,Haα2:frontalalphapowerincreases
with increasing arousal, Haβ1: posterior beta power is positively correlated with arousal
and Haγ1: high arousal correlates with a power increase in the high-gamma band. These
hypothesesaresummarisedinTable1.
Table1 Hypothesesforvalence,arousalanddominancecorrelationsinthefrequencydomain
Dimension Delta Theta Alpha Beta Gamma
Valence↑ Hvδ:fron.-med.↑Hvθ1:l-hemi.↑ Hvα1:l-hemi.↓ – Hvγ1:l-temp.↓
Hvθ2:r-hemi.↓ Hvα2:r-hemi.↑ Hvγ2:r-temp.↑
Hvθ3:fron.-med.↑
Arousal↑ Haδ:posterior↑ Haθ:posterior↑ Haα2:global↓ Haβ:parietal↑Haγ:gamma↑
Haα1:frontal↑
Dominance↑ – – – – –
4 Methods
Totesttheaboveformulatedhypotheses,thefollowingmethodologywasused:
4.1 Datacollection
ToobtainEEGdatacontainingdifferentemotionalstates,weusedagamedesignedtoinduce
frustrationforshortperiodsbyignoring15%ofthekeyboardinput(Reuderinketal.,2009).
Intheblockswherefrustrationisinduced,thescreenperiodicallylagsaswelltosimulate
an underpowered computer. The experiment runs through a sequence built from random
permutationsoftwonormal,andonefrustrationblockof2mineach.Aftereachblock,the
userisaskedtoratehiscurrentmentalstateontheemotionaldimensionvalence,arousal
anddominancebypressinganumerickeydisplayedbelowtheSelf-AssessmentManikin
(SAM)(BradleyandLang,1994),seeFigure1.
50 B.Reuderink,C.MühlandM.Poel
Figure1 Thegame,andtheself-assessmentscreenforvalence,arousalanddominance
(seeonlineversionforcolours)
Note:In our analysis, we inverted the scale of valence and arousal, so that high valence
correspondedtopositiveness,andhigharousaltoamorearousedstate.
4.1.1 Experimentalprocedure
Users were asked to read and sign a form of consent, and were subsequently wired with
theEEGandphysiologicalsensors.Theexperimenterbrieflyexplainedthegameandthe
self-assessmentprocedure.Theparticipantwasallowedtopractisethecontrolsfor2min,
andthentheexperimentwasstarted.Whenusersmentionedthatthegamewasunresponsive
duringtheexperiment, theexperimenteraskedthemtocontinueplayingandpromisedto
findthesourcelater.After30min,theexperimenterstoppedtheexperimentandtheusers
weredebriefed.
4.2 Subjects
Twelve healthy users (age 27±3.9) participated in the experiment. All participants had
normal, or corrected to normal vision, and reported no use of medication. Only three of
oursubjectswerefemale,andallsubjectswereright-handed.Mostparticipantshadsome
experiencewithvideogames.
4.2.1 Sensorsandrecording
ABioSemiActiveTwoEEGsystemwasusedtorecordtheEEGandphysiologicalsignals
atasamplerateof512Hz.ForEEG,32Ag/AgClactiveelectrodeswereused,placedatthe
Valence,arousalanddominanceintheEEGduringgameplay 51
locationsoftheExtendedInternational10–20system.Tomeasuretheinfluenceofocular
andmuscularartefacts,werecordedtheEOG(horizontalandverticalpairs)andtwopairs
ofEMGsignalsforthefingermovementusedtocontrolthegame.Additionalphysiological
sensors,suchastemperature,respiration,thegalvanicskinresponseandthebloodvolume
pulsewererecordedaswell,butnotusedinthepresentstudy.
4.3 Preprocessing
RawEEGdatacancontainnoisefromtheenvironment,eyemovementsormuscletension.
Weappliedthefollowingpreprocessingproceduretoreducetheinfluenceofthesesignals.
First, the recording was decimated to 128Hz. After decimating, the channels were high-
passfilteredusingafourthorderButterworthfiltertoremovefrequenciesbelow0.2Hz,and
notch-filteredusingafourthorderButterworthfilterfrom49to51Hztoremovepowerline
noise.TheEEGwassubsequentlycorrectedforeyemovementsusingthe(linear)regression
analysissubtractionmethodofSchlögletal.(2007)basedonthecovariancebetweenEEG
andEOGchannels.OnlyaftertheremovaloftheEOGinfluence,thedatawasre-referenced
tothecommonaveragereference(CAR),thatsubtractstheaveragesignalfromeachsensor
ateachtimepoint.
4.4 EEGfeatures
Alpha asymmetries were calculated for lateral sensor pairs, using the procedure outlined
inAllenetal.(2004):foreachexperimentalblock,Welch’smethodwasusedtoestimate
power in different frequencies for each EEG-channel with 1Hz resolution. Within each
block,wesummedthe(natural)log-powerinthealphaband(8–12Hz)foreachelectrode,
and subtracted the band power of electrodes on the left hemisphere from corresponding
electrodes on the right hemisphere. This results in an alpha-asymmetry index for each
sensorpairforeachexperimentalblock,whichshouldcorrelatepositivelywithvalence.
The same spectral estimates for each experimental block were used to calculate the
correlationsbetweenlog-powerofnarrow-bandEEGoscillationsandSAMratings. Note
thatwehavedeliberatelychosenforananalysisusingspectralbinsof1Hz,astheemotional
response can differ for small differences in frequency bands (Krause et al., 2000). The
drawbackofthischoiceisthatwehavetoperformmorestatisticaltests,andthattherefore
theeffectshavetobequitestrongtopassthemultiple-testcorrection.
4.5 Statisticaltests
Forthealphaasymmetry,wewillsimplyreportcorrelationsofthealphaasymmetryindex
ofdifferentsensorpairs,withtheself-reportedratingsforvalence,arousalanddominance.
StatisticalsignificanceisdeterminedwithaWilcoxonsigned-ranktestoverthecorrelations
ofasensorpairswithemotionaldimensions.Wehavechosennottolimitoursignificance
tests to only correlations with valence, but to include the frustration condition, arousal
anddominanceaswell.Theseextratestsaremeanttobeillustrative;thereforewedonot
performmultipletestcorrectiononthealpha-asymmetrycorrelations.
For the narrow-band oscillations, we do perform multiple test correction using
a combination of Bonferroni correction and Fisher’s method. Bonferroni correction
conservatively corrects for performing multiple tests by dividing the critical value α by
52 B.Reuderink,C.MühlandM.Poel
thenumberoftestsperformed.Butwhenalargenumberoftestsisused,itsconservative
formulationresultsincriticalαvaluesthatcannotbepassedbyindividualstatisticaltests.
This problem is well-known in neuroscience, and there are various statistical methods to
overcomethisproblem,e.g.bycombiningevidenceoversubjects(Lazaretal.,2002).
One method to combine evidence is to use the distribution of the p-values associated
withmultiplestatisticaltestsinsteadofusingthedistributionofthecorrelationcoefficients
ρ. Fisher’s method is suggested by Loughin for testing distributions of p-values for
significance:
(cid:2)k
X2 =−2 logepi (1)
i=1
where pi is the p-value for subject i. When all the null hypotheses are true, and all the
p-valuesareindependent,X2followsaχ2distributionwith2kdegreesoffreedom.
In our experiment, we have k = 12 independent subjects, therefore we compare the
X2 ofthe12p-valuestoaχ2 distributionwith24degreesoffreedom.Notethatwenow
combinethedifferentp-valuesfromdifferentsubjectsintoacombinednullhypothesisH ,
0
that states that each of the individual null-hypothesis is true. The combined alternative,
HA, is that at least one is not true (Loughin, 2004). While it is tempting to use Fisher’s
methods not only over subjects, but also within the frequency band, we cannot because
theobservationswithinafrequencybandarenotindependent.Bonferronidoesnotrequire
independent p-values, and is therefore used to correct for the number of tests within a
specificfrequencyband.
To summarise: for the 12 subjects, the correlation coefficient ρ and it’s associated
p-valueofthebandpowerataspecificsensorwithaemotiondimensionwascalculated.
Within a frequency band (delta: 0–3Hz, theta: 4–7Hz, alpha: 8–11Hz, beta: 12–29Hz
and gamma: 30+Hz), the one-sided p-values were combined over the 12 subjects using
Fisher’smethod,andthesecombinedp-valuesweresubsequentlycorrectedforthenumber
oftestswithintheband(α = 0.05/(2×32×b),wherebisthenumberof1Hzbandsin
thefrequencyband).
5 Results
5.1 Self-assessments
For each user, we have approximately 15 different measures of subjective valence,
arousal and dominance. These self-assessments were rescaled to unit intervals for easy
interpretation.Mostsubjectsratedthefrustrationinducingconditionlesspositivethanthe
normalcondition,oversubjectsthisdifferenceissignificant(T =3,p <0.01).Whilewe
expectedtofindatrendtowardsmorearousalinthefrustrationcondition,thereappearsto
benodifference(T = 23,p = 0.26).Thedominancescaleindicatesthatpeopleseemto
besignificantly(T =4,p <0.01)moreincontrolinthenormalcondition.
Individualratingsforthevalenceanddominancedimensionsshowspreadvalueswith
generally higher ratings for both valence and dominance in the frustration condition
(Figure 2). Users enjoyed the game; therefore the negative emotions are slightly
underrepresented.
Valence,arousalanddominanceintheEEGduringgameplay 53
Figure2 Individualvalenceanddominanceratingsforthenormal(bluecircles)andfrustration
(redsquares)condition(seeonlineversionforcolours)
Note:Formostsubjects,theratingsarenicelyspreadwithhigherdominanceandvalencefor
theratingsfortheblockswithfrustrationinduction.
Table2 Thecorrelationamongexperimentalblocks(time), condition(frustrationinduction),
valence,arousalanddominanceoftheself-assessments
Time Cond. Val. Ar. Dom.
Time 1.00 0.01 −0.04 0.07 0.01
Condition 0.01 1.00 −0.32 0.08 −0.32
Valence −0.04 −0.32 1.00 0.10 0.43
Arousal 0.07 0.08 0.10 1.00 −0.15
Dominance 0.01 −0.32 0.43 −0.15 1.00
Note:Therearenostrongcorrelationswithtime,butvalence,dominanceandtheexperimental
conditionseemtobecorrelated.
The correlation coefficients in Table 2 confirm the observed relation among ratings of
valence, dominance and the frustration condition. While dominance and valence are
supposed to be orthogonal emotional dimensions, they do not seem to be uncorrelated.
Correlations with time are included as well to show trends of the self-assessments over
time.Asexpected,theexperimentalconditionbarelycorrelateswithtime,andtheemotional
ratingsdonotdriftovertime.
Interestingly, the dominance ratings correlate more strongly with the valence ratings
(ρ = 0.43) than with the experimental condition (ρ = −0.32). This shows that the
dominance ratings do not simply capture the physical lack of control of the user, but the
subjectiveexperienceofdominance.
54 B.Reuderink,C.MühlandM.Poel
The non-orthogonality of the ratings complicate the interpretation of the results. In
additiontotheoriginalratings,wealsoanalysedcorrelationswiththescoresofaprincipal
component analysis (PCA) of these ratings. The loadings for the new dimensions are
displayed in Table 3. More than half of the variance is explained by a component in
the direction of normal keyboard control, and positive valence (happiness), which is not
surprising as this is the main effect in our experiment. Orthogonal to this direction, we
findasecondprincipalcomponent(PC)thatmodelsvarianceinthedirectionofthenormal
conditioncombinedwithnegativevalence(sadness).Arousalanddominancedonotplay
a big role in PC0 and PC1, but are modelled in PC2 (high dominance with low arousal,
orsuperiority)andPC3(highdominancewithhigharousal,oranger)instead.Thesenew
dimensionscanbeusedtodisentangleeffectsfoundforbothvalenceanddominance.
Table3 The PCs for the space spanned by the frustration condition, valence, arousal and
dominance
PC0 PC1 PC2 PC3
Mainlypositive Mainlynegative Mainlyaroused
emotionsand emotionsand Relaxedand anddominant
controllablegame controllablegame dominantmental mentalstate
Description (happiness) (sadness) state(superiority) (anger)
Var.explained 52% 32% 11% 5%
Condition −0.72 −0.70 0.05 0.00
Valence 0.66 −0.69 −0.13 −0.27
Arousal 0.06 −0.11 −0.67 0.73
Dominance 0.22 −0.17 0.73 0.62
5.2 Alphaasymmetries
Foreachuserindividually,wecalculatedthealphaasymmetryindexasoutlinedinSection4
for each experimental block. Table 4 lists the Pearson correlation coefficients ρ between
alphaasymmetriesandtheemotionalratings,averagedoversubjects.WeusedaWilcoxon
signed-rank test to find ρ’s that deviate significantly from zero over subjects. While the
correlationsarequitelow,wefindsignificantcorrelationsfortheunmodifiedvalenceand
dominancescale.Anotableobservationisthatdominancecorrelatesbetterwiththealpha
asymmetrythanvalenceoverthefronto-centralcortex.InthePCAspace,alphaasymmetry
in the fronto-central is correlated more strongly with PC2, the PC that is associated with
‘superior’feelings(relaxedanddominant).
For PC0 (happiness), we do find a correlation with alpha asymmetry for frontal alpha
power(Fp1–Fp2).Forfrontalalphapower,wealsofoundasignificantnegativecorrelation
with the condition variable, and a positive correlation with valence. PC0 is oriented in
exactlythisdirection,soitisnotsurprisingthatwedofindasignificantpositivecorrelation
ofalphaasymmetryinFp1–Fp2withPC0,butitremainsunclearwhetherthisasymmetry
iscausedbyvalence,ormainlybytheexperimentalcondition.
Ourresultsrevealedacorrelationoversubjectsofalphaasymmetrywithvalence,arousal
anddominance.Traditionally,alphaasymmetryisassociatedwithvalence,butourresults
suggestthatthereisastrongerlinkbetweenalphaasymmetryinfronto-centralsensorpairs
withdominancethanwithvalence.