The roles of superficial amygdala and auditory cortex in music-evoked fear and joy

13 Pages·2013·1.35 MB·English

This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for internal non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of the
article (e.g. in Word or Tex form) to their personal website or
institutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies are
encouraged to visit:
http://www.elsevier.com/authorsrights
Author's personal copy
NeuroImage81(2013)49–60
ContentslistsavailableatSciVerseScienceDirect
NeuroImage
journal homepage: www.elsevier.com/locate/ynimg
The roles of superficial amygdala and auditory cortex in music-evoked
fear and joy
Stefan Koelscha,b,c,⁎,1, Stavros Skourasa,b,c,1, Thomas Fritzd, Perfecto Herrerae, Corinna Bonhaged,
Mats B. Küssnerf,g, Arthur M. Jacobsa,c
aClusterLanguagesofEmotion,FreieUniversitätBerlin,Germany
bUniversityofSussex,Falmer,UK
cDahlemInstituteforNeuroimagingofEmotion,Berlin-Dahlem,Germany
dMaxPlanckInstituteforHumanCognitiveandBrainScience,Leipzig,Germany
eUniversitatPompeuFabra,Barcelona,Spain
fKing'sCollegeLondon,UK
gGoldsmiths,UniversityofLondon,UK
a r t i c l e i n f o a b s t r a c t
Articlehistory: Thisstudyinvestigatesneuralcorrelatesofmusic-evokedfearandjoywithfMRI.Studiesonneuralcorrelates
Accepted2May2013 ofmusic-evokedfeararescant,andthereareonlyafewstudiesonneuralcorrelatesofjoyingeneral.Eigh-
Availableonline17May2013 teenindividualslistenedtoexcerptsoffear-evoking,joy-evoking,aswellasneutralmusicandratedtheir
ownemotionalstateintermsofvalence,arousal,fear,andjoy.ResultsshowthatBOLDsignalintensityin-
Keywords:
creasedduringjoy,anddecreasedduringfear(comparedtotheneutralcondition)inbilateralauditorycortex
Emotion
(AC)andbilateralsuperficialamygdala(SF).Intherightprimarysomatosensorycortex(area3b)BOLDsig-
Music
nalsincreasedduringexposuretofear-evokingmusic.Whileemotion-specificactivityinACincreasedwith
Fear
Joy increasingdurationofeachtrial,SFrespondedphasicallyinthebeginningofthestimulus,andthenSFactiv-
fMRI itydeclined.PsychophysiologicalInteraction(PPI)analysisrevealedextensiveemotion-specificfunctional
Auditorycortex connectivityofACwithinsula,cingulatecortex,aswellaswithvisual,andparietalattentionalstructures.
Superficialamygdala Thesefindingsshowthattheauditorycortexfunctionsasacentralhubofanaffective-attentionalnetwork
that is more extensive than previously believed. PPI analyses also showed functional connectivity of SF
withACduringthejoycondition,takentoreflectthatSFissensitivetosocialsignalswithpositivevalence.
Duringfearmusic,SFshowedfunctionalconnectivitywithvisualcortexandarea7ofthesuperiorparietal
lobule,takentoreflectincreasedvisualalertnessandaninvoluntaryshiftofattentionduringtheperception
ofauditorysignalsofdanger.
©2013ElsevierInc.Allrightsreserved.
Introduction Messiah(“AndHeShallPurify”),Mozart'sIdomeneo,thethunderstorm
portrayedinBeethoven'ssixthsymphony,Berlioz'Songed'unenuitdu
Of all emotions, fear is the one that has been investigated most Sabbat,Herrmann'smusicforPsycho,andPenderecki'sPolymorphia.
intensely in affective neuroscience over the last decades. However, Nevertheless,onlytwopreviousfunctionalneuroimagingstudies
there is scarcity of functional neuroimaging studies on fear with haveinvestigatedbrainresponsestofear-evokingmusic.Oneofthese
music,andneuralcorrelatesofmusic-evokedfearhavethusremained studiesexploredhowfearmusiccanenhancefeelingsoffearevoked
elusive. This stands in gross contrast to a long musical tradition of byimages(Baumgartneretal.,2006),butthatstudydidnotpresent
usingmusicalmeanstoevokefearinthelistener.Theearliesttheoreti- fearmusicalone,thusleavingopenthequestionastowhichactivation
caltreatiseonsuchmeansistheAffektenlehre(“theoryofaffects”)of patternswouldbeevokedbyfearfulmusicalone(i.e.,withoutnegative
theBaroque,whichprescribedmusicalmethodsandfiguresforimitat- visualimages).Theotherstudyinvestigatedhowmusicevokingfearor
ing,orportraying(andthus,accordingtotheAffektenlehre,summoning) joycanchangetheperceptionofneutralfilmclips(Eldaretal.,2007).
emotions, including fear (Mattheson, 1739/1999). Among countless Thelatterstudyalsoinvestigatedbrainresponsesevokedbythefear
well-known examples of fear-evoking (Western) music are Handel's musicalone(withoutfilmclips),comparedtoabaselinecondition,in
selectedregionsofinterest(amygdala,anteriorhippocampalformation,
prefrontal cortex, and auditory cortex). However, no effects of fear
⁎ Correspondingauthorat:FreieUniversitätBerlin,ClusterLanguagesofEmotion, musicwereobservedwithoutfilm-clips(noreffectsofjoyorneutral
HabelschwerdterAllee45,14195Berlin,Germany.Fax:[phone].
musicwithoutfilm-clips),neitherintheamygdala,norinthehippo-
E-mailaddress:[email protected](S.Koelsch).
1 Thefirsttwoauthorscontributedequallytothiswork. campusortheprefrontalcortex.Inaddition,astudybyLerneretal.
1053-8119/$–seefrontmatter©2013ElsevierInc.Allrightsreserved.
http://dx.doi.org/10.1016/j.neuroimage.2013.05.008
Author's personal copy
50 S.Koelschetal./NeuroImage81(2013)49–60
(2009)showedthatlisteningtofear-evokingmusicwithclosedeyes music. Similarly, musical and acoustical parameters such as tempo
(compared to listening with open eyes), evoked greater activation andpitchvariationcanwellbematchedbetweenjoyandfearmusic.
thanopeneyesintheamygdala/anteriorhippocampalformationand Moreover,joyfulmusicwaschosentoreplicateresultsofpreviousstudies.
anterior temporal poles (this effect of eyes open/closed was not ob- Althoughonlyafewpreviousfunctionalneuroimagingstudiesspecifically
servedwhenlisteningtoneutralmusic).Maineffectsoffearcompared used“happy”(Bratticoetal.,2011;Brownetal.,2004;Mitterschiffthaler
toneutralmusicwerenotreportedinthatstudy.Finally,arecentstudy etal.,2007)or“joyful”(Koelschetal.,2006;Muelleretal.,2011)music,
byTrostetal.(2012)reportedbrainactivationsduetomusic-evoked these studies, along with other studies investigating musical frissons
“tension”(characterizedbyfeelingsofhigharousalandlowvalence), (BloodandZatorre,2001;Salimpooretal.,2011),ormusicevokingemo-
underwhichtheauthorsalsosubsumed“feelingsofanxietyandsus- tionalresponseswithpositivevalenceandhigharousal(Trostetal.,2012)
penseinducedbyscarymusic”(brainactivationsincludedbilateralsupe- indicateanumberofrelativelyconsistentfeatures,namelystrongerBOLD
riortemporalgyrus,rightparahippocampalgyrus,motorandpremotor signalintensity(a)intheauditorycortex(Bratticoetal.,2011;Koelschet
areas,cerebellum,rightcaudatenucleus,andprecuneus).Notably,the al.,2006;Mitterschiffthaleretal.,2007;Muelleretal.,2011;Trostetal.,
conceptof“tension”alsoincludesemotionalphenomenanotrelatedto 2012),(b)theventralstriatum(BloodandZatorre,2001;Brownetal.,
fear,suchasemotionalreactionstounexpectedmusicalevents(Huron, 2004;Koelschetal.,2006;MenonandLevitin,2005;Mitterschiffthaler
2006),and,therefore,Trostetal.(2012)arguedthatitisnotclearwheth- et al., 2007; Trost et al., 2012), (c) the anterior insula (Blood and
ertheobservedbrainactivationswereduetofearresponses,ortomore Zatorre,2001;Brownetal.,2004;Koelschetal.,2006),and(d)the
generalfeelingsoftensionandunease.Thus,therearenofunctionalneu- anterior cingulate cortex (Blood and Zatorre, 2001; Janata, 2009;
roimagingdataavailablethatwouldallowustodrawconclusionsabout Mitterschiffthaleretal.,2007).Moreover,(e)severalstudiesonmusic-
neuralcorrelatesofmusic-evokedfear. evokedemotionsshowedsignalchangesintheanteriorhippocampal
Withregardtolesionstudies,Gosselinetal.(2005)showedim- formation in response to stimuli with positive emotional valence
pairedrecognitionofscarymusicinepilepticpatientsfollowinguni- (e.g., Blood and Zatorre, 2001; Mueller et al., 2011; Trost et al.,
lateral medial temporal lobe excision (including the amygdala). In 2012).Basedonthesefindings,wehypothesizedincreasedBOLDsig-
thatstudy,bothpatientswithleftorrightmedialtemporalloberesec- nalsinresponsetojoystimuli(comparedtoneutralorfearstimuli)in
tions showed impaired recognition of scary, but not happy or sad, the auditory cortex, ventral striatum, insula, ACC, and hippocampal
music. Corroborating this finding, data from a patient with bilateral formation.
damagerestrictedtotheamygdalashowedaselectiveimpairmentin Anotheraspectofourstudywastheinvestigationofthetemporal
therecognitionofscaryandsadmusic(Gosselinetal.,2007),indicating dynamicsofemotionacrosstime.Toourknowledge,onlytwoprevi-
thattherecognitionoffearexpressedbymusicinvolvestheamygdala. ousfunctionalneuroimagingstudieshaveinvestigatedthetemporal
Thesefindingsarereminiscentoffindingsreportingsimilarimpairment dynamics of neural correlates of emotion (for habituation-effects
fortherecognitionoffearfulfaces(reviewedinPeretz,2010),suggesting across an experimental session see Mutschleret al., 2010). A study
thatscarymusicandfearfulfacesareprocessed,atleastinpart,bycom- bySalimpooretal.(2011)reportedthatBOLDsignalintensityincreased
moncerebralstructures.Supportingthisassumption,patientswithuni- (a)inthedorsalstriatumduringtheanticipationofamusic-evokedfris-
lateralanteromedialtemporallobeexcisionwerefoundtobeimpaired son,and(b)intheventralstriatumduringtheexperienceofthefrisson
intherecognitionofbothscarymusicandfearfulfaces(Gosselinetal., (notably,additionalPETdatashowedthatthesesignalincreaseswere
2011),withresultsinbothtasksbeingcorrelated.Thissuggestedamul- relatedtodopaminergicsynapticactivityinthesestructures).Another
timodalrepresentationoffearwithintheamygdala(althoughrecog- study (Koelsch et al.,2006),in which stimuli of60s were split into
nition of fearful faces was preserved in some patients, while their two 30-second halves, showed that significant signal differences be-
recognitionofscarymusicwasimpaired).However,duetothesize tweenpleasantandunpleasantmusicweremostpronouncedduring
ofthelesionsinthereportedstudies,itremainsunclearwhichnuclei thesecondhalfofthetrials.Thestructureswithsuchtemporaldynam-
oftheamygdaloidcomplexplayedaroleinthereportedfindings. ics of activation included the auditory cortex, inferior fronto-lateral
Functionalneuroimagingstudiesonfearevokedbyvisualstimuli, areas(area45andtheposteriorpartoftheinferiorfrontalsulcus),an-
recall/imagery,orauditory(butnotmusical)stimulihavealsoimplicat- teriorinsula,theamygdaloidcomplex(probablybasolateralamygdala),
edtheamygdaloidcomplex(LeDoux,2000),inparticularthebasolateral hippocampalformation,temporalpoles,andparahippocampalcortex
amygdala(BL),aswellasarangeoffunctionallyconnectedstructuresin (asimilartrendwasobservedintheventralstriatum).
fearresponses(e.g.,Phanetal.,2002).Suchstructuresincludetheaudi- Particularcarewastakenwithregardtotheacousticparameters
torycortexinauditoryfearconditioningparadigms(LeDoux,2000),as ofourstimuli:numerousacousticalfeaturesofthestimuliweremea-
wellasalargearrayofbothcorticalandsubcorticalstructures,suchas sured, which allowed us (1) to match joy, fear, and neutral stimuli
cingulate and insular cortex, hippocampus, parahippocampal cortex, withregardtonumerousacousticalparameters(e.g.,pitchvariation,
orbitofrontalcortex,dorsolateralprefrontalcortex,striate(visual)cor- tempo,intensity,and spectralflux),and (2) tointroduceacoustical
tex,basalganglia,cerebellum,aswellasbrainstemregionssuchasthe factorsthatdifferedbetweenconditionsasregressorsofnointerest
periaqueductalgray(Royetal.,2009;Steinetal.,2007;Williamsetal., intheanalysisoffMRIdata.Providedthatnocrucialacousticalfeatures
2006). weremissed,thisenabledustoinvestigatetheroleoftheauditorycortex
Basedonthereportedfindings,weaimedtoinvestigatetheroleof with regard to its emotion-specific interfacing with limbic/paralimbic
the amygdaloid complex and the auditory cortex, including their structures. Previous work has implicated auditory association cortex
functionalconnections,forfearevokedbymusic.Theculturalpractice (auditoryparabelt),aswellasitsconnectionswiththelateralamygdala,
of using music to evoke fear makes music an important means to infearconditioning(LeDoux,2000).However,auditoryparabeltregions
investigate neural circuits underlying fear (Eerola and Vuoskoski, projecttonumerouslimbic/paralimbicstructures(suchasorbitofrontal
2011),inadditiontothevastnumberofstudiesusingvisualstimuli cortex, insula, and cingulate cortex; e.g. Petrides and Pandya, 1988;
toinvestigateneuralcorrelatesoffear.Besidesfearstimuli,thepresent Smileyetal.,2007;Yukie,1995),andtheroleoftheseauditoryprojec-
studyalsousedjoyfulandneutralmusic.Joywaschosenaspositive tionsforemotionalprocesses,andthustherolethattheauditorycortex
emotionbecause,ontheonehand,bothjoyandfearareconsideredas playsforemotionalprocesses,islargelyunknown.
“basic emotions” (Ekman, 1999), and both the expression of joy as
wellasoffearinWesternmusiccanberecognizeduniversally(Fritz Summaryofhypotheses
etal.,2009).Ontheotherhand,otherthan,e.g.peacefulmusic(which
isalsoperceivedaspositive,e.g.Vieillardetal.,2008),arousallevels Motivatedbythereportedfindings,wetestedwhethermusic-evoked
evokedbyjoymusiccanwellbematchedwiththoseevokedbyfear fear,ascomparedtoneutralorjoystimuli,wouldelicitsignalchangesin
Author's personal copy
S.Koelschetal./NeuroImage81(2013)49–60 51
thebasolateralnucleusoftheamygdaloidcomplex.Forjoy,ascompared triggerinstrumentsamplesfromahighqualitynaturalinstrumentli-
toneutralorfear,weexpectedstrongerBOLDsignalintensityintheven- brary (X-Sample Chamber Ensemble, Winkler & Stahl GbR, Detmold,
tralstriatum,auditorycortex,hippocampalformation,insula,andcingu- Germany)andfromtheAbletonInstrumentlibrary(AbletonAG,Berlin,
latecortex.Finally,toexploreneuralnetworksunderlyingjoyandfear, Germany).Stimuliwerethenrenderedaswav-filesusingAbletonLive.
weperformedaPsychophysiologicalInteraction(PPI)analysisusingthe UsingPraat(version5.0.29;Boersma,2002),allexcerpts(joy,fear,and
peak voxels indicated by the contrast analysis between conditions as neutral)wereeditedsothattheyallhadthesamelength(30s),1.5s
seed voxels. More specifically, we were interested in emotion-specific fade-in/fade-outramps,andthesameRMSpower.
functionalconnectivitybetweenamygdaloidcomplexandauditorycor- Importantly,joyandfearstimuliwerechosensuchthateachjoyful
tex,betweenauditorycortexandinsula,aswellasbetweenauditorycor- excerpthadafearfulcounterpartthatmatchedwithregardtotempo
texandcingulatecortex. (beatsperminute),meanfundamentalfrequency,variationoffunda-
mentalfrequency,pitchcentroidvalue,spectralcomplexity,andspec-
Materialsandmethods tral flux. This was confirmed by an acoustic analysis of the stimuli
using‘Essentia’,anin-houselibraryforextractingaudioandmusicfea-
Participants turesfromaudiofiles(http://mtg.upf.edu/technologies/essentia).The
Essentia software was also used to specify acoustical differences be-
18individuals(aged20–31years,M=23.78,SD=3.54,9females) tween stimuli withregard to other acousticalfactors:177 acoustical
took part in the experiment. All participants had normal hearing descriptorswereextractedframe-by-frame(framelength=21.5ms,
(as assessed with standard pure tone audiometry) and were right- 50%overlap),averagedalongtheentiredurationofthefile,andthen
handed(accordingtoself-report).Noneoftheparticipantswasapro- compared between conditions (joy, neutral, fear) using one-way
fessionalmusician,noramusicstudent.Sevenparticipantshadnofor- ANOVAs. Bonferroni-corrected significance-level was 0.05/177=
malmusicaltraining,eightparticipantshadoncereceivedmusiclessons 0.00028 (lowering this thresholdfor one-sided tests,i.e.0.00056,
(mean duration of formal training was 2.81years, SD=2.36, instru- didnotchangeanyoftheresults).Theextractedfeaturesrepresent
ments were: flute, drums, piano, violin, guitar and melodica) but had acousticandmusicalfeaturesusedinmusicinformationretrieval,
notplayedtheirinstrumentsforseveralyears(M=8.83,SD=7.52), i.e.,differentcombinationsofthemareusedforpredictivemodels
and three participants had learned a musical instrument that they ofmusicallyrelevantcategorizationssuchasgenredetection,instrument
were still playing (mean duration of formal training was 12.5years, detection,keyandmodedetection,oremotionalexpression.Although
SD=3.5,instrumentswere:guitar,violin,pianoandelectricbass).Ex- thesefeatureshavemostlybeenvalidatedinmachine-learningcontexts
clusioncriteriawereleft-handedness,professionalmusicianship,past (Huqetal.,2010;Kimetal.,2010;Laurier,2011),itispossiblethatthey
diagnosisofaneurologicalorpsychiatricdisorder,ascoreof≥13on alsoplayaroleforhumanauditoryperception.Inaddition,manyofthe
Beck's Depression Inventory (BDI; Beck et al., 1993), excessive con- usedparametershavebeenvalidatedinperceptualexperiments,suchas
sumption of alcohol orcaffeineduringthe24hpriortotesting, and featuresrelatedtospectralcomplexity,F0,andF0variations(Agrawalet
poorsleepduringthepreviousnight.Allsubjectsgavewritteninformed al., 2012; Alluri et al., 2012; Coutinho and Dibben, 2012; Juslin and
consent. The study was conducted according to the Declaration of Laukka,2003;Kumaretal.,2012),sensorydissonance (Coutinho and
HelsinkiandapprovedbytheethicscommitteeoftheSchoolofLifeSci- Dibben,2012;Koelschetal.,2006;PlompandLevelt,1965;Vassilakis
encesandthePsychologyDepartmentoftheUniversityofSussex. andKendall,2010),spectralflux(CoutinhoandDibben,2012;Menon
et al., 2002), spectral centroid (Coutinho and Dibben, 2012), spectral
Stimuliandprocedure crest (Laurier, 2011), temporal modulation frequencies (Kumar et al.,
2012),keystrength(Allurietal.,2012;Krumhansl,1990),andpulseclar-
Musicalstimuliwereselectedtoevoke(a)feelingsofjoy,(b)feelings ity(Allurietal.,2012).Significanteffectsofconditionwereindicatedfor
of fear, or(c)neitherjoy norfear (henceforth referred to asneutral thefollowingacousticfactors(withF-valuesinparentheses,degreesof
stimuli).Thereweren=8stimulipercategory(thecompletelistof freedom: 2, 21): (a) Mean (72.3) and variance (13.8) of F0 salience
joyandfearstimuliisprovidedinSupplementaryTableS1).Joystimuli (thismeasureishighestforsingletones,intermediateforchords,and
hadbeenusedinpreviousstudies(e.g.,Fritzetal.,2009;Koelschetal., lowestfornoises;notethatmeanF0andvarianceofF0didnotdifferbe-
2010a,2011;Muelleretal.,2011)andconsistedofCD-recordedpieces tweenjoy,fear,andneutralstimuli).ThemeanF0saliencewashighest
fromvariousepochsandstyles(classicalmusic,Irishjigs,jazz,reggae, forneutral,intermediateforjoy,andlowestforfearstimuli(pb.0001
SouthAmericanandBalkanmusic).Fear-evokingmusicalstimuliwere inallpairwisecomparisons).Thisreflectsthatbothjoyandfear(but
excerptsfromsoundtracksofsuspensemoviesandvideogames.Toin- notneutral)stimulicontainednumerousharmonies,andthatfear(but
crease the fear-evoking effect of the fear stimuli, their relatively high not joy) stimuli contained numerous percussive sounds, as well as
acoustic roughness (see also next paragraph) was further increased: hissingandwhooshingnoises.(b)Mean(41.3)andvariance(28.0)of
from each fear excerpt, two copies were obtained and pitch-shifted, sensorydissonance.Sensorydissonancewaslowestforneutral,interme-
one copy was shifted one semitone higher, the other copy a tritone diateforjoy,andhighestforfearstimuli.Meansensorydissonancedif-
lower(seealsoFritzetal.,2009;Koelschetal.,2006).Then,allthreever- feredsignificantly between joy and neutral (pb.0001), between fear
sionsofoneexcerpt(originalpitch,onesemitonehigher,andatritone and neutral (pb.0001), and between joy and fear stimuli (pb.05).
lower) were rendered as a single wav-file (pitch-shift and rendering (c)Meanchordstrength(25.2)andkeystrength(14.7);thesefactors
was performed using Ableton Live, version 8.0.4, Ableton AG, Berlin, measurehow stronglya sound resembles thesoundofa chord,and
Germany). Neutral stimuli were sequences of isochronous tones, for howclearlythesoundsofastimuluscanbeattributedtoakey.Chord
which the pitch classes were randomly selected from a pentatonic strength was higher for joy compared to fear stimuli (pb.0001), as
scale.ThesetonesequencesweregeneratedusingtheMIDI(musicalin- wellasforjoycomparedtoneutralstimuli(pb.0006),whereasfear
strument digital interface) toolbox for Matlab (Eerola and Toiviainen, and neutral stimuli did not differ significantly from each other. Key
2004).Importantly,foreachjoy–fearstimuluspair(seebelow),aneutral strengthwashigherforjoycomparedtofearstimuli(pb.0001),and
controlstimuluswasgeneratedthatmatchedjoyandfearstimuliwith forneutralcomparedtofearstimuli(p=.01);joyandneutralstimuli
regardtotempo,F0 range(i.e.,rangeof thefundamental frequency), didnotdiffersignificantlyfromeachother(p>.15).(d)Mean(30.0)
and instrumentation (using the two main instruments or instrument and variance (16.4) of spectral flux (a measure of spectral variation
groupsoftherespectivejoy-fearpair).Tocreatestimulithatsounded withinsounds),mean(30.0)spectralcrest(ameasureoftheinhomoge-
likemusicalcompositionsplayedwithrealinstruments(similartothe neity,ornoisiness,ofthespectrum)andmean(10.6)spectralcomplexity
joy andfear stimuli),the tonesfrom the MIDIsequenceswere setto (whichcorrelateswiththeamountofdifferenttimbresthatarepresent
Author's personal copy
52 S.Koelschetal./NeuroImage81(2013)49–60
inapiece).Meanspectralflux,spectralcrest,andspectralcomplexity USA)withintegratedeye-trackerthatallowedustoguaranteethatpartic-
werelowestforneutralstimuli(withsignificantdifferencesbetween ipantsopenedandclosedtheireyesaccordingtotheinstruction.
neutralandjoy,aswellasbetweenneutralandfearstimuli,pb.05in
eachtest),anddidnotdiffersignificantlybetweenjoyandfearstimuli MRscanning
(p>.2ineachtest).
PriortotheMRIsession,participantswerepresentedwithshort Scanningwasperformedwitha3TSiemensMagnetomTrioTim.Prior
(12s)versionsofeachstimulustoobtainfamiliarityratings:Partici- tothefunctionalMRmeasurements,ahigh-resolution(1×1×1mm)
pants rated their familiarity with each piece on a four-point scale T1-weightedanatomicalreferenceimagewasacquiredfromeachpartic-
(rangingfrom“TomyknowledgeIhaveneverheardthispiecebefore”, ipantusingarapidacquisitiongradientecho(MP-RAGE)sequence.Con-
to“Iknowthispiece,andIknowwhocomposedorperformedit”).Par- tinuousEchoPlanarImaging(EPI)wasusedwithaTEof30msandaTR
ticipantswerethentrainedontheratingprocedure,using12slongex- of2000ms.Slice-acquisitionwasinterleavedwithintheTRinterval.The
cerptsofmusicalpiecesthatdidnotbelongtothestimulussetusedin matrix acquired was 64×64 voxels with a field of view of 192mm,
thefMRIscanningsession. resultinginanin-planeresolutionof3mm.Slicethicknesswas3mm
During the fMRI scanning session, stimuli were presented in a withaninterslicegapof0.6mm(37slices,wholebraincoverage).The
pseudo-randomordersothatnomorethantwostimuliofeachstim- acquisitionwindowwastiltedatanangleof30°relativetotheAC-PC
ulus category (joy, fear, neutral) followed each other. Participants lineinordertominimizesusceptibilityartifactsintheorbitofrontalcortex
were asked to listen to the musical stimuli with their eyes closed (Deichmannetal.,2002,2003;Weiskopfetal.,2007).Giventheduration
(see also Lerner et al., 2009). Each musical stimulus was followed ofourstimuli(30s),acontinuousscanningdesignwasrequiredtoper-
byanintervalof2sinwhichabeeptoneof350Hzand1sduration form the PPI analysis (so that enough data points were available for
signaledparticipantstoopentheireyesandtocommencetherating meaningfulcorrelationestimations,seebelow).
procedure. During the rating procedure, participants indicated how
theyfeltattheendofeachexcerptwithregardtovalence(‘pleasantness’), Dataanalysis
‘arousal’,‘joy’and‘fear’.Thatis,participantsprovidedratingsabouthow
theyfelt,andnotaboutwhichemotioneachstimuluswassupposedto FMRIdatawereprocessedusingLIPSIA2.1(Lohmannetal.,2001).
express(GabrielsonandJuslin,2003;JuslinandVästfjäll,2008).Ratings Data were corrected for slicetime acquisition and normalized into
were obtainedwith 6-pointLikertscales (ranging from “not atall” to MNI-space-registeredimageswithisotropicvoxelsof3cubicmillime-
“verymuch”).Thetimeintervalfortheratingprocedurewas12sand ters.Atemporalhighpassfilterwithacutofffrequencyof1/90Hzwas
eachratingperiodwasfollowedbya4srestperiod(duringwhichpartic- appliedtoremovelowfrequencydriftsinthefMRItimeseries,anda
ipantsclosedtheireyesagain),amountingtoatotallengthof48spertrial spatialsmoothingwasperformedusinga3DGaussiankernelandafil-
(seeFig.1).TheentirestimulussetwaspresentedtwiceduringthefMRI tersizeof6mmFWHM.
scanning session. Musical stimuli were presented using Presentation AmixedeffectsblockdesignGLManalysiswasemployed(Friston
(version13.0,Neurobehavioralsystems,Albany,CA,USA)viaMRIcom- etal.,2007).Valenceratings,arousalratings,familiarityratings,psy-
patibleheadphones(underwhichparticipantsworeearplugs).Instruc- choacousticparametersthatdifferedsignificantlybetweenconditions
tionsandratingscreensweredeliveredthroughMRIcompatibleliquid (seeStimuliandprocedure),andrealignmentparameterswereinclud-
crystal display goggles (Resonance Technology Inc., Northridge, CA, edinthedesignmatrixascovariatesofnointerest(Johnstoneetal.,
2006).Then,one-samplet-testswerecalculatedvoxel-wiseforthecon-
trastbetweenfearvs.joy,andcorrectedformultiplecomparisonsby
theuseofcluster-sizeandcluster-valuethresholdsobtainedbyMonte
Carlosimulationswithasignificancelevelofpb0.05(Lohmannetal.,
2008).Thesignificantclustersidentifiedinthisanalysiswereusedas
regionsofinterest(ROIs)tocomparetheaveragesignalintensity(aver-
agedacrossallvoxelsineachcluster)withinthoseclustersbetween
fearandneutral,aswellasbetweenjoyandneutral.Inaddition,toex-
plorethetemporalnatureofthesignificantdifferencesinactivitybe-
tweenfearandjoy,foreachpeakvoxelofeachsignificantcluster,the
timecourseofactivitywasdeterminedbycomputingthevoxelintensity
separatelyforeachscan(i.e.,withatemporalresolutionof2s)andfor
eachcondition.
Temporalinteractionanalysis
Toinvestigatepossibleinteractionsbetweenemotionandtimewe
splitthedatafromeachtrialintofirsthalf(seconds1to15)andsec-
ond half (seconds 16 to 30), and calculated a statistical parametric
mapbasedontheinteractionbetweenemotion(twolevels:joy,fear)
andtime(twolevels:firsthalf,secondhalf).Afirst-levelinteractioncon-
trastwascalculatedforeachsubject,andthecontrastimageswerethen
usedforvoxel-wiseone-samplet-testsatthesecondlevel(correctedfor
multiplecomparisonsbytheuseofcluster-sizeandcluster-valuethresh-
oldsobtainedby Monte Carlo simulationswith asignificancelevelof
pb.0.05)toidentifyclustersofvoxelsforwhichtheemotion×timein-
teractionwassignificantlydifferentfromzero.
Fig.1.Experimentaldesign.Ineachtrial,amusicstimuluswaspresentedfor30s.
Musicstimuliwerepseudorandomlyeitherajoy,afear,oraneutralstimulus.Partici- PPIanalysis
pantslistenedtothemusicwiththeireyesclosed.Then,abeeptonesignaledtoopen The timecourses of activity at the peak voxels identified in the
theeyesandtocommencetheratingprocedure.Fourratings(feltvalence,arousal,joy,
contrastjoyvs.fear,averagedtogetherwiththetimecoursesfromad-
andfear)wereobtainedin12s,followedbya4spause(duringwhichparticipants
closedtheireyesagain).Trialdurationwas48s,theexperimentcomprisedof48trials. jacentvoxels,wereusedasseedsforPsychophysiologicalInteraction
Author's personal copy
S.Koelschetal./NeuroImage81(2013)49–60 53
(PPI)analysestoidentifytargetregionsforwhichthecovariationof beinginbetween.Althoughthedegreeofexperiencedfearwasrelatively
activity between seed and target regions was significantly different moderate(4.02onascalefrom1to6),fearratingsdifferedsignificantly
between experimental conditions. At the first level, contrasts were betweenfearandneutralstimuli(t(15)=17.71,pb0.0001),fearand
calculated for each subject based on the interaction term between joy stimuli (t(15)=33.16, pb0.0001), and between joy and neutral
emotion (joy vs. fear) and each seed voxel's timecourse of activity stimuli (t(15)=9.93, pb0.0001). Average familiarity ratings were
(Fristonetal.,1997).Foreachseedvoxel,thecontrastimagesfrom highestforjoystimuli,lowestforneutralstimuli,withratingsforfear
allsubjectswereusedinvoxel-wiseone-samplet-testsatthesecond stimuli being in between. Familiarity ratings differed significantly be-
level(correctedfor multiplecomparisonsby theuse of cluster-size tween joy and fear stimuli (t(7)=3.659, pb0.05), fear and neutral
and cluster-value thresholds obtained by Monte Carlo simulations stimuli (t(7)=4.41, pb0.01), and between joy and neutral stimuli
with a significance level of pb.0.05) to identify clusters of voxels (t(7)=5.06,pb0.0005).Duetothedifferencesinthebehavioralratings
forwhichthepsychophysiologicalinteractioneffectwassignificant. betweenstimuluscategorieswithregardtovalence,arousal,andfamil-
iarity, each participant's valence, arousal, and familiarity ratings were
used in the fMRI data analysis as regressors of no interest (see Data
Results analysis). Therefore, these variables (valence, arousal, and familiarity)
didnotcontributetothefMRIresultspresentedinthefollowing.
Behavioraldata
Behavioral data are summarized in Fig. 2 and Table 1. Valence fMRIdata
(pleasantness) ratings were lower for fear than for joy stimuli
(t(15)=42.29,pb0.0001),higherforjoythanforneutralstimuli GLManalysis
(t(15)=16.10, pb0.0001), and did not differ significantly between Thestatisticalparametricmaps(SPMs)ofthecontrastjoy>fear
neutralandfearstimuli(t(15)=−1.94,p=.072).Arousalratingswere (correctedformultiplecomparisons,pb.05)revealedsignificantBOLD
higher for fear than for neutral stimuli (t(15)=11.84, pb0.0001), signaldifferencesintheauditorycortex(AC)bilaterally,andinthesu-
higherforjoythanforneutralstimuli(t(15)=12.26,pb0.0001),and perficial amygdala (SF) bilaterally (see also Table 2 and Fig. 3a). The
didnotdifferbetweenjoyandfearstimuli(t(15)=.94,p=.36).Joyrat- activationoftheACcoveredauditorycore,belt,andparabeltregionsbi-
ingswerelowestforfearstimuli,andhighestforjoystimuli,withratings laterally.Thevoxelswithmaximumz-valueswerelocatedalongHeschl's
forneutralstimulibeinginbetween.Joyratingsdifferedsignificantlybe- gyrus (HG), with the peak voxel in the left AC being located on the
tweenfearandneutralstimuli(t(15)=9.03,pb0.0001),fearandjoy postero-lateral rim of HG (30% TE 1.2 according to Morosan et al.,
stimuli(t(15)=32.32,pb0.0001),andbetweenjoyandneutralstimuli 2001),andthepeakvoxelintherightACbeinglocatedmoremedially
(t(15)=16.73,pb0.0001).Correspondingly,fearratingswerehighest onHG(90%TE1.0accordingtoMorosanetal.,2001).Inbothleftand
forfearstimuli,lowestforjoystimuli,withratingsforneutralstimuli rightamygdala,thepeakvoxelwaslocatedinSF(left:80%probability,
Fig.2.Behavioralratingsofparticipantsonthefouremotionscalesusedinthepresentstudy:(a)valence,(b)arousal,(c)joy,and(d)fear.Rangeofscaleswas1to6.Ratingsare
depictedseparatelyforeachstimuluscategory(fear,neutral,joy).Notethatjoystimuliwereratedasmorepleasantthanfearandneutralstimuli(valence/pleasantnessratingsfor
fearandneutralstimulididnotdifferfromeachother).Alsonotethatarousalratingsofjoyandfearstimulididnotdifferfromeachother,andthatbothjoyandfearstimuliwere
ratedasmorearousingthanneutralstimuli.
Author's personal copy
54 S.Koelschetal./NeuroImage81(2013)49–60
Table1 (andasimilartrendisobservableintherightSF).Differencesbetween
Descriptivestatisticsofbehavioraldata(mean,withstandarddeviationinparentheses). conditions emerged several seconds after stimulus onset, were most
Rangeofvalence,arousal,joy,andfearscaleswas1to6,rangeofthefamiliarityscale
pronouncedataround10s,andvanishedtowardstheendofthestimuli
was1to4.Forstatisticaltestsseemaintext.
(seenextparagraphforstatisticalanalysis).IntherightS1,allcondi-
Fear Neutral Joy tionsevoked aninitial signal decrease, followed bya signal increase
Valence 2.43(0.20) 2.69(0.48) 4.84(0.21) (whichwasstrongestforfearstimuli),andadeclineofsignalintensity
Arousal 3.97(0.21) 3.03(0.22) 4.05(0.21) towardstheendofthestimuli.
Joyfulness 1.62(0.16) 2.40(0.29) 4.83(0.42)
Fearfulness 4.02(0.23) 2.22(0.30) 1.31(0.17)
Temporalinteractionanalysis
Familiarity 1.44(0.11) 1.17(0.10) 2.01(0.42)
Tostatisticallytestthetemporaldynamicsobservedinthetime-
lines,andtofurtherexplorethetemporaldynamicsofdifferencesbe-
right:90%probabilityaccordingtothecytoarchitectonicprobabilitymap tweenconditionsinotherstructures(seeIntroduction), atemporal
byAmuntsetal.,2005).ThesignaldifferencesinSFextendedbilaterally interactionanalysiswascomputedwithfactorsemotion(twolevels:
intothehippocampal-amygdaloidtransitionarea(HATA,Amuntsetal., joyandfear)andtime(twolevels:firsthalfandsecondhalfofeach
2005).Theoppositecontrast(fear>joy)showedsignaldifferencesin stimulus,seeMaterialsandmethods).Results(correctedformultiple
theanteriorbankoftherightpostcentralgyrus(area3boftheprimary comparisons,pb.05)arelistedinTable3andsummarizedinFig.3b.
somatosensorycortex,S1,thepeakvoxelwaslocatedwith80%probabil- SignificantinteractionswereobservedintheACbilaterally,andinthe
ityinthisareaaccordingtoGeyeretal.,1999).Contrastswiththeneutral leftSF.Thisconfirmstheobservationsbasedonthetimelinesthatdif-
conditiondidnotyieldanyadditionalactivations(seealsoTable2and ferencesintheACweremorepronouncedduringthesecondhalf,and
nextsection),exceptactivationsinthevisualcortexforbothjoy>neutral intheSFduringthefirsthalfoftrials.Moreover,accordingtothehy-
(leftV1,MNI-coordinate:−1,−82,−5;leftV4:−33,−82,−14;right potheses(seeIntroduction),significantemotion×timeinteractions
V2:32,−99,3)andfear>neutral(leftV2,MNI-coordinate:−8,−95, wereobservedbilaterally(a)intheposteriorportionoftheinferior
25;rightV2:23,−93,26). frontalsulcus(IFS),(b)theanteriorpartofBroca'sarea(BA45/46),
and (c) in the ventral pallidum/ventral striatum (see also Fig. 3b).
ROIanalysis These interactions were due to more pronounced differences be-
Tospecifywhethertheobserveddifferencesbetweenfearandjoy tweenconditionsinthesecondcomparedtothefirsthalf.Nointerac-
wereduetosignalincreaseordecreasecomparedtotheneutralcon- tions were observed in the hippocampus, parahippocampal gyrus,
trolcondition,ROIanalyseswereconductedforthesignificantclus- temporalpoles,norintheRolandicoperculum.
tersidentifiedintheGLManalysis(AC,SF,S1),comparingthemean
signalintensityofthevoxelsineachclusterbetweenfearandneutral, PPIanalysis
aswellasbetweenjoyandneutral.Resultsoftheseanalyses(corrected Finally,weconductedaPsychophysiologicalInteraction(PPI)analy-
formultiplecomparisons,pb.05)showedthat,comparedtotheneu- sis(fordetailsseeMaterialsandmethods).Seedregionswerethepeak
tralcondition,therewasstrongersignalintensityduringjoyandweak- voxels(aswellasthedirectlyadjacentvoxels)identifiedintheGLM
ersignalintensityduringfearintheACbilaterallyaswellasintheleftSF analysisinthedirectcontrastbetweenfearandjoystimuli.Resultsof
(seealsoTable2).IntherightSF,signalintensitywasweakerduring thisanalysis(correctedformultiplecomparisons,pb0.05),arelisted
fearcomparedtoneutral(withnodifferencebetweenjoyandneutral). inTable4andsummarizedinFig.3c.
IntherightS1,signalintensitywasstrongerduringfearcomparedto Both left and right AC showed stronger functional connectivity
neutral(joyandneutraldidnotdifferfromeachother). duringjoy(comparedtofear)withbothipsilateralandcontralateral
AC.Inspecific,theleftposterior-lateralauditorybeltshowedstronger
Timelines functional connectivity during joy with both left and right primary
Toexplorethetemporaldynamicsoftheobserveddifferences,the auditorycortex(left:80%,right:100%probabilityforTE1.0according
signalintensityofthepeakvoxelofeachsignificantcluster(AC,SF,S1) toMorosanetal.,2001),aswellaswithlateralauditorybelt-regions
wascomputedseparatelyforeachscan(i.e.,withatemporalresolution ofbothhemispheres.Therightauditorycoreregionshowedstronger
of2s)ineachcondition.ThesetimelinesareshowninFig.4.IntheAC, functionalconnectivityduringjoywithlateralauditorybeltregionsof
theauditorystimulievokedasignalincrease(inallconditions),with bothhemispheres(TE2accordingtoMorosanetal.,2001,noproba-
thesignalintensitybeinggenerallyhighestforjoy,lowestforfear,and bilisticmapsareavailableforthisregion).Duringfear(comparedto
intermediate for neutral (see next paragraph for statistical analysis). joy),bothleftandrightACshowedstrongerfunctionalconnectivity
Themostpronounceddifferencesbetweenconditionsemergedat,and withthecuneus(areas17and18),themedianwalloftheprecuneus
afteraround10safterstimulusonset.InSF,joystimulievokedasignal (areas5and7),andalmosttheentirecingulatesulcus(CS),fromthe
increasebilaterally,whilefearstimulievokedasignalincreaseonlyin pre-genual CStotheascendingbranchofthe(posterior)CS.More-
therightSF.IntheleftSF,differencesinsignalintensitybetweenfear over,bothleftandrightACshowedstrongerfunctionalconnectivity
and joy were particularly strong during the first half of the stimuli duringfearwiththeanteriorinsulabilaterally,andtheleft(butnot
Table2
ResultsofGeneralLinearModel(GLM)contrasts,correctedformultiplecomparisons(pb.05):(a)joy>fear,(b)fear>joy.Thetwooutermostrightcolumnsprovidethep-values
forcomparisonsinvolvingtheneutralconditionwithinthesignificantclustersidentifiedintheGLManalysis(regionofinterestanalysis).Thediamondsintheoutermostrightcolumn
indicatethatdifferencesbetweenfearandneutralwereduetohighersignalintensityduringneutralthanduringfear.Abbreviations:ROI:regionofinterest;l:left;r:right;n.s.:not
significant.
MNIcoordinate Clustersize(mm3) z-Value:max(mean) p-ValueROI:joyvs.neutral p-ValueROI:fearvs.neutral
(a)joy>fear
lHeschl'sgyrus −56−147 16,038 6.36(3.76) .0002 .0001⋄
rHeschl'sgyrus 50−168 13,176 5.55(3.72) .0006 .0007⋄
lsuperficialamygdala −17−7−15 486 4.32(3.36) .02 .009⋄
rsuperficialamygdala 22−6−13 324 3.40(3.09) n.s. .03⋄
(b)fear>joy
rpostcentralgyrus(area3b) 52−1336 297 −3.50(−3.11) n.s. .0001
Author's personal copy
S.Koelschetal./NeuroImage81(2013)49–60 55
Fig.3.FMRIresults(allcorrectedformultiplecomparisons,pb.05).(a)showsthestatisticalparametricmaps(SPMs)ofthedirectcontrastbetweenjoyandfearstimuli,red:
joy>fear,blue:fear>joy.TheSPMsshowstrongerBOLDsignalsduringjoy(comparedtofear)intheauditorycortex(AC),andtheSFbilaterally.StrongerBOLDsignalsduring
fear(comparedtojoy)wereyieldedinarea3boftheprimarysensorycortex.TheinsetshowsthecoordinatesofthepeakvoxelsintheSF(indicatedbytheblackcrosses)projected
onthecytoarchitectonicprobabilitymapaccordingtoEickhoffetal.(2005);green:superficialamygdala,red:basolateralamygdala,yellow:hippocampal-amygdaloidtransition
area,blue:hippocampus(cornuammonis).(b)showstheinteractioncontrastbetweenemotion(joyvs.fear)andtime(1sthalfofeachtrialvs.2ndhalfofeachtrial).Significant
interactionswereindicatedintheauditorycortexbilaterally,theleftSF,leftarea45(parstriangularisoftheinferiorfrontalgyrus),inferiorfrontalsulcus,andventralpallidum/
ventralstriatum.(c)showsresultsofthePsychophysiologicalInteractionAnalysis(PPI)fortheregionsthatsignificantlydifferedintheSPMcontrastbetweenfearandjoy,seed
voxelswerelocatedinleftAC(Heschl'sgyrus),rightAC(Heschl'sgyrus),leftSF,andrightSF.Red/yellowcolorsindicateregionsthatexhibitedstrongerfunctionalconnectivity
withtheseedregionsduringthejoythanduringthefearcondition.Bluecolorsindicateregionsthatexhibitedstrongerfunctionalconnectivitywiththeseedregionsduringthe
fearthanduringthejoycondition.
theright)ACshowedstrongerfunctionalconnectivityduringfearwith connectivityduringjoy(comparedtofear)withthemediodorsalnucle-
the fundus of the central sulcus and the anterior bank of postcentral usofthethalamus(43%th-temporalaccordingtoEickhoffetal.,2005).
gyrus(areas3a&bofS1). TheS1regiondidnotshowanysignificantPPIresultsinourdata.
The left SF showed stronger functional connectivity during joy
(comparedtofear)withrightposteriorHG(posteriorauditorycore Discussion
andbeltregions,80%TE1.1accordingtoMorosanetal.,2001).During
fear(comparedtojoy),theleftSFshowedstrongerfunctionalconnec- Summaryofresults
tivitywithcuneus(V1–V4),andarea7aofthesuperiorparietallobule
(precuneus)bilaterally(left:70%,right:40%probability accordingto ThecontrastanalysisshowedthatBOLDsignalsintheauditorycor-
Scheperjans et al., 2008). The right SF showed stronger functional tex(AC)bilaterallywerestrongestduringjoy,weakestduringfear,with
Author's personal copy
56 S.Koelschetal./NeuroImage81(2013)49–60
a) b)
c) d)
e)
Fig.4.TimelinesdepictingaverageBOLDsignalintensityintheregionsthatsignificantlydifferedintheGLMcontrastjoyvs.fear.Theordinaterepresentsvaluesofvoxelintensity,
theabscissarepresentstime(inseconds),zerocorrespondstotheonsetoftrials.
neutralinbetween.Asimilarpatternwasobservedforthesuperficial (comparedtofear)withboththeipsilateralandthecontralateralAC.
amygdala (SF), except that joy vs. neutral did not differ from each Duringfear(comparedtojoy),bothleftandrightACshowedstronger
other in the right SF. In S1, fear evoked stronger BOLD signals than functionalconnectivitywiththecuneus(areas17and18),themedian
bothneutralandjoy(joyvs.neutraldidnotdiffer).InACbilaterally,re- walloftheprecuneus(areas5and7),andalmosttheentirecingulate
sponses were stronger during the second half of each trial (and the sulcus(CS).Moreover,bothleftandrightACshowedstrongerfunction-
samephenomenonwasobservedinarea45,theIFS,andtheventral alconnectivityduringfearwiththeanteriorinsulabilaterally,andthe
pallidum/ventral striatum). By contrast, BOLD signals in the left SF left(butnottheright)ACshowedstrongerfunctionalconnectivitydur-
werestrongerduring thefirsthalf.PPIresults showed thatbothleft ingfearwithareas3a&bofS1.TheleftSFshowedstrongerfunctional
and right AC showed stronger functional connectivity during joy connectivityduringjoy(comparedtofear)withrightposteriorHeschl's
gyrus.Duringfear(comparedtojoy),theleftSFshowedstrongerfunc-
tionalconnectivitywithcuneus(V1–V4),andarea7aofthesuperior
Table3
Resultsoftheinteractioncontrastofemotion(joyvs.fear)×time(1sthalfvs.2ndhalf parietallobule(precuneus)bilaterally.TherightSFshowedstrongerfunc-
ofeachtrial),correctedformultiplecomparisons(pb.05).Abbreviations:AC:auditory tionalconnectivityduringjoy(comparedtofear)withthemediodorsal
cortex;FOp:frontaloperculum;SF:superficialamygdalaSTG:superiortemporalgyrus; nucleusofthethalamus.
l:left;r:right.
MNI Clustersize z-Value:max Auditorycortexandemotionalprocessing
coordinate (mm3) (mean)
lpost.IFS −511831 1809 4.02(3.39) Pronouncedemotion-specificeffectswereobservedintheauditory
rpost.IFS 452134 1134 3.76(3.17) cortex:IntheGeneralLinearModel(GLM)contrast,BOLDresponses
lparstriangularis(area45)a −513010 – 3.76(–)
intheentiresupratemporalcortex(auditorycore,belt,andparabelt)
rparstriangularis(area45) 513010 513 3.27(3.26)
lantinsula/deepFOp −34344 621 4.22(3.27) were stronger for joy than neutral stimuli, and stronger for neutral
lant.STG −60−9−2 1377 4.56(3.45) thanfearstimuli.Aswillbearguedinthefollowing,theseresultsindi-
rant.STG/planumpolare 510−8 2214 4.81(3.40) cateaprominentroleoftheauditorycortexintheemotionalprocessing
lplanumtemp.(AC) −42−3013 2457 3.94(3.35) ofauditoryinformation.Importantly,therearefivereasonsastowhy
rplanumtemp.(AC) 45−2719 5589 4.50(3.42)
lSF −21−6−14 324 3.51(3.19) the activity differences between conditions cannot simply be due to
lpallidum −1507 1296 4.42(3.47) acousticalfactors:(1)thevaluesofacousticaldescriptorsthatsignifi-
rpallidum 1134 378 3.79(3.31) cantlydifferedbetweenconditionswereincludedascovariatesofno
a Thepeakvoxelinthelparstriangulariswaspartoftheclusterwiththemaximum interest,andshouldthereforenothavecontributedtodifferencesbe-
peakvoxelinthelinsula/deepfrontalFOp. tweenconditionsobservedintheGLMcontrasts.(2)However,evenif
Author's personal copy
S.Koelschetal./NeuroImage81(2013)49–60 57
Table4 the interactions of the auditory cortex with limbic/paralimbic brain
ResultsofPPIanalysis(correctedformultiplecomparisons,pb.05),separatelyforthe structures are likely to be due to emotional processes, rather than
seedvoxelsin:(a)leftAC(Heschl'sgyrus),(b)rightAC(Heschl'sgyrus),(c)leftSF,and
beingmerelyduetocognitiveprocessesrelatedtothekey-strength
(d)rightSF(thePPIanalysiswithS1asseedregiondidnotindicateanyresults).Pos-
ofsounds.(5)Althoughfearstimulihadahigherdegreeofsensory
itivez-values(outermostrightcolumn)indicatestrongerfunctionalconnectivitydur-
ing joy compared to fear, whereas negative z-values indicate stronger functional dissonancethanjoystimuli,activitychangesintheauditorycortex
connectivityduringfearcomparedtojoy.Abbreviations:AC:auditorycortex;FOp: areunlikelytobeduetothisdifferenceonly,becauseneutralstimuli
frontaloperculum;HG:Heschl'sgyrus;ITS:inferiortemporalsulcus;MD:mediodorsal; were even more consonant than joy stimuli. The pattern of BOLD
MTG:middletemporalgyrus;PAC:primaryauditorycortex;SF:superficialamygdala;
signalintensityobservedintheGLMcontrastis,thus,notrelatedto
SFS: superior frontal sulcus; SPL: superior parietal lobule; STG: superior temporal
gyrus;l:left;r:right. thedegreeofsensorydissonanceofthestimuli.
Instead,theobservedpatternofBOLDsignalintensityin theAC
MNI Clustersize z-Value:max
corresponds to the emotion ratings for joy (and inversely for fear,
coordinate (mm3) (mean)
respectively),indicatingthatactivityoftheauditorycortexisrelated
(a)Leftauditorycortex
totheemotionalqualityofauditoryinformation:Comparedtoneu-
lHG(PAC) 51−187 2349 4.26(3.17)
rHG(PAC) −51−187 1431 3.77(3.16) tral,BOLDsignalshadahigherintensityduringthejoycondition,and
rsupramarginalgyrus 63−2434 89,235 −4.97(−3.22) alowerintensityduringthefearcondition.Inotherwords,weobserved
cuneus(area17,18) 9−664 19,764 −3.99(−3.07) anactualincreaseinBOLDactivityduringlisteningtojoystimuliandan
lpost.MTG/ITS −60−607 2646 −4.19(−3.17) actualdecreaseduringlisteningtofearstimuli(comparedtoneutral
lanteriorinsula −4291 7263 −4.52(−3.16)
stimuli).Withregardtothepronouncedregionalactivityintheaudito-
lmid-insulaa −41−514 – −3.5(–)
lpost.insulab −36−2011 – −3.68(–) rycortexduringthejoy-evokingmusic(asindicatedbytheGLMs),itis
(b)Rightauditorycortex likelythatthiswasinpartduetoamoredetailedacousticalanalysisof
lSFS −243052 2997 −4.07(−3.19) the joyful music, which was probably related to a voluntary shift of
lSFS −245125 756 −3.92(−3.04)
attention:participantshadapreferenceforthejoystimuli(asindicated
rSFS 274237 3672 −3.76(−3.05)
lant.insula/deepFOp −45121 648 −4.06(−3.27) bythevalenceratings),andthereforeprobablypaidmorevoluntaryat-
lmid-insula −34316 459 −3.48(−3.05) tentiontothosestimuli,leadingtoastrongerauditorycortexactivation
rant.insula&putamen 27127 2295 −4.35(−3.17) (Jänckeetal.,1999).Similarfindingshavepreviouslybeenreportedfor
lplanumtemporale −60−247 2970 4.34(3.22) pleasantcomparedtounpleasantmusic(Koelschetal.,2006;Muelleret
rplanumtemporale 60−184 1188 3.68(3.13)
al., 2011) or pleasant vs. unpleasant sounds from the International
rsupramarginalgyrus 63−4237 3429 −4.65(−3.17)
lpost.MTG/ITS −60−667 3240 −4.40(−3.24) AffectiveDigitizedSoundSystem(IADS,Plichtaetal.,2011).However,
rpost.MTG 54−5713 5427 −3.81(−3.06) it is unlikelythatmerelypreference(and,correspondingly, volun-
pre-genualcingulate −6427 2268 −3.64(−2.98) taryshiftsofattention)explainsthiseffect,becausethepreference
cingulatesulcus 31543 3888 −4.38(−3.13)
of participants was comparable between fear and neutral music
post.cingulatesulcus 6−3049 33,264 −5.40(−3.18)
(c)Leftsuperficialamygdala (again,asindicatedbythevalenceratings),andyetBOLDsignalin-
rplanumtemporale/post.HG 42−3313 567 3.75(3.05) tensitydifferedbetweenfearandneutral.
rSPL(area7) 18−4854 351 −4.13(−2.94) Theroleoftheauditorycortexintheemotionalprocessingofau-
lSPL(area7) −21−5154 756 −4.56(−3.21) ditoryinformationisfurtherhighlightedbythePPIresultsinvolving
rsup.occipitalgyrus(area18) 26−9520 7020 −3.68(−2.87) auditoryseedregions:Theseresultsrevealedemotion-specificfunc-
lmiddleoccipitalgyrus −30−7819 11,043 −4.24(−2.99)
llingualgyrus(V4) −18−73−5 675 −2.94(−2.69) tionalconnectivity(a)betweenauditorycorticalareasandcingulate,
rlingualgyrus(V3&V4) 18−81−8 648 −2.85(−2.67) aswellasinsularcortexduringjoystimuli,and(b)betweenauditory
(d)Rightsuperficialamygdala areasandparietal,aswellasvisualcortex(V1–V5)duringfearstimuli.
MDthalamus 3−124 297 3.22(2.93) Bothcingulateandinsularcortexareinvolvedinemotionalprocesses,
a Thepeakvoxelinthelmid-insulawaspartoftheclusterwiththemaximumpeak inparticularwithregardtoautonomicregulationaswellastheproduc-
voxelinthelant.insula. tionofsubjectivefeelings(Craig,2009;MedfordandCritchley,2010).In
b Thepeakvoxelinthelpost.-insulawaspartoftheclusterwiththemaximumpeak
addition,thecingulatecortexhasbeenimplicatedinthecoordinationof
voxelinthelant.insula.
autonomic activity, behavior, motor expression, as well as cognitive
processes in response to emotionally salient stimuli (Koelsch et al.,
thisproceduredidnotcanceloutacousticaldifferencesbetweencondi- 2010b;MedfordandCritchley,2010).
tions,joyandfearstimulididnotdifferwithregardtotheirintensity, Withregardtothemarkedfunctionalconnectivitybetweenaudi-
mean F0 frequency, variation of F0 frequency, pitch centroid value, tory areas and parietal as well as visual cortex, anatomical studies
spectralcomplexity,andspectralflux.(3)F0salienceandchordstrength indicatethatcore,beltandparabeltregionsprojecttoV1andV2ofvi-
differedsignificantlybetweenjoyandfearstimuli,aswellasbetweenjoy sualcortex,andthatneuronsinV2projectbackintotheseauditory
andneutralstimuli(F0saliencewashighestforneutral,intermediatefor regions (reviewed in Smiley et al., 2007). The observed functional
joy,andlowestforfearstimuli;chordstrengthwashighestforjoystim- connectivitybetweentheseareasinthepresentstudyhighlightsthe
uli,anddidnotdifferbetweenneutralandfearstimuli).Nevertheless,in role of auditory–visual interactions, in particular during emotional
theGLM,BOLDsignalintensityintheauditorycortexwasstrongerinre- statesoffear.Thefunctionalsignificanceofsuchinteractionsisprob-
sponsetojoycomparedtoneutral,andduringneutralcomparedtofear ablyincreasedvisualalertnessinthefaceofdangersignaledbyaudi-
stimuli;thispatterndoesnotcorrelatewiththepatternofF0salience toryinformation(probablyincludinginvoluntaryshiftsofattention).
(beingstrongest for neutral stimuli) or the pattern of chord strength Ourresultsarethefirsttoshowthattheauditorycortexisacentral
(which did not differ between neutral and fear stimuli). (4) Key- hubofanaffective-attentionalnetworkthatismoreextensivethan
strength showed differences between joy and fear, as well as be- previously believed, involving functional connectivity of auditory
tween fear and neutral stimuli, but not between joy and neutral associationcortexwithadiverserangeofvisual,attentional,andlimbic/
stimuli.Again,thispatternisnotconsistentwiththepatternofBOLD paralimbicstructures.Thisfindingalsosupportsthenotionthatmultisen-
responsesobservedintheauditorycortex.Althoughnotwellknown, sory interactions in the cerebral cortex are not limited to established
itishighlylikelythatextractionofthekeyoftonalmusic(including polysensoryregions,butthat“interactionswithothersensorysystems
extraction of a tonal center) involves both posterior and anterior alsotakeplaceinauditorycortex”(Smileyetal.,2007).Notably,thislatter
supratemporalcortexbilaterally(e.g.,Koelsch,2011;Liegeois-Chauvel conclusion holds even if such multisensory interactions were due to
etal.,1998;Pattersonetal.,2002;PeretzandZatorre,2005).Therefore, acoustical features which were possibly not accounted for by the