Are 6-month-old human infants able to transfer emotional information (happy or angry)

17 Pages·2017·3.54 MB·English

RESEARCHARTICLE
Are 6-month-old human infants able to
transfer emotional information (happy or
angry) from voices to faces? An eye-tracking
study
AmayaPalama1,JenniferMalsert1,2,EdouardGentaz1,2,3*
1 SensoriMotor,AffectiveandSocialDevelopmentLaboratory,FacultyofPsychologyandEducational
Sciences,UniverstyofGeneva,Geneva,Switzerland,2 SwissCenterforAffectiveSciences,Campus
Biotech,UniversityofGeneva,Geneva,Switzerland,3 CNRS,Grenoble,France
a1111111111 *[email protected]
a1111111111
a1111111111
a1111111111 Abstract
a1111111111
Thepresentstudyexaminedwhether6-month-oldinfantscouldtransferamodalinformation
(i.e.independentlyofsensorymodalities)fromemotionalvoicestoemotionalfaces.Thus,
sequencesofsuccessiveemotionalstimuli(voiceorfacefromonesensorymodality-audi-
tory-toanothersensorymodality-visual-),correspondingtoacross-modaltransfer,were
OPENACCESS
displayedto24infants.Eachsequencepresentedanemotional(angryorhappy)orneutral
Citation:PalamaA,MalsertJ,GentazE(2018)Are
6-month-oldhumaninfantsabletotransfer voice,uniquely,followedbythesimultaneouspresentationoftwostaticemotionalfaces
emotionalinformation(happyorangry)from (angryorhappy,congruousorincongruouswiththeemotionalvoice).Eyemovementsin
voicestofaces?Aneye-trackingstudy.PLoSONE
responsetothevisualstimuliwererecordedwithaneye-tracker.First,resultssuggestedno
13(4):e0194579.https://doi.org/10.1371/journal.
differenceininfants’lookingtimetohappyorangryfaceafterlisteningtotheneutralvoiceor
pone.0194579
theangryvoice.Nevertheless,afterlisteningtothehappyvoice,infantslookedlongeratthe
Editor:JordyKaufman,SwinburneUniversityof
incongruentangryface(themouthareainparticular)thanthecongruenthappyface.These
Technology,AUSTRALIA
resultsrevealedthatacross-modaltransfer(fromauditorytovisualmodalities)ispossible
Received:July12,2017
for6-month-oldinfantsonlyafterthepresentationofahappyvoice,suggestingthattheyrec-
Accepted:March6,2018
ognizethisemotionamodally.
Published:April11,2018
Copyright:©2018Palamaetal.Thisisanopen
accessarticledistributedunderthetermsofthe
CreativeCommonsAttributionLicense,which
permitsunrestricteduse,distribution,and
Introduction
reproductioninanymedium,providedtheoriginal
authorandsourcearecredited. Expressingemotionsviafacialexpressions,voicesorevenbodymovementshelpstotransmit
one’sinternalstateandintentionstoothers[1].Humaninfantsareabletorecognizeemotions
DataAvailabilityStatement:Allrelevantdataare
withinthepaperanditsSupportingInformation expressedbythepeopleintheirenvironment(parents,brothersandsisters,etc.),thisadaptive
files. abilityisessentialforinfantstointeractwiththesepeople[2].However,perceivingemotional
expressionsisnottrivialforinfantsandthedevelopmentofthisabilitydependsonthetypeof
Funding:Thisresearchwassupportedbythe
SwissNationalFundfortheresearchgrant emotionsexpressedandtheirmodeofpresentation[3–5].
100019-156073awardedtoE.G. Thespontaneousvisualpreferenceforhappyfaces,observedinspecificconditionsinnew-
borns[6,7]generallypersistsuntil5monthsofageandseemstodeclineafterthat.Morepar-
Competinginterests:Theauthorshavedeclared
thatnocompetinginterestsexist. ticularly,at3monthsold,theamountoftimeinfantslookatahappyfaceisgreaterthanthe
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 1/17
Cross-modaltransferofemotionsininfants
amountoftimetheyspendlookingataneutralone[8].Additionally,at4monthsold,infants’
firstfixationsaremoreoftendirectedtowardhappyfacesthanneutralfaces[9].Nevertheless,
resultsshowthatthisvisualpreferenceisinfluencedbyotherfacialdimensions,forexample,
thepreferenceforhappyfacesislimitedtofemalefacesin3.5-month-olds[10].Thisdifference
maybeexplainedbythedifferentexperienceswithmaleandfemalefacesacquiredoverthe
firstfewdaysoflife[11].Althoughthispreferenceforhappinessisnotreportedafter5months
[12],itmaystillbeobservedinsomecasesat7months[13,14].
Thevisualdiscriminationbetweenhappinessandotherexpressionsisdemonstratedfrom2
to5months[3].Discriminationbetweensurprised[15,16]andangry(frowning)faces[17]
occursat3months,andbetweenhappyandsadfacesat3–5months[18].At5months,infants
areabletodiscriminatebetweenhappyandneutralfaces[19],aswellasbetweenhappyand
fearfulfaces[20].Studieshaveshownacategoricaldiscriminationbetweenhappinessandsev-
eralotheremotions(surprise,sadness,fear)for6–7month-oldchildren(forreviews[3–5]),as
demonstratedbyidentity-invariantcategorization(i.e.infantscancategorizetheemotionpre-
sentedbyadifferentidentityasthesameemotion)e.g.[5,15,18,21,22]foraudio-visualstimuli
andcategoricalboundaryeffects(i.e.inemotionalmorphing,thepointinthecontinuumof
emotionalexpressionwhentheinfantperceivedthefaceasaspecificemotionalcategory)(e.g.
[23]).However,noevidenceforavalence-basedcategorizationofexpressions(i.e.thecatego-
rizationbetweenthesameemotionalvalence,positiveornegative)wasfoundin7-month-old
infants[24].Overall,thepositivediscriminationforspecificcontrastsappearsearlierinpara-
digmsinvolvingjustoneoralimitednumberoffaceidentities(e.g.[17]),andlaterinpara-
digmsinvolvingtheextractionofexpressionsacrossmultipleidentities[15].
Areviewofthestudiesregardingtheperceptualdevelopmentofemotionalexpressionssug-
geststhatthesensorymodeinwhichastimulusispresented,whetheritbeunimodalormulti-
modal,playsasignificantroleinaninfant’sabilitytodiscriminateemotions[3].Forexample,
FlomandBahrick[25]showedthatinfantscandiscriminateamonghappiness,angerandsad-
nessasof5monthswithunimodalauditorystimuliandasof7monthswithunimodalvisual
stimuli.Furthermore,at4months,infantsareabletodiscriminateamonghappiness,anger
andsadnesswithmultimodaldynamic(audio–visual)stimuli,i.e.whenthesoundsandthe
emotionalfacesareshownsimultaneouslyandsynchronized.Moreevidenceofmultimodal
matchinghasbeenreportedin3–4month-oldinfantsforaudio-visualmatchingofhappiness
versussadness(concordant>discordant)andhappinessversusanger
(discordant>concordant)expressionsofthemother[26],aswellasforvisual-olfactory
matchingofhappyversusdisgustedexpressions[27].However,thevisual-olfactorymatching
appearslimitedtothehappyexpressionandisnotpresentat5months.Evidenceofaudio-
visualmatchingalsoexistsforhappyandangry(concordant>discordant)expressionsat6–7
months[13,28].
Itshouldbenotedthatmostofthepreviousbehavioralstudiesusedvideosinindividual
testingsessions.Generally,theexperimentersmanuallycodedtheinfant’sgazeasbeingeither
totheleftside,therightside,oroutsideofthescreen—generatingraw-lookingdata.Thisanal-
ysisprocedureisnotveryaccurateanddoesnotallowforanexaminationofthespecificface
areas(eyesandmouth)exploredbyeachinfantinfunctionoftheconditions.Fortheaimof
thisstudy,werecordedtheeyemovementsthatoccurredinresponsetothevisualstimuliin
eachofthe6testphasesusinganeye-tracker.Therehavebeenfewstudiesthathaveexamined
ocularmovementwithaneye-trackingdeviceininfants.However,eye-tracking,whichpre-
ciselycalculatesthetimeanddirectionofthegaze,allowsforspatialandtemporalprecision
andaccuracy.Besidesfixationandsaccades,theeye-trackerallowsonetoexaminespecific
areasofinterest(AOIs)onthestimuluspresented,suchastheeyesandmouth.Dependingon
thetypeofemotion,someregionsofthefacemaybemoreusefulthanothersinhelpingan
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 2/17
Cross-modaltransferofemotionsininfants
infanttodetermineanemotion.Schurginetal.’sstudy[29]showsthatbyobservinganadult’s
eye-movementsonapicture,onecanpredicttheemotionthatispresentedonit.
Arecentstudy[30]usingeye-trackinganddynamicemotionalfaceswithinfantsagedfrom
3to12months,showedthatyoungerinfantsfocusedtheirattentionontheeyesandthe
externalfeaturesofemotionalfaces.However,thevisualattentionofolderinfants(7-and
12-month-olds)dependedontheemotionthatwasdisplayed.Inthisstudy,themouthdrew
themostattentionforsmilingfaces,theeyesandeyebrowsdrewthemostattentionforfearful
andangryfaces,andtheuppernoseareadrewattentionfordisgustedfaces.Anotherstudyby
thesameauthors[31]demonstratedthat7-month-oldinfantslookedlongeratareasofinterest
ofaneutralfaceaccordingtothevalenceoftheodorssmeltbefore.Withapleasant(straw-
berry)scent,theinfantslookedmoreattheneutralface,particularlytheeyes,eyebrows,nose
andmouthareas,whereaswithanunpleasant(strongcheese)odor,theylookedmoreat
theuppernosearea.Asafunctionoftheinternalstatesprovokedbythesmell,theinfants
searchedforreactioncuesonthefacespresented.Amsoetal.[32]foundapositivecorrelation
betweenthetimespentlookingattheeyeareaandtheabilitytodiscriminatebetweenhappy
andfearfulexpressionsafterhavingbeenhabituatedtofearfulexpressionsat6to11months
old.Inanotherstudy,Hunniusetal.[33]showedthat4-and7-month-oldinfantslookedless
atinnerfeaturearea(mouth,eyeandnose)ofthreat-relatedexpressions(angerorfear)com-
paredtonon-threat-relatedexpressions(happy,sadorneutral).
Nonetheless,theexistenceoftheabilitytodiscriminateemotionalexpressionsinunimodal
ormultimodalconditionsdoesnotallowustodeterminewhetheritresultsfromanamodal
representationoftheemotionorfromasensitivitytospecificperceptualfeatures,whether
theybevisualand/orauditory.Somestudiesshowedthatinfantsusecuessuchasthesalience
ofteethat4months,ratherthanemotions,whencomparingtwoemotionalfaces[34].The
findingsofthebehavioralstudieswhichusedexperimentalparadigmsinvolvingjustoneora
limitednumberoffaceidentitiesdonotprovethatinfantsareunabletoformemotional
representations.However,theyconfirmthatsensitivitytoperceptualvariablescontributesto
infants’performancesinmanyexperimentsdesignedtoassesssensitivitytoemotion.
Arelevantwaytoruleoutthisdifficultyistostudytherecognitionofemotionalexpression
inacross-modaltask(forreview[35]).Thesedatagaveevidencethatinfantscancodeinfor-
mationinanauditoryortactilemodeandthenperceivethisinformationinavisualmode,
despiteseveraldifferencesinsize,volume,texture,shape,etc(suchasnumber[36]orobject
unity[37]).Inthisperspective,asimilarwaytoaddressthequestionofamodalrepresentation
ofemotionswouldbetoinvestigatecross-modalemotionalcorrespondencefromauditoryto
visualemotionalstimuli.
Theaimofthispresentstudyistoevaluateiftheabilitytodiscriminateemotionalexpres-
sionsisfoundedonthenatureofemotionperse,amodally(i.e.independentlyofsensory
modalities)oronspecificphysicalcharacteristicsofstimuli(facesorvoices).Tohelpanswer
thisquestion,wechosetouseaparadigmwithasuccessivecross-modaltransferfromemo-
tionalvoicestoemotionalfaces.Toourknowledge,nosuchexperimenthasbeenconducted
oninfants.Thiscross-modaltransferofemotionalinformationfromauditorytovisualcon-
sistsoftwosuccessivephases:anauditoryfamiliarizationphaseandthenavisualtestphase.
Thistaskisverydifficultbecauseitinvolvesaserialmappingprocessinwhichemotional
informationisextractedinanaudioformatandtransformedintoavisualformat.Thus,if
infantsareabletosuccessfullytransfertheemotionalinformation,itwouldsupportthe
hypothesisthattheyareabletorecognizetheemotionsamodally,notsimplyviaphysicalfea-
tures(pictorialoracoustic).Thestudiesshowingthecategoricaldiscriminationofemotions
(i.e.theextractionofexpressionsacrossmultipleidentities)alsosupportthehypothesisthat
infantsareabletoformamodalemotionalrepresentations.
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 3/17
Cross-modaltransferofemotionsininfants
Ourexperimentconsistsofsixsequencesofcross-modaltransfersthatwereindividually
showntoeachinfant.Thisstudybeganwithabaselineconditioninwhichaneutralvoicewas
presentedfor20secondsduring2trials,followedbythetwoemotionalfaces(happyand
angry)presentedsimultaneouslyfor10seconds.Thegoalwastoobtainthebaselineofany
spontaneouspreferencesofthelookingtimebetweenhappyandangryfaces.Thiscontinued
withtheexperimentalconditionsinwhichinfantsreceivedfourdifferentsequencescorre-
spondingtoanemotionalvoice(happyorangry)presentedfor20seconds(auditoryfamiliari-
zationphase),followedbythetwoemotionalfacesbeingpresentedsimultaneously(one
familiarandtheothernovelvis-à-vistheemotionalvoice)for10seconds(visualtestphase
withoutanysound).
Wehypothesizedthatifinfantshadanamodalrepresentationofemotion,theywouldbe
abletodetectthecorrespondencebetweenanemotionalvoiceandavisualfacecontainingthe
sameemotion.Inthiscase,areactiontonoveltywasexpected:i.e.alongerlookingtimefor
thenon-matchingface.Thus,weexpectedthatinfantswouldpreferthenovelface.Further-
more,duetothefactthathappinessisthefirstemotioninfantsareabletodiscriminate,we
expectedthehappyexpressiontobebettertransferredthantheangryone.
Additionally,weexaminedwhethervisualpreferenceisdependentonspecificareas,such
astheeyesand/ormouthofeachofthefaces,aftertheauditoryfamiliarization.Interestingly
enough,theresultsfromtwodifferentinfantstudiesregardingthefaceareaslookedatinfunc-
tionoftheemotionpresentedprovidedcontradictoryresults.Onestudy[30]showedthat
infantslookedlongeratthemouthareaforhappyfacesandattheeyesforangryfaces.How-
ever,theotherstudy[33]showedthatinfantslooklongeratthemouthfortheangryfaces.Evi-
dently,infantsseemtobedrawntothesetwoareaswhenpresentedwiththeseemotionalfaces.
Therefore,weexaminedthemouthandeyeareasforbothoftheemotionalfaces.Inaddition,
toexplorethegazefurther,wealsoexaminedthefirstfixationsofeachvisualtestphasefor
eachinfantandpeaklooksattheface,themouthandtheeyesforeachemotionalface.
Finally,wedecidedtoinvestigatetherarelyanalyzedgendereffectduetothefactthatcon-
tradictoryeffectshavebeenreportedinpreviousexperiments.Ofthosethatstudiedthiseffect,
twodidnotobserveanydifferencesbetweenmalesandfemales[38,39]whileoneobservea
significantdifferencebetweenfemalesandmalesinemotionrecognition,demonstratingthat
5-month-oldgirlsrecognizedemotionssimilarlyto6-month-oldboys[15].
Method
Participants
Twenty-fourfull-term(atleast37weeksofgestation)6-month-oldinfants(13females;mean
age=6.03months±0.32,range=6.5–5.2months)wereincludedinthefinalsampleofthe
study.Becauseofthedifficultytoapplytheeye-trackingtechnictoinfants,agreatnumberof
datahasbeennotrecorded.Thirty-oneadditionalinfantswereobservedbutexcludedfrom
thefinalsampleduetotechnicalfailureoftheeye-trackingnotbeingabletofindthepupil
(seventeen),excessivemovement(two)resultinginlossofgazedata,noisyeyetrackingdata
duetounsuccessfulcalibration(three)definedasmorethan2˚ofdeviationinthexandy
axes,inattentivenesstostimuli(lookingatthescreenlessthanathirdoftheentiretime)(one),
crying(four)orfussiness(four).Thedescriptivecharacteristicsofthefinalsampleareasfol-
lows:themeanageofthemotherswas33.01(±4.6)yearsand35.56(±5.9)yearsforthe
fathers.Themajorityoftheparentsthatparticipatedinthestudyweremarried(N=14)or
cohabitating(N=9),whileoneparentwasasinglemotherraisingherchildalone(N=1).The
familys’socioeconomicstatus(SES)wascalculatedusingtheLargoscalebasedonpaternal
occupationandmaternaleducation,rangingfrom2(thehighestSES)to12(thelowestSES)
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 4/17
Cross-modaltransferofemotionsininfants
[40].Themeansocioeconomicstatus(SES)ofthefamiliesusedinthesamplewas3.42±1.47,
range=2–8.ApprovalforthestudywasgivenbytheEthicsCommitteesoftheFacultyofPsy-
chologyandEducationalSciencesofGenevaandallparentsgavewritteninformedconsentfor
theirchildren’sparticipationintheexperiment.Theexperimentwasperformedinaccordance
withtherelevantguidelinesandregulations.
Stimuli
Theemotionalnonverbalauditorystimuliofhappiness,angerandneutralcomefromthe
"MontrealAffectiveVoice"database[41].Theyareexpressiveonomatopoeicstimulibasedon
theemissionofthevowel/a/.Thisauditorystimuluswasaloopofaonesecondvoicewitha
breakof1secondbetweeneachrepetitionforatotalclipof20seconds.Notethatthesearethe
vocalproductionsofonlyonewoman(ref:SF60).Thevolumeofauditorystimulipresentedto
babiesdidnotexceed60dBA.
Thevisualstimuliusedwereemotional(happyandangry)facesofawomantakenfromthe
database"TheKarolinskaDirectedEmotionalFaces—KDEF"[42].Thesepicturesare9.1x9.1
cm,inblackandwhite,andarepresentedonamediumgraybackground(RGB100,100,100).
Thehairisnotvisibleonthestimulitoavoidpotentialbiasesofattentionontheexternalele-
mentsoftheface[43].Becausestudiesshowedthat4-month-oldinfantsdiscriminatefemale
facesmoreeasilythanmalefaces[10],wetestedtheemotionalfacesrepresentedbythesame
woman(ref:SF4).Facesarepresentedinpairs,pseudo-randomizedfortheleftandrightpre-
sentation(Fig1).
Experimentalprocedure
Eachinfantwascomfortablyinstalledinasuitableseat,placedinanexperimentalcubiclein
Geneva’sBabyLab.Thestimulusdisplayscreenmeasured47.5cmx30cmwithaspatialreso-
lutionof1680x1050pixels.Thebabywasplacedatadistanceof60cmfromthescreen,atthis
Fig1.Visualstimuli.Theangryface(right)andthehappyface(left)withfacesfromTheKarolinskaDirected
EmotionalFaces—KDEF.
https://doi.org/10.1371/journal.pone.0194579.g001
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 5/17
Cross-modaltransferofemotionsininfants
distance,visualstimuliwere8.7˚ofvisualangle.Tofocustheinfant’sattentiononthescreen,
justbeforestartingtheexperiment,wepresentedacartoonextractedfrom“LeMondedes
petits”.Thegazeonvisualstimuliwasrecordedwithaneye-trackerSMIRED250(SensoMo-
toricInstrumentsGmbH,Teltow,Germany).
Theexperimentstartedwitha5-pointcalibrationphasewiththeeye-tracker,ananimated
imageat5differentlocationscoveringthewholesurfaceofthescreen.Thisphasewasrepeated
untilasatisfactorycalibration(lessthan2˚ofdeviationinthexandyaxes)wasachieved.
Inthisexperiment,eachtrialconsistedofexposuretoavoice(neutral,happyorangrypros-
ody)for20secondsaccompaniedbyablackdisplayscreen,foranauditoryfamiliarization
phase.Afterwards,apairofemotionalfaces(happyandangry)waspresentedfor10seconds
duringthevisualtestphase.Thesideofpresentationofthehappyandangryfaceswerecoun-
terbalancedforeachvoice.
Theexperimentwascomposedof6trialsinthisorder:first,inordertoobtainthebaseline
ofspontaneouspreferencesforinfants,aneutralvoicewaspresentedduringthefirst2trials,
followedbythe2emotionalfaceswhichwerelaterallycounterbalanced.Thenext4trials,the
testtrials,consistedofthepresentationofthe2emotionalvoices,eachfollowedbythe2emo-
tionalfaces,laterallycounterbalancedforeachemotionalvoice,insuccession.Thehappyvoice
waspresentedfirst,toavoidthetriggeringofanegativereactionbythenegativestimulus[44].
Thepresentationofthe6trials(sequencesofaudio-visualtransfer)lasted3minutesforeach
infant(Fig2).
Dataanalysis
AllthedatawereextractedbyusingBegazeSMI’sanalyzersoftware.Thetotallookingtimein
secondstothewholefaceandtotheAreasofInterest(AOI)wascalculatedbythenetdwell
time(lengthoftimespentlookingtheAOIs).WedefinedAOIsasonegeneralforthewhole
face(Fig3)andtwospecificonesfortheeyesandthemouth(Fig4)foreachtypeofemotional
expression(datainS1Dataset).Peaklookdurationwascalculatedinmillisecondsasthelon-
gestunbrokenlookatthescreenforthesame3AOIsineachemotionalface(datainS2Data-
set).Weperformedrepeatedmeasuresanalysisofvariance(ANOVA)onthewholefaceand
specificAOIlookingtimesandpeaklooks.Theproportionoffirstfixationstowardthefacesof
eachtrial(24infantsx2trialsbyvoice=maximum48firstlooksforeachvoice)werealsoana-
lyzedwithT-test(datainS3Dataset).StatisticalanalyseswereconductedusingStatistica13.
Thesignificancethresholdwas.05andBonferronitestwasperformedtodeterminesignificant
differences,effectsizesaregiveninpartialeta-squaredηforANOVAs.
Results
Baselinecondition
Table1presentstheresultsofthebaselineconditionforthelookingtimeatthewholefaceand
AOIs(mouthandeyes)aswellasthefirstfixationsforthehappyorangryfacepresentedafter
theneutralvoice.Wefoundnosignificantdifferenceconcerningthelookingtimeattheemo-
tionalfacesF(1,23)=3.135,p=.09,η=0.12,thefirstfixationsatfaces(t(47)=0.58,p=.56;
singleStudent’sT-test)andthepeaklooks(F(1,23)=2.02,p=.168,η=0.08).Nodifference
concerningthelookingtimeattheemotionalAOIsofthemouthF(1,23)=2.89,p=.103or
eyesF(1,23)=0.15,p=.701.However,withthepeaklooks,wefoundanemotionalfaceeffect
(F(1,23)=5.34,p<.05,η=0.18)suggestingangryAOIstriggeredalongerfixation(420±67
ms)thanthehappyones(307±39).Evenmore,wefoundasignificantinteractionbetween
emotionalfacesandAOIs(F(1,23)=5.59,p<.05,η=0.19).Apre-plannedcomparison
showedthattheangrymouth(523±125ms)seemedtoinvolvelongerfixationsthanthe
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 6/17
Cross-modaltransferofemotionsininfants
Fig2.Schematicrepresentationofthesuccessivepresentationofallstimuli.
https://doi.org/10.1371/journal.pone.0194579.g002
happymouth(277±61ms)(F(1,23)=6.93,p<.05)whileboththeangryandhappyeyes
werelookedatequally(F(1,23)=0.16,p=.688.Theseresultsareinaccordancewiththe
resultsafterBonferronicorrections;onlytheangrymouthseemedtotriggerlongerfixations
thanthehappymouth(p=.03).
Preliminaryanalysesaboutthegendereffectonlookingtimes
A2(emotionalvoicefamiliarizationcondition:angryorhappy)x2(gender:maleorfemale)x
2(emotionalface:happyorangry)ANOVAwasperformedonthelookingtimeswiththe
voiceconditionsandemotionalfacesasawithin-subjectsfactorandgenderbetween-subject
factors.Thegendereffectwasnotsignificant(F(1,22)=.36,p=.56,η=.02)anddidnotinter-
actwithotherfactors(allp>.05).
A2(emotionalvoicefamiliarizationcondition)x2(gender)x2(emotionalface)x2(AOIs:
mouthoreyes)ANOVAwasperformedonthelookingtimes,withtheemotionalvoicecondi-
tions,AOIs,andtheemotionalfacesasawithin-subjectsfactorandgenderasabetween-
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 7/17
Cross-modaltransferofemotionsininfants
Fig3.Areaofinterestrepresentingthewholeface.Theangryface(right)andthehappyface(left).FacesfromTheKarolinskaDirectedEmotional
Faces—KDEF.
https://doi.org/10.1371/journal.pone.0194579.g003
subjectsfactor.Thegendereffectwasnotsignificant(F(1,22)=.47,p=.50,η=.02)anddid
notinteractwithotherfactors(allp>.05).Consequently,resultswerefurthercollapsedacross
gender.
Mainanalyses:Lookingtimes,firstfixationsandpeaklookatwholefaces
andAOIs
Table2presentstheresultsofthevisualtestphaseforthelookingtimeatfaces,theAOIs,the
visualpreferencesoftheinfants,andtheirfirstfixationsforthehappyorangryfacepresented
aftertheemotionalvoices(angryorhappy).
A2(emotionalvoicefamiliarizationcondition:angryorhappy)x2(emotionalface:happy
orangry,Fig5)ANOVAwasperformedonthelookingtimes,withthevoiceconditionsand
emotionalfaceasawithin-subjectsfactor.
Theemotionalvoicefamiliarizationconditionwasnotsignificant(F(1,23)=1.51,p=.23,
η=.06).Theeffectoftheemotionalfacewassignificant(F(1,23)=7.42,p<.05,η=.244),
withaclearvisualpreferencefortheangryface(mean±s.e.m.;seconds,2.76±0.19s.)com-
paredtothehappyface(2.35±0.22s.)presented.Theinteractionbetweentheemotionalvoice
familiarizationconditionandtheemotionalfacewasnotsignificant(F(1,23)=1.43,p=.24,
η=.058).Nevertheless,accordingtoIacobucci[45],itispossibletoexaminetheeffectofa
non-significantinteractiongivencertainconditions.Hestatedthatifasimpleeffectissignifi-
cant,wecanexploreitseffectonthesecond,non-significantone.Underthesecircumstances,
wecanexploreourapriorihypotheses.Therefore,pre-plannedcomparisonsshowthatinfants
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 8/17
Cross-modaltransferofemotionsininfants
Fig4.Areaofinterestrepresentingtheeyesandthemouth.Theangryface(right)andthehappyface(left).FacesfromTheKarolinskaDirected
EmotionalFaces—KDEF.
https://doi.org/10.1371/journal.pone.0194579.g004
Table1. Resultsinthebaselineconditionofthevisualtestphaseanalyses.
Bothfaces Angryface Happyface Valuetest Pvalue
mean±s.e.m mean±s.e.m mean±s.e.m
% % %
Neutralvoice Lookingtimeatfaces 6.75±0.53 3.75±0.29 3.00±0.23 F(1,23)=3.135 .09NS
68% 56% 44%
Peaklooksatfaces 691±81 389±51 301±30 F(1,23)=2.02 .168NS
LookingtimeatAOIs:
Mouth 1.51±0.36 0.94±0.19 0.57±0.17 F(1,23)=2.89 .103NS
22% 62% 38%
Eyes 1.43±0.24 0.74±0.12 0.69±0.12 F(1,23)=0.15 .701NS
21% 52% 48%
PeaklooksatAOIs:
Mouth 800±186 523±125 277±186 F(1,23)=5.34 .014(cid:3)
Eyes 653±125 317±57 336±68 F(1,23)=5.34 .689NS
FirstfixationsatfacesNumber;%(Ntot=48) 46;96% 121;46% 25425;4% t(47)=0.58 .56NS
Infants’mean±standarderrorandpercentagelookingtime(s)andmean±standarderrorofpeaklooks(ms)atfacesandtoAOIsandnumberandpercentageoffirst
fixationsforthehappyorangryfaceaftertheneutralvoice.
(cid:3)p<.05,
NS=NonSignificantresult.
https://doi.org/10.1371/journal.pone.0194579.t001
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 9/17
Cross-modaltransferofemotionsininfants
Table2. Resultsofthevisualtestphaseanalyses.
Bothfaces Angryface Happyface Valuetest Pvalue
mean±s.e.m mean±s.e.m mean±s.e.m
% % %
Happyvoice Lookingtimeatfaces 5.33±0.52 3.04±0.24 2.29±0.28 F(1,23)=4.85 .037(cid:3)
53% 56% 44%
Peaklooksatfaces 796±140 446±79 350±61 F(1,23)=1.30 .265NS
LookingtimeatAOIs:
Mouth 1.21±0.31 0.83±0.20 0.38±0.11 F(1,23)=8.32 .008(cid:3)(cid:3)
12% 69% 31%
Eyes 0.99±0.25 0.44±0.11 0.55±0.13 F(1,23)=0.54 .470NS
9% 45% 55%
PeaklooksatAOIs:
Mouth 589±114 347±51 242±64 F(1,23)=2.43 132NS
Eyes 546±102 243±37 303±65 F(1,23)=1.09 .307NS
Firstfixationsatfaces(Ntot=48) 44;92% 29;66% 15;34% t(47)=2.19 .033(cid:3)
Angryvoice Lookingtimeatfaces 4.91±0.53 2.48±0.24 2.42±0.29 F(1,23)=0.04 .843NS
49% 56% 44%
Peaklooksatfaces 667±101 313±38 354±63 F(1,23)=0.36 .553NS
LookingtimeatAOIs:
Mouth 1.19±0.29 0.72±0.13 0.47±0.16 F(1,23)=2.24 .148NS
12% 60% 40%
Eyes 0.44±0.10 0.46±0.09 0.69±0.12 F(1,23)=0.04 .845NS
10% 49% 51%
PeaklooksatAOIs:
Mouth 740±182 429±74 311±109 F(1,23)=1.21 .285NS
Eyes 492±99 214±46 278±53 F(1,23)=1.06 .313NS
FirstfixationsatfacesNumber;%(Ntot=48) 44;92% 17;36% 27;64% t(47)=1.85 .069NS
Infants’mean±standarderrorandpercentagelookingtime(s)andmean±standarderrorofpeaklooks(ms)atfacesandtoAOIsandnumberandpercentageoffirst
fixationsforthehappyortheangryfaceafterthehappyorangryvoice.
(cid:3)p<.05,
(cid:3)(cid:3)p<.01,
NS=NonSignificantresult.
https://doi.org/10.1371/journal.pone.0194579.t002
lookedatthehappyandtheangryfaceequallyafterhearingtheangryvoice(F(1,23)=.04,p=
.843).Bycontrast,infantslookedlongerattheangryfacethanthehappyfaceafterhearingthe
happyvoice,(F(1,23)=4.85,p<.05)(Fig5).Insum,thelookingtimeforthehappyfaceis
notaffectedbyeitheremotionalvoice.However,thelookingtimefortheangryfaceincreases
afterhearingthehappyvoice.
A2(emotionalvoicefamiliarizationcondition)x2(emotionalface)x2(AOIs:mouthor
eyes,Fig6)ANOVAwasperformedonthelookingtimes,withtheemotionalvoiceconditions,
AOI,andtheemotionalfacesaswithin-subjectsfactors.Fig6presentsthemeansandstandard
errorsoflookingtimesofthevisualtestphasefortheAOIs(mouthandeyes)infunctionof
theemotionalfaces(angryorhappy)aftereachemotionalvoicefamiliarizationcondition
(angryorhappy).
TheeffectofAOIswasnotsignificant(F(1,23)=.56,p=.46η=.024).Infantsseemtohave
lookedatthemouth(mean±s.e.m.;seconds,0.60±0.11s.)andeye(0.47±0.08s.)areasfor
thesameamountoftime.Theeffectoftheemotionalvoicefamiliarizationconditionwasnot
PLOSONE|https://doi.org/10.1371/journal.pone.0194579 April11,2018 10/17