Embodied abstract semantics The representation of abstract words

66 Pages·2010·0.44 MB·English

Embodiedabstract semantics 1
RUNNINGHEAD: Embodiedabstract semantics
Therepresentationof abstract words:
Whyemotionmatters
Stavroula-ThaleiaKousta*
GabriellaVigliocco*
DavidP.Vinson*
MarkAndrews*
ElenaDel Campo^
*UniversityCollege London
^Universita’degli Studi di Padova
Embodiedabstract semantics 2
Abstract
Althoughmuchis known about therepresentation andprocessingof concreteconcepts, our
knowledgeofwhat abstract semantics might beis severelylimited. Inthis paperwefirst address the
adequacyofthetwodominant accounts (dual codingtheoryandthe context availabilitymodel)put
forwardinorderto explainrepresentationandprocessingdifferences betweenconcreteandabstract
words. Wefindthat neitherproposal canaccount forexperimental findings andthat this is, at least
partly,becauseabstract words areconsideredto beunrelatedtoexperiential informationinbothof
theseaccounts. Wethenaddress oneparticulartypeofexperiential information,emotional content,
anddemonstratethat it plays acrucial roleintheprocessingandrepresentationofabstract concepts:
statistically,abstract words aremoreemotionallyvalencedthanconcrete words andthis accounts for
aresidual latencyadvantagefor abstract words, whenvariables suchas imageability(aconstruct
derived from dual codingtheory)and rated context availabilityareheld constant.Weconcludewith
adiscussionofournovel hypothesis for embodied abstract semantics.
Therepresentationof abstract wordmeanings: Whyemotionmatters
Concreteentities exist inspace-time andareindependent ofhumanminds/language; abstract entities,
ontheotherhand,donot exist inspace-timebut theirexistencedepends onhumanminds/language
(Hale,1988). “Concreteness”,therefore,indexes abasicontological distinction,dividingentities into
thesetwokinds. This ontological distinctionis reflectedinourepistemologies, andconcreteness is
arguablyanorganizingprincipleofsemanticknowledge.Uptothepresent,researchintosemantic
andconceptual representation has focused almost exclusivelyonhowconcretewordmeanings and
Embodiedabstract semantics 3
concepts arerepresented andprocessed,totheexclusionofabstract meanings and concepts.
However,theabilityto communicatethroughlanguageabout abstract concepts, suchas courage,
dignity,revenge, lies at theheart ofwhat it means tobehuman,andnotheoryofsemanticor
conceptual representationis completewithout anexplicit account ofhowabstract knowledgeis
acquired, represented, andprocessed.
Inthis paperwefirst demonstrate,combiningexperiments withlargescaleregressionanalyses of
datafrom theEnglish LexiconProject (ELP, Balotaet al.,2007),that thedual codingtheoryandthe
context availabilityhypothesis -twoofthemost popularaccounts ofdifferences inrepresentation
andprocessingbetween concreteand abstract words -donot exhaustivelyaccount forprocessing
(and hencerepresentational)differences between thetwotypes ofwordmeanings. In fact,once
imageabilityand context availability(alongwitha largenumberofotherlexical andsublexical
variables) are controlled, thereis aresidual advantagefor abstract wordprocessing.Weshowthat
this advantagecanbe explainedbydifferences inemotional valencebetweenconcrete andabstract
words, andwediscuss a newhypothesis ofhowthesemanticsystem is organizedwithrespect tothe
distinctionbetweenconcreteand abstract concepts. Specifically,weproposethat bothconcreteand
abstract concepts binddifferent types ofinformation: experiential information(sensory,motorand
affective), andalso linguisticinformation.However,concreteand abstract semanticrepresentations
differinterms of whethersensory,motororaffectiveinformationhavethe greatest weight,with
sensory-motorinformationbeingmorepreponderant forconcreteconcepts andaffectiveinformation
playinga greaterrolefor abstract concepts. Thus, acentral andnovel element ofthis proposal is the
ideathat experiential informationcontributes totherepresentationofboth concreteand abstract
words, however,whereas sensory-motorinformationis statisticallymoreimportant forthe
Embodiedabstract semantics 4
representationofconcretewords, emotional content,alargelyneglectedtypeofexperiential
informationintheliteratureonsemantic representation/processing, contributes toword
representationandprocessing,particularlyfor abstract concepts.
TheConcreteness Effect:Dual-CodingTheoryand theContextAvailabilityModel
It has been demonstrated repeatedly,and withavarietyofmethodologies,that concretewords have a
cognitive advantageover abstract words—anadvantage,labelledthe ‘concreteness effect’.With
respect tolexical processing,earlydemonstrations ofaprocessingadvantageforconcreteover
abstract words wereprovidedbyJames (1975),Whaley(1978), andRubin(1980).James showed
that at least whenlowfrequencywords are considered, concretewords are identifiedas words faster
thanabstract words. Whaley(1978)andRubin(1980)adopteda correlational approach,showing
that thereis asignificant negativecorrelationbetweenconcreteness ratings andlexical decision
reactiontimes forthesameitems. This processingadvantagehas sincethenbeenreplicatedinboth
lexical decision(Binder et al.,2005; Bleasdale,1987; deGroot,1989; Howell & Bryden,1987;
Kroll &Merves, 1986; Schwanenflugel, Harnishfeger &Stowe,1988; Schwanenflugel &Stowe,
1989)andwordnamingtasks (deGroot,1989; Schwanenflugel andStowe, 1989).
Withrespect tomemoryforconcrete andabstract words,it has been againrepeatedlydemonstrated
that concretewords have anadvantageoverabstract words inbothlong-term andshort-term memory
tasks (e.g.paired-associatelearning(Paivio,Yuille,&Smythe,1966); serial recall (Allenand
Hulme,2006; Romani,McAlpine,&Martin,2007; Walker&Hulme,1999),freerecall (Romani,
Embodiedabstract semantics 5
McAlpine,&Martin,2007; terDoest &Semin,2005),reconstructionoforder(Neath,1997); and
recognitionmemory(Fliessbach,Weis, Klaver,Elger&Weber,2006)).
Amongthehandful ofproposals that havebeenput forwardto explainthe‘concreteness effect’,two
havebeenparticularlyinfluential: dual codingtheory(Paivio,1971; 1986; 1991; 2007)andthe
context availabilitymodel (Schwanenflugel andShoben,1983; Schwanenflugel, 1991). Inbothof
theseaccounts, concrete wordrepresentations are assumedtobericherthan abstract word
representations (see also Plaut &Shallice,1993). Accordingtodual codingtheory,concretewords
arerepresentedintworepresentationallydistinct but functionallyrelatedsystems: averbal,linguistic
system and anon-verbal, imagisticsystem.Abstract concepts, ontheother hand,areprimarilyor
exclusivelyrepresentedintheverbal system.The cognitive advantagefor words referringto
concreteconcepts is attributedtothefact that theyhaveaccess toinformationfrom multiplesystems.
Accordingtothe context availabilitymodel,bothconcrete andabstract concepts are representedina
singleverbal codeandneithertherepresentations northeprocesses that operateonthese
representations differfor thetwotypes ofconcepts.Theargument hereis that comprehensionrelies
onverbal context (eithersuppliedbythediscourse orbythecomprehender’s ownsemanticmemory)
inorderto beeffective. Accessingthemeaningofawordinvolves accessinganetworkof associated
semanticinformationand theadvantagefor concretewords arises becausetheyhavestrongerand
denserassociations tocontextual knowledgethan abstract words. Thesetwoproposals have guided
researchon concrete/abstract semantics; results, however,havebeeninconclusive.Themajorityof
recent workis neuroscientificinnature,employingeitherelectrophysiological orneuroimaging
techniques inordertodeterminetheneural bases ofthedistinctionbetween concreteand abstract
words.
Embodiedabstract semantics 6
Aseries ofstudies usingevent-relatedpotentials (ERPs)suggestedcombiningdual-codingtheory
andthecontext availabilitymodel inexplainingtheconcreteness effect (“context-extendeddual
codingtheory”—Holcombet al.,1999; West &Holcomb,2000).ERP studies haveidentifiedtwo
components associated withconcreteness: theN400andalatenegativecomponent peakingaround
700-800milliseconds. Withrespect tothefirst component,all relevant studies havefoundthat
concretewords elicit alargerN400than abstract words (Holcombet al.,1999; Kanske &Kotz,
2007; Kounios &Holcomb,1994; Nittonoet al.,2002; West &Holcomb,2000; vanSchieet al.,
2005).Theobservationthat theeffect has ananteriormaximum but is widelydistributedacross the
scalp(West &Holcomb,2000)andthefailuretofindanystructural overlapbetween concreteness
andvisual object workingmemoryonthat component (vanSchie et al.,2005)has ledtothe
suggestionthat theeffect arises withinaverbal semanticsystem that is commontobothconcreteand
abstract words. This N400component has beenarguedto reflect postlexical processinginasemantic
memorysystem,possiblyinvolvingtheintegration ofsemanticinformation intohigherlevel
representations (Osterhout &Holcomb,1995).Accordingtothecontext availabilitymodel,concrete
words areassumedtohavestronger anddenserinterconnections withother concepts insemantic
memorythanabstract words (Schwanenflugel &Shoben,1983). IntheEEGliterature, concrete
words aresaidtoactivate thesemanticnetworkmoreextensivelythan abstract words andthis
extensiveactivationis reflectedinan amplifiedN400forconcrete words. Thesecond,later,
component is assumedtoreflect thecontributionofmental imageryforconcrete words: it is more
sustainedovertime,peakingat around700or800 ms post-stimulus. It is saidtobeassociatedwith
theretrieval ofmental imageryassociatedwithconcrete words and thus consistent withdual coding
theory.Althoughtheimagery-related component is consistent withdual-codingclaims that imagery
Embodiedabstract semantics 7
has alate effect inprocessing,the greaterN400amplitudeforconcretewords is hardertoreconcile
withcontext availabilityclaims. Thestrongerinterconnections insemanticmemoryforconcrete
words accordingtothemodel leadtofacilitatedintegrationofinformation.TheincreasedN400
amplitudeforconcrete words, however,has beeninterpreted as indexingdifficultyin integrating
appropriateinformation(seeKutas, VanPetten& Kluender,2006 forareview).Sotheextent to
whichEEGdata actuallysupport thecontext-extendeddual codingtheoryis questionable.
Acase foraqualitativedifferencebetweenconcreteandabstract wordmeanings, thus compatible
withdual-codingviews, comes from neuropsychological studies where adoubledissociation
between concreteand abstract words has beenobserved.Althoughcases where concretewords are
betterpreservedinthedamaged/agingbrain arethemost frequentlyreported(e.g.,Coltheart et al.,
1980; Franklinet al.,1995; Katz &Goodglass,1990; Martin&Saffran,1992; Roeltgenet al.,1983;
Warrington,1975),there are cases reportingbetter performanceon abstract overconcrete words
(e.g.,Breedinet al.,1994; Cipolotti andWarrington,1995; Marshall et al,1996; Papagno et al.,
2007; Reillyet al.,2007; Sirigu,Duhamel, &Poncet,1991; Warrington,1975; Warrington &
Shallice,1984).
Intheimagingliterature, althoughabstract wordprocessingseems toinvolveactivations inamore
distributednetwork ofbrainregions thanconcrete wordprocessing(Pexmanet al.,2007),thereis
convergingevidencethat abstract wordprocessingis associatedwithhigher activationinleft
hemisphericareas that areknowntobeinvolvedinsemanticprocessing, e.g,theleft inferior frontal
gyrus (LIFG)(Perani et al.,1999; Jessenet al.,2000; Fiebach andFriederici,2003; Noppeneyand
Price,2004; Binderet al.,2005)andthesuperiortemporo-lateral cortex (Mellet et al.,1998; Kiehl et
Embodiedabstract semantics 8
al.,1999; Wiseet al.,2000; Binderet al.,2005; Binderet al,2009).Withrespect to greater
activation forabstract overconcrete words intheleft inferior frontal gyrus, this findinghas been
interpreted as indicatingmoreeffortful retrieval ofsemanticinformationforabstract words, a
findingthat has beeninterpretedinsomestudies as consistent withcontextavailabilitypredictions.
Again,however,themajorityofthestudies useitems matched onfrequency,but not onfamiliarity
orotherrelevant variables. Forinstance,inoneof theotherwisebest-controlledstudies inthe
imagingliterature(Binderet al.,2005),althoughitems werematchedon frequency,we foundthat
concretewords weresignificantlymorefamiliarthanabstract words (average familiarityratings of
534vs. 471respectively, t(98)=3.956,p<.001). It maywell bethat suchdifferences infamiliarity
underliesomeofthe effects reportedintheneuroimagingliterature.
Whenconcretewords are comparedto abstract words, results havebeenextremelyvariable.
Althoughsomestudies havefound activations ofleft hemispheric regions associatedwithhigher
levels ofvisual processingsuch as theleft fusiform gyrus (D’Espositoet al.,1997; Mellet et al.,
1998; Fiebach & Friederici,2003; Sabsevitz et al.2005),consistent withthedual codingprediction
that concretewordmeanings activaterelevant imagisticinformation,anumberofstudies havefailed
tofindanyregions at all that are activatedmoreduringconcretewordprocessing(Grossmanet al.,
2000; Friederici et al.,2000; Kiehl et al.,1999; Krauseet al.,1999; NoppeneyandPrice,2004;
Perani et al.,1999; Pexmanet al.,2007; Tyler et al.,2001).Somestudies havefoundmorebilateral
activations duringconcretewordprocessing(Binderet al.,2005; Sabsevitz et al.,2005),whileother
studies haveshownthat thereis noright hemisphereinvolvement intheprocessingofconcrete
words, andifanything,there aremoreright-hemisphericactivations forabstract ratherthanconcrete
words (see Fiebachand Friederici,2003 fora review).Oneofthereasons forthelackof consistency
Embodiedabstract semantics 9
intheresults maybethat theconcretewords used withinandacross studies differinterms oftheir
featural composition,whichquitereasonablyleads toactivationofdifferent brainnetworks in
different studies ortolackofconsistent areas ofactivationwithinthesamestudy.Thus, just as the
behavioral andEEGevidence reviewedabove,imagingstudies donot provideclearsupport for
eitherdual-codingorcontext availabilitycallingfornewtheoretical directions andfurther empirical
investigation.
TheConcreteness Effect: TestingDual-CodingTheoryandtheContext-AvailabilityHypothesis
Intheliterature,it is invariablyassumedthat thepsycholinguisticconstructs ofconcreteness and
imageabilitytapinto the sameunderlyingtheoretical construct,i.e.,theontological distinction
between concrete,spatiotemporally-boundconcepts andabstract,non-spatiotemporallybound
concepts. Afterall,when nothingelseis takenintoaccount,imageabilityratings explainmorethan
72%ofthevarianceinconcreteness ratings, and uptonowthevariancethat is not explainedby
imageabilityhas been consideredtobepurenoise,dueperhaps totheimprecisenatureofsubjective
norms. This general assumptionis illustratedinthefollowingquote: “Although imageabilityand
concreteness aretechnicallydifferent psycholinguisticconstructs, thecorrelationbetweenthese
variables is so strongthat manyauthors usetheterms interchangeably. Herewemakethesame
assumptionofsynonymybetweenimageabilityandconcreteness interms oftheory(i.e.,
concreteness effects =imageabilityeffects).”(ReillyandKean,2007: 158). In fact, concreteness and
imageabilityratings have beenusedinterchangeablyinmost oftherecent literatureinthe field(e.g.
Binder et al.,2005; Richardson, 2003; Fliessbach et al.,2006; Giesbrecht et al.,2004).However,
concreteness andimageabilitytapinto,at least partially,different aspects of semanticrepresentations
ifnativespeakerintuitions about them aretakenseriously: our analyses of ratings formorethan
Embodiedabstract semantics 10
4,000words intheMRC PsycholinguisticDatabaseshowthat thefrequencydistributionof
concreteness ratings is bimodal,withtwodistinctmodes forabstract and concrete words (seealso
Cartwright &Nickerson, 1979; Nelson &Schreiber,1992),whilethedistributionofimageability
ratings is unimodal (Figure1). Inotherwords, concreteness ratings capture thecategorical
ontological distinctionbetweenconcreteandabstract words (andtheirunderlyingconceptual
representations),whileimageabilityratings index agradedpropertythat is meant tocapturethe
differential association of words withsensory(primarilyvisual)properties.
Moreover, from atheoretical point ofview,imageabilityratings areaproxyforconcreteness onlyin
thedual-codingtheory,not,forexample,intheContext Availabilityhypothesis becauseonlythe
former explains differences betweenconcrete and abstract words interms ofwhether (andtowhat
extent)thenonverbal imagisticsystem is engaged (Reilyand Kean,2007; Fliessbach,Weis, Klaver,
Elger,&Weber,2006). AccordingtotheContext Availabilityhypothesis, however,imageability
wouldnot exhaust thedifferences between concreteandabstract words which,instead,ariseas a
consequenceofdifferent degrees ofrichness ofsemanticrepresentationwithinaverbal system.
Figure1.Densityplots for concreteness andimageabilityratings for 4,274words fromtheMRC
PsycholinguisticDatabase.Usingthediptest (Hartigan &Hartigan,1985)werejectedthe
hypothesis of unimodalityfor theconcreteness distribution(dip=.0244,p<.001)but not for the
imageabilitydistribution(dip =.0058,n.s.).