ebook img

Deconstructing the seductive allure of neuroscience explanations PDF

13 Pages·2015·0.2 MB·English
by  
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Deconstructing the seductive allure of neuroscience explanations

JudgmentandDecisionMaking,Vol.10,No.5,September2015,pp.429–441 Deconstructing the seductive allure of neuroscience explanations DeenaSkolnickWeisberg∗ JordanC.V.Taylor† EmilyJ.Hopkins† Abstract Previousworkshowedthatpeoplefindexplanationsmoresatisfyingwhentheycontainirrelevantneuroscienceinformation. Thecurrentstudiesinvestigatewhythiseffecthappens.InStudy1(N=322),subjectsjudgedpsychologyexplanationsthatdid ordidnotcontainirrelevantneuroscienceinformation.Longerexplanationswerejudgedmoresatisfying,aswereexplanations containing neuroscience information, but these two factors made independent contributions. In Study 2 (N=255), subjects directlycomparedgoodandbadexplanations. Subjectsweregenerallysuccessfulatselectingthegoodexplanationexcept whenthebadexplanationcontainedneuroscienceandthegoodonedidnot. Study3(N=159)testedwhetherneuroscience jargon was necessary for the effect, or whether it would obtain with any reference to the brain. Responses to these two conditionsdidnotdiffer.Theseresultsconfirmthatneuroscienceinformationexertsaseductiveeffectonpeople’sjudgments, whichmayexplaintheappealofneuroscienceinformationwithinthepublicsphere. Keywords:explanation,neuroscience,reasoning,seductiveallure. 1 Introduction cially, the neuroscience information was irrelevant to the logicoftheexplanationsandevenmadethegoodexplana- Attention to neuroscience is growing within the public tionsworse,accordingtotheratingsofexperts. sphere. Neuroscientificfindingsnowplayakeyroleinpub- When the explanations contained neuroscience informa- licconversationsabouteconomics, marketing, andthelaw, tion, ratings were significantly higher than when they did among other areas (e.g., Ariely & Berns, 2010; Camerer, not. Thiswasespeciallytrueforthebadexplanations(per- Loewenstein & Prelec, 2005; Farah, 2012; Greene & Co- haps because people have trouble detecting circularity in hen, 2004; Roskies, 2002; Satel & Lilienfeld, 2013). For arguments, see Rips, 2002). That is, non-experts judged example, neuroscience data are often used in courtrooms that psychological phenomena are explained better using asevidenceofadefendant’sresponsibilityorguilt(Morse, thelanguageofneuroscience,althoughthislanguageshould 2011;Saks,Schweitzer,Aharoni&Kiehl,2014;Schweitzer makenodifference, assumingthatanexplanation’s quality etal.,2011). Butitisnotentirelyclearhowmembersofthe is drawn primarily from the strength of its logic. One re- publicviewthesefindings. Dotheyunderstandtherolethat cent study (Scurich & Shniderman, 2014) also found that neuroscience information plays in explanations of people’s subjects give higher ratings to studies that included neuro- beliefsandbehaviors? scienceinformation,butonlywhentheconclusionsofthese Previous research suggests that the answer to this ques- studiesconfirmedtheirpriorbeliefs. However,theabsence tion is “no”. People are unduly swayed to think favor- of a no-neuroscience control condition in this study makes ably of psychology explanations that include references to itdifficulttodrawfirmconclusionsaboutthegeneraleffect neuroscience—even when such neuroscience information ofneuroscienceinformation. is logically irrelevant to the explanations (Weisberg, Keil, Goodstein, Rawson & Gray, 2008). In this study, subjects Three other studies did include the appropriate controls, read descriptions of psychological phenomena. Each phe- and both confirmed Weisberg et al.’s (2008) findings. One nomenonwasfollowedbyoneoffourtypesofexplanation, used the same stimuli in an exact replication (Fernandez- constructed by crossing explanation quality (good or bad) Duque, Evans, Christian & Hodges, 2015). The other with neuroscience information (present or absent). Cru- two used different sets of stimuli in a conceptual replica- tion (Rhodes, Rodriguez & Shah, 2014; Rhodes & Shah, WethankMatthewBateman,MarthaFarah,DiegoFernandez-Duque, 2015), inwhichsubjectsreadamocknewsarticledescrib- FrankKeil,andthemembersofthePennCognition&DevelopmentLab. ing psychological research; the article either did or did not ThisresearchwasfundedbytheTempletonFoundation(VarietiesofUn- contain irrelevant neuroscience information. Neuroscience derstandingprojectgranttoDSW). Copyright: ©2015. Theauthorslicensethisarticleunderthetermsof information thus exerts a seductive allure effect, whereby theCreativeCommonsAttribution3.0License. peoplewithoutadvancedtrainingbelievethatreferencesto ∗University of Pennsylvania, Department of Psychology, 3720 brain processes improve the quality of a psychological ex- Walnut St., Solomon Labs, Philadelphia, PA 19104. Email: planation, even when these references are logically irrele- [email protected]. †UniversityofPennsylvania. vant. Thiseffectcouldbethoughtofaspartofafamilyof 429 JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 430 heuristicsthatpeopleuseforjudgingthequalityofexplana- thantheexplanationsthatdidnot. Subjectsmayhavesim- tions, which includes teleological information (Lombrozo ply rated longer explanations as better. Indeed, other work &Carey,2006)andanintuitivesenseofsatisfaction(Trout, showed that people prefer longer explanations, even if the 2002). addedlengthdidnotaddtotheexplanation’squality(Kikas, Onestudyclaimedthatneuroscience imagesarerespon- 2003).Study1thusbeginsourinvestigationofthiseffectby sible for the effect (McCabe & Castel, 2008), suggesting replicatingWeisbergetal.(2008)withtheadditionofacon- thatpeopleareseducedbythevisualappealofimagesgen- trolforthelengthoftheexplanations. erated by fMRI scans and other neuroscientific techniques. A second possible explanation for the effect is that neu- However, many later studies have failed to replicate this roscience information may appeal due to its authoritative finding (Gruber & Dickerson, 2012; Hook & Farah, 2013; aesthetic: Explanations containing neuroscience informa- Keehner, Mayberry & Fischer, 2011; Michael, Newman, tion may look as though they have come from a suitably Vuorre,Cumming&Garry,2013;seeFarah&Hook,2013, scientific process, and so may be perceived as trustworthy forreview). Totestdirectlywhetherbrainimagesaddvalue and therefore convincing, regardless of their content (see to explanations that already contained neuroscience text, Sperber, 2010). We address this issue in Study 2 by ask- Fernandez-Duque et al. (2015) presented subjects with ex- ingsubjectstodirectlycomparegoodandbadexplanations planations that either contained no neuroscience informa- whentheydoanddonotcontainneuroscienceinformation. tion,containedirrelevantneuroscienceinformation,orcon- Thethirdpossibilitythatweinvestigateisthatpeopleare tainedirrelevantneuroscienceinformationandwereaccom- attracted to any kind of scientific-sounding jargon because panied by a neuroscience image. These researchers found they believe that use of these fancy terms signals higher- that people rated explanations with neuroscience informa- quality science. Indeed, math-based jargon has precisely tion as better than explanations without this information, thiseffect(Eriksson,2012). WeaddressthisissueinStudy as noted above, but images did not have any additional ef- 3 by comparing subjects’ ratings of explanations that use fect.Further,Weisbergetal.(2008),Fernandez-Duqueetal. simplereferencestobrainprocesseswiththeirratingsofex- (2015), Rhodes et al. (2014), and Rhodes and Shah (2015) planationsthatusemoretechnicalterms. obtained the seductive allure effect without the use of any pictures. These studies strongly suggest that neuroscience imageryisnotthesourceoftheeffect. 2 Study 1 Why, then, does this effect happen? The importance of answeringthisquestionbecomesevidentwhenweexamine Study1wasdesignedtodeterminewhethertheseductiveal- the many ways in which neuroscience information is used lureeffectresultsfromsubjects’responsestoneuroscience (and misused) in the public sphere. The proliferation of informationitselforfromthetendencyforexplanationscon- headlines proclaiming that some drug or activity “literally tainingneuroscienceinformationtobelongerthanexplana- changes your brain” illustrates both how appealing neuro- tionswithoutthisinformation. Previousworksuggeststhat scienceinformationistothegeneralpublicandhowpoorly lengthdoesnotaccountfortheeffect: Fernandez-Duqueet thisinformationisunderstood.Totakeaweightierexample, al.(2015)foundthatexplanationswithaddedneuroscience attorneysmayappealtoneuroscience-basedevidenceinor- informationwereratedmorehighlythanunembellishedex- dertoconvinceajuryofalegalfact. Butbecausethiskind planations, but explanations with added social psychology of information is intuitively compelling even when it is ir- information were not. In addition, Rhodes et al. (2014) relevant,suchevidencemayundulybiasthejury,potentially foundthatstimuliwithneuroscienceinformationwererated threateningthefairnessofthejudicialsystem(seeGreene& more highly than length-matched stimuli without neuro- Cohen, 2004; Morse, 2004). Similarly, in the field of edu- scienceinformation.Theseresultssuggestthattheseductive cation, unsubstantiated claims about how children’s brains allureeffectcannotbeaccountedforsolelybytheexplana- changeorfundamentaldifferencesbetweenboys’andgirls’ tions’length. brains can lead to the implementation of educational poli- Study1continuedthisinvestigationoftheroleoflength cies or practices that seem appealing but may not actually and addressed a potential issue with the method used in benefitthestudents(seeBruer,1997;Goswami,2006). previous studies. Both Fernandez-Duque et al. (2015) Learning why neuroscience information is alluring can and Rhodes et al. (2014) controlled for length by mak- help us to develop techniques to reverse some of these ing the without-neuroscience explanations longer, so as to trends. The current studies begin to address this issue by match the length of the with-neuroscience stimuli. But investigatingthreefactorsthatmightcontributetotheseduc- thisadditionalinformationmayhaveaffectedhowsubjects tiveallureeffect: length(Study1),explicitappealofneuro- rated the without-neuroscience explanations. For example, science(Study2),andjargon(Study3). Intermsoflength, Fernandez-Duqueetal.(2015)comparedexplanationswith the explanations in Weisberg et al. (2008) that contained superfluousinformationfromsocialscienceorhardsciences irrelevant neuroscientific information were always longer to those with superfluous neuroscience information. How- JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 431 ever, this information from other fields may have seemed Materials. We selected four of the 18 items presented to less relevant to the explanations than the neuroscience in- subjects in Weisberg et al. (2008) and Fernandez-Duque et formation,potentiallyloweringsubjects’ratings. Thusthis al. (2015) (babies’ abilities to do simple arithmetic, atten- design does not separate the effect of length from the ef- tionalblink,genderdifferencesinspatialreasoning,anddif- fect of different types of added information. The current ferencesbetweenseeingandimaginingobjects;seesupple- study made the with-neuroscience explanations shorter so mentalmaterialsforfullstimulusitems). Thesewereitems astomatchthelengthofthewithout-neuroscience stimuli. for which subjects in a pilot sample consistently judged This more fully un-confounds the variables of length and the bad version of the without-neuroscience explanation as neuroscienceinformation. worse than the good version of that explanation. Each of Study 1 thus provides a more complete investigation of the four items consisted of a description of a psycholog- the potential effect of length on the seductive allure effect, ical phenomenon and eight different explanations for that whichwillallowustodeterminehowneuroscienceinforma- phenomenon. The good explanations are the ones that the tionaffectspeople’sjudgments.Iftheseductiveallureeffect researchersthemselvesprovidedforthephenomenaorthat is only due to a general tendency to judge longer explana- were provided in psychology textbooks. The bad explana- tions as better, then it should disappear when the explana- tions were circular restatements of the phenomena with no tions that do and do not contain neuroscience are matched mechanisticinformationthatcouldgiveareasonforthephe- for length. But if something about neuroscience informa- nomenon. Itemsinallstudiesareinthesupplement. tionleadstomorepositivejudgmentsofexplanations,then The explanations used in the Without Neuroscience- the effect of neuroscience should remain regardless of the Short and the With Neuroscience-Long conditions exactly lengthoftheexplanation. matchedthoseusedinWeisbergetal. (2008). Toconstruct theWithoutNeuroscience-Longexplanations,weaddedsu- perfluous wording to the existing Without Neuroscience- 2.1 Method Shortexplanationstomakethemthesamelengthasthecor- Subjects. We recruited subjects from two populations: responding With Neuroscience-Long explanations. Impor- undergraduatestudentsfromthepsychologysubjectpoolat tantly,thisadditionalwordingreferredonlytopsychological theUniversityofPennsylvaniaandworkersonMechanical constructsandnevertoothersciences,mathematics,orneu- Turk. Because previous work on this topic has primarily roscience,andthisinformationdidnotaddanyvaluetothe usedundergraduatesassubjects,weaddedtheMTurkwork- explanation. ToconstructtheWithNeuroscience-Shortex- ers in order to assess the generality of the effect in a more planations,weeditedtheexistingWithNeuroscience-Long representative population. This study included 204 under- explanations to make them the same length as the corre- graduates (143 women, 61 men; mean age = 19.8 years, sponding Without Neuroscience-Short explanations. Re- range = 18–50) and 177 MTurk workers (85 women, 92 gardless of length, the irrelevant neuroscience information men;meanage=37.5years,range=19–70). Undergradu- wasidenticalacrossthegoodandbadversionsoftheexpla- atesreceivedcoursecreditforparticipatinginthestudy,and nationsforeachphenomenon. MTurkworkersreceived20cents. Procedure. All subjects completed an online survey dis- Design. Subjects were divided into 4 conditions accord- tributedonQualtrics. Ineachtrial,subjectsreadadescrip- ingtoa2(Neuroscience: with,without)x2(Length: long, tionofapsychologicalphenomenon,whichappearediniso- short)design. Thesewerebothbetween-subjectsvariables, lationonthescreenfor10secondsbeforetheywereallowed soanindividual subjectsawexplanations thateitherallin- toadvancetothenextscreen. Onthesecondscreenofeach cludedoralldidnotincludeneuroscienceinformation,and trial,thephenomenonappearedagainatthetop,followedby theirexplanationswouldallcomefromthesamelengthcat- oneoftheeightpossibleexplanationsforthatphenomenon. egory. There were 43 MTurk workers and 44 undergrad- Subjectswereaskedtojudgehowsatisfyingtheyfoundthis uates in With Neuroscience-Long, 40 MTurk workers and explanationonaseven-pointscale,from–3(veryunsatisfy- 65 undergraduates in With Neuroscience-Short, 49 MTurk ing)to+3(verysatisfying),with0astheneutralmidpoint. workers and 50 undergraduates in Without Neuroscience- Eachsubjectsawallfourstimulusitems,onepertrial,in Long, and 45 MTurk workers and 45 undergraduates in arandomizedorder.Oneachtrial,thephenomenonwaspre- WithoutNeuroscience-Short. Qualitywasawithin-subjects sentedalongwithoneversionoftheexplanation. Subjects’ variable; for each trial, the survey software randomly de- conditiondeterminedwhethertheysawalongorshortver- termined whether to show the good or bad version of the sionandawith-orwithout-neuroscienceversion. Whether explanation.1 theysawagoodorbadversionoftheexplanationwasran- domly determined on each trial (as described above in the 1Duetotherandomization,therewere47subjectswhosaweithergood Designsection). Attheendofthesurvey,subjectsprovided explanationsoneverytrialorbadexplanationsoneverytrial.Theinclusion ofthesesubjectsdidnotaffectanyanalyses,sotheywereleftinthesample. basic demographic information: age in years, gender, and JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 432 Table 1: Study 1 mixed-effects linear regression model (∗ Figure 1: Average ratings of explanation quality in Study p<.05). 1. Errorbarsrepresent95%confidenceintervalsaroundthe means. Predictor Estimate [95%CI] t Intercept 0.24 [0.14,0.34] 5.00∗ Study 1 Item2 0.24 [0.10,0.38] 3.21∗ 3 Good Explanations Item3 –0.61 [–0.74,–0.47] –8.82∗ Bad Explanations Item4 0.65 [0.21,0.79] 9.41∗ 2 Length 0.12 [0.02,0.21] 2.55∗ Neuroscience 0.13 [0.03,0.22] 2.70∗ 1 Group 0.15 [0.06,0.25] 3.18∗ Rating QNueualriotsyciencexItem2 –00..0356 [[–00..2189,,00..4140]] –80..4772∗ Average 0 NeurosciencexItem3 0.20 [0.06,0.34] 2.93∗ −1 NeurosciencexItem4 –0.24 [–0.37,–0.10] –3.45∗ GroupxItem2 0.03 [–0.11,0.17] 0.50 −2 GroupxItem3 0.16 [0.01,0.30] 2.34∗ GroupxItem4 –0.19 [–0.34,–0.05] –2.71∗ −3 Without Neuroscience With Neuroscience QualityxItem2 0.08 [–0.08,0.20] 1.06 QualityxItem3 –0.14 [–0.28,–0.02] –1.94∗ QualityxItem4 –0.25 [–0.38,–0.12] –3.57∗ Note:Thisregressionpredictedsubjects’ratingsofthequal- This test revealed a main effect of Quality (Figure 1), ityoftheexplanations. TheinterceptrepresentstheWithout wheregoodexplanations(M =0.58,SD=1.68)wererated Neuroscience condition, short explanations, undergraduate morehighlythanbadexplanations(M =–0.13,SD=1.80). subjects, and bad explanations. Item is deviation coded, TherewasalsoamaineffectofNeuroscience: Explanations such that the coefficient for each level represents deviation that contained neuroscience (M = 0.34, SD = 1.75) were fromthegrandmean;Item1isthereferencelevel. ratedmorehighlythanexplanationsthatdidnot(M =0.11, SD = 1.80). We also found a main effect of Length: Long levelofeducation(fortheMTurksubjects)orclassyearand explanations(M =0.34,SD=1.76)wereratedmorehighly major(fortheundergraduates). thanshortexplanations(M=0.12,SD=1.79).Finally,there wasamaineffectofGroup: MTurkworkers(M =0.37,SD 2.2 Results =1.73)gaveoverallhigherratingsthanundergraduates(M =0.10,SD=1.80). Unlike in Weisberg et al. (2008), some subjects in the cur- rentstudyreceivedunequalnumbersofgoodandbadexpla- TheeffectsofGroup,Neuroscience,andQualityalsovar- nations. In order to deal with this, we conducted a mixed- iedbyitem, asindicatedbythesignificantinteractions. To effects linear regression. The model included random in- examinetheseinteractions,weconductedseparatelinearre- terceptsbysubjectaswellasrandomslopesbysubjectfor gressions for each item examining main effects of Group, theeffectofQuality(theonlywithin-subjectsvariable). We Neuroscience,Length,andQuality. Theresultsaresumma- tested effects of Item, Group (MTurk or undergraduates), rized in Table 2. Although the magnitudes (and therefore Length (long or short), Neuroscience (present or absent), significance levels) of the effects varied by item, only two andQuality(goodorbad)andtheirinteractions2;themodel effectswerenotinthepredicteddirections;forItem4,there thatbestfitthedataisshowninTable1. were non-significant negative effects of Group and Neuro- 2Preliminaryanalysesrevealedoneeffectofgender:aninteractionbe- science. TheeffectsofneuroscienceforItem2andquality tweengenderandexplanationlength.Men’sratingsdidnotdifferforlong for Item 4 were small and non-significant, but in the pre- explanations(M=0.27,SD=1.75)andshortexplanations(M=0.33,SD =1.74). However,womenratedlongexplanations(M=0.38,SD=1.76) dicteddirections. morehighlythanshortexplanations(M=–0.03,SD=1.81). Wehaveno explanationforthisunexpectedgenderdifference,andbecausegenderdid notaffecttheothervariables,wedidnotconsidergenderfortheremainder ofouranalyses. JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 433 maybeduetothefactthattheneuroscienceinformationin Table2:Regressioncoefficientsforindividualitemanalysis theexplanationsofthisphenomenonisentirelycontainedin inStudy1(∗p<.05,+p<.10). the first sentence, separate from the explanatory (or circu- Item1 Item2 Item3 Item4 lar)informationinthesecondsentence. Thisstructuremay havemadeiteasierforsubjectstoseethattheneuroscience Length 0.04 0.05 0.17+ 0.19∗ informationwasnotrelevanttotheexplanation’squality. Neuroscience 0.22∗ 0.08 0.34∗ –0.10 Overall,Study1demonstratesthattheseductiveallureef- Group 0.14 0.18∗ 0.30∗ –0.06 fectreplicatesandisnotsolelyduetolength.Study2begins Quality 0.69∗ 0.44∗ 0.23∗ 0.08 tomoredirectlyaddresswhytheeffecthappens. Todoso, ratherthanaskingsubjectstoratesingleexplanationsfora phenomenon,weaskthemtochoosewhichoftwoexplana- tionstheyfindmostsatisfying. Eachpaircontainedagood 2.3 Discussion explanationandabadexplanation,buteitherbothcontained Study1wasdesignedtoreplicatetheseductiveallureeffect neuroscience, neither contained neuroscience, or only the andtestforthecontributionofexplanationlength. Subjects bad explanation contained neuroscience. This is a some- did indeed judge longer explanations as better than shorter whatlessecologicallyvaliddesign,sinceitisrarethatpeo- ones overall, demonstrating a general bias towards longer plewouldneedtoevaluatemultipleexplanationsforasingle explanations.However,thislengthpreferencedoesnotfully phenomenon. However,thisdesignallowsustotestdirectly explaintheseductiveallureofneuroscience. Makingexpla- how neuroscience information may interfere with people’s nationslongerdoesmakethemseembetter,butaddingneu- ability to distinguish good from bad explanations. Given roscienceinformationdoesaswell,andthesetwomodifica- previousresults,weexpectedthatsubjectswouldgenerally tionshadindependenteffects. Thisresultconfirmsotherre- beabletodistinguishgoodfrombadexplanationsifbothor centstudiesthatshowthattheseductiveallureeffectobtains neither contained neuroscience. If, however, neuroscience whenexplanationlengthiscontrolled(Fernandez-Duqueet informationhastheeffectofmaskingthepoorqualityofthe al.,2015;Rhodesetal.,2014). badexplanations,peopleshouldbelesslikelytodistinguish goodfrombadexplanationswhenonlythebadonecontains There was also a strong effect of explanation quality: neuroscience. Good explanations were judged as better than bad expla- nationsoverall. Thisresultdemonstratesthatpeoplearenot generally confused about what makes certain explanations 3 Study 2 betterthanothersandareabletodistinguishbetweengood andbadexplanations.However,asnotedabove,theaddition ofneuroscienceinformationinterfereswiththisability. 3.1 Method Finally,wefoundthatundergraduatesgaveoveralllower Subjects. Thisstudyincluded130undergraduates(86fe- ratingsthanMTurkworkers.Thisislikelynotduetotheun- male,44male; meanage=19.5years,range=18–27)and dergraduateshavingahigherlevelofeducation,since99% 130MTurkworkers(90female,37male,threeunreported; oftheMTurkworkersreportedhavingatleastsomecollege mean age = 40.6 years, range = 19–71). The undergradu- education, and 50% reported earning an advanced degree. ateswererecruitedfromthepsychologysubjectpoolatthe Instead, the experience of participating in research as part University of Pennsylvania and received course credit for ofaclass,orofcurrentlybeingamemberofaneducational theirparticipation. TheMTurkworkerswererecruitedfrom community, may serve to increase overall skepticism. Re- Amazon’s system and were paid 20 cents for their partici- gardless,bothpopulationsshowedthesamegeneralpattern pation. Anadditionalsevensubjects(threeMTurkworkers ofresponsestotheexplanations(i.e.,therewerenosignifi- andfourundergraduates)wererecruitedbutexcludedfrom cantinteractionswithsubjectgroup). the final analyses for failing an attention check (described These main effects appeared in nearly the same way for below). allfouritems;however,thereweresomedifferencesinhow subjects responded to the four phenomena. Notably, the phenomenondescribingthedifferencesbetweenseeingand Design. There were three between-subjects conditions in imaginingobjects(Item4)didnotshowaneffectofQuality thisstudy. AsinStudy1,therewerefourtrialspersubject, orNeuroscience.Thisitemwasratedhigheroverallthanthe eachofwhichusedadifferentphenomenon(orderrandom- others(asindicatedbythesignificantmaineffectforItem4 ized). Eachphenomenonwasaccompaniedbybothagood in the regression), and there was little difference in ratings and a bad explanation. In the Without Neuroscience con- between the different versions of the explanation. In addi- dition (41 MTurk workers and 43 undergraduates), neither tion, the phenomenon describing attentional blink (Item 2) explanationcontainedanyneuroscienceinformation,andin did not show as strong of an effect of Neuroscience. This theWithNeurosciencecondition(42MTurkworkersand45 JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 434 undergraduates), both explanations contained neuroscience Table3: Study2mixed-effectslogisticregressionmodel(∗ information. ThecrucialconditionwastheMixedcondition p<.05). (47 MTurk workers and 42 undergraduates), in which the goodexplanationdidnotcontainneuroscienceinformation Predictor Estimate SE z andthebadonedid,pittingqualityandneuroscienceagainst eachother. Intercept 0.05 0.14 0.33 Item2 0.67 0.13 5.05∗ Materials. WeusedthesamefourphenomenaasinStudy Item3 –0.25 0.12 –2.08∗ 1, accompanied by the short versions of the four possible Item4 –0.94 0.13 –7.44∗ explanationsforeachphenomenon: goodandbadexplana- Group 0.25 0.14 1.75 tions both with and without irrelevant neuroscience infor- WithoutNeurosciencecondition 0.63 0.21 3.08∗ mation(seesupplementalmaterials). WithNeurosciencecondition 1.12 0.22 5.18∗ GroupxItem2 –0.20 0.13 –1.52 Procedure. Subjects completed an online survey dis- GroupxItem3 0.19 0.12 1.53 tributedonQualtrics.Foreachtrial,theyfirstreadadescrip- tionofapsychologicalphenomenon,whichappearediniso- GroupxItem4 0.37 0.12 3.05∗ lationonthescreenfor10secondsbeforetheywereallowed WithoutNeurosciencecond.xGroup–0.49 0.21 –2.37∗ toadvancetothenextscreen. Onthesecondscreenofeach WithNeurosciencecond.xGroup –0.53 0.21 –2.48∗ trial, the phenomenon appeared again at the top, followed Note. TheinterceptrepresentstheMixedconditionandun- by the prompt, “Please choose which explanation you find dergraduatesubjects. Itemisdeviationcoded,suchthatthe more satisfying.” Subjects always saw one good explana- coefficientforeachlevelrepresentsdeviationfromthegrand tionandonebadexplanationaswellasthechoice“bothare mean;Item1isthereferencelevel. equal.” The “equal” option always appeared in the center, withtheleft/rightpositionofthegoodandbadexplanations randomized across trials. After making their choice, sub- jectswereaskedtoexplainwhytheyhadmadethatchoice of gender, so this variable is not considered further. The inone ortwosentences. Therewerefour suchtrialsinthe modelthatbestfitthedataispresentedinTable3. experiment, eachinvolvingadifferentphenomenon andits The primary result from these analyses is the significant accompanyingexplanations. main effect of condition: Subjects found it more difficult Afterthesecondofthefourtrials,subjectsengagedinan todeterminewhichwasthegoodexplanationintheMixed attention check. Following methods recommended in Op- condition where only the bad explanation contained neu- penheimer,Meyvis&Davidenko(2009),thischeckingtrial roscience (Figure 2). Specifically, subjects were signifi- presentedanotherphenomenonandtwoexplanationssothat cantly more likely to select the good explanation in either itlookedsuperficiallyliketheotherfourtrials.Forthistrial, theWithNeuroscience(72.0%oftrials)orWithoutNeuro- weincluded instructionsattheendofthephenomenon de- science(63.5%oftrials)conditionsthanintheMixedcon- scriptiontellingsubjectstochoosethe“equal”optionandto dition(51.7%oftrials). write that they had done so as their justification. As noted There was also a significant Group x Condition interac- above, we excluded seven subjects who failed this check tion. Follow-up regressions conducted on each group sep- by selecting a different option. At the end of the survey, arately showed that the MTurk workers were significantly subjects provided the same demographic information as in morelikelytoselectthegoodexplanationintheWithNeu- Study1: age,gender,andeducationlevel. roscienceconditionascomparedtotheMixedcondition(β =0.58,p<.05),buttherewasnosignificantdifferencebe- 3.2 Results tween the Without Neuroscience and Mixed conditions (β = 0.15, p = .59). For undergraduate subjects, subjects se- To analyze the data, we conducted a mixed-effects logistic lected the good option significantly more often in both the regressionpredictingwhethersubjectsselectedthegoodex- With Neuroscience (β = 1.68, p < .001) and Without Neu- planationoneachtrial,consideringthisresponseascorrect roscience(β =1.14,p<.001)conditionsthanintheMixed and the other two responses (selecting the bad explanation condition. Thus,althoughthemaineffectofconditionwas orthe“bothareequal”option)asincorrect. Themodelin- significant in the whole sample, it was driven more by the cluded random intercepts by subject. We tested effects of undergraduatesubjectsthanbytheMTurkworkers. Finally, Item,Group(MTurkworkersorundergraduates),Condition although there were significant differences between items, (WithNeuroscience,WithoutNeuroscienceorMixed),and asinStudy1,therewasnotasignificantItemxConditionor possible interactions. Preliminary analyses found no effect ItemxConditionxGroupinteraction. Therefore,theeffect JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 435 frontallobeswithoutexplainingwhatisactuallyhappening Figure 2: Average number of trials on which subjects se- isbullshit.” lectedthegoodexplanationinStudy2. Errorbarsrepresent Eachsubjectwasgivenascore(outof4)forthenumber 95%confidenceintervalsaroundthemeans. Thedottedline ofpositivebrain-basedjustificationstheygave. A2(Group: representschanceperformancesinceselectingthegoodex- MTurkworkers,undergraduates)x2(Condition:WithNeu- planationwasoneofthreepossibleresponsesoneachtrial. roscience,Mixed)ANOVArevealedonlyasignificantmain effectofGroup, F(1,166) =7.08, p<.01, η2 =.04: Under- Study 2 graduates (M = 0.62, SD = 0.84) were overall more likely 4 than MTurk workers (M = 0.31, SD = 0.66) to refer to the MTurk Workers Undergraduates brainasaddingvaluetoanexplanation. Therewasneither was selected 3 aCosnigdnitiifiocnanintteerffaecctitoonf. Condition nor a significant Group x which good explanation 2 3Wof.3haepnsaDysckihseocdluotgosiscciaholonpohseenboemtweneeonn,gsouobdjeacntsdsbealedcetexdpltahneagtiooonds Average # of trials on 1 eawpxloshps.iilctTahinvhaeestlusiyeob.njreeTcastsausklbtetseenincnodtgonegmdfierottmhoreesrrs,auatbtethjiesetcfhsyteesi’ngrgreoasootuidnnltgstehsxdefperlmomamnaoajnotSisrottiurtnyadstyeomf1t,othriraine-t subjects understand the difference between good and bad explanations. 0 Mixed Without Neuroscience With Neuroscience The one exception to this conclusion is the Mixed con- dition, in which the bad explanation contained irrelevant neuroscienceinformationandthegoodexplanationdidnot. Here, subjects were seduced by the presence of neuro- science information, which made them less likely to pre- ofConditionandtheConditionxGroupinteractionwerenot fer the good explanations than in the other conditions. Al- significantlydifferentacrossitems. though the main effect of condition was significant in the To gain further insight into subjects’ responses, for the full sample, it was driven primarily by the undergraduates. two conditions that used explanations containing neuro- Indeed, the undergraduates’ justifications were more likely science language (With Neuroscience and Mixed), we per- to mention neuroscience in a positive light. This suggests formedatextsearchonsubjects’justificationsforwordsre- thatthepresenceofneuroscienceinformationplayedakey latedtoneuralprocessesgenerally(“brain”,“lobe,”,“scan”, role in convincing these subjects that the bad explanations “neur*”)andforthespecificneurosciencetermsusedinthe were satisfactory. As the justifications quoted above illus- explanationsthemselves(“premotor,”“cortex”);58%ofjus- trate,someundergraduatesappearedtorelyonthepresence tifications inthesetwoconditions contained atleastone of ofneuroscienceasaheuristictojudgethequalityofanex- these terms. A research assistant, who was blind to condi- planation. tion,group,andstudyhypotheses,furthercodedthesejusti- Itisnotentirelyclearwhyundergraduateswouldbemore ficationsforwhethersubjectsreferredtothebrainasadding attracted to explanations containing neuroscience informa- valuetoanexplanation. Forexample,“thisgivesabiologi- tionthanMTurkworkers,althoughcurrentlylearningabout calexplanationandusesbrainpartstoexplain”,“brainscans psychological and neuroscientific phenomena might have andtimingseemmoreaccurate”,and“it’smoreindepthand swayedtheundergraduatestolendmoreweighttothepres- seems more factual because it is talking about brain parts enceofneuroscience. Onepossibilityisthat,sincetheycur- and stuff.” Of justifications that referenced the brain, 84% rentlyarelearningaboutthefunctionsofthebrain,reading didsointhispositiveway.Theremainingjustificationssug- aboutspecificphenomenainwhichthebrainappearstoplay gested a good grasp of the irrelevance of this information, a causal role leads them to judge explanations with neuro- suchastheseundergraduates: “Idonotwanttohearabout scienceexplanationsmorefavorably.Insupportofthisargu- what the brain is doing. I am interested in WHY the phe- ment, students who were currently taking a course on neu- nomenon occurs in a more general sense” and “Saying be- roscience(Weisbergetal.,2008,Study2)showedastronger cause of frontal lobe areas is not a sufficient explanation, attractiontoexplanationscontainingirrelevantneuroscience but just states where the processing is occurring.” Or, as information than students recruited from the introductory oneMTurkworkerputit,“Talkingmumbojumboaboutthe psychologypool(Weisbergetal.,2008,Study1). JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 436 Finally, subjects’ performance in this study can start to gon,thentheNeurosciencePlusJargonexplanationsshould explain one of the item effects seen in Study 1. Specifi- be rated more highly than the other two, which should not cally, in Study 1, the seeing/imagining item (Item 4) was differ. Finally, there might be an additive effect of jargon judged similarly regardless of quality or presence of neu- andneurosciencelanguage,inwhichcasetheNeuroscience roscience. Here in Study 2, about twice as many subjects Plus Jargon explanations would be rated more highly than chosethe“bothareequal”optionforthisitemthanforthe theSimpleNeuroscienceexplanations,whichwouldinturn other three, indicating that they could not see a difference beratedmorehighlythantheWithoutNeuroscienceexpla- between the good and bad explanations for this item. The nations. differencebetweenthetwoversionswasveryslight,chang- ingonly“usesthesameprocess”to“resultsinthesamear- 4.1 Method rayofresponses”.Infact,anumberofparticipantsexplicitly statedintheirjustificationsthatthetwoexplanationsseemed Subjects. Thefinalsampleforthisstudyincluded88un- thesame(e.g., “Bothexplanationssoundliketheyaresay- dergraduates (63 female, 25 male; mean age = 19.5 years, ingthesamething”,“Bothexplanationsaresimilarandsay range=18–22)and82MTurkworkers(42female,38male, thesamethingjustinaslightlydifferentway.”) twounreported;meanage=35.0years,range=19–67). As inpreviousstudies,theundergraduateswererecruitedfrom thepsychologysubjectpoolattheUniversityofPennsylva- 4 Study 3 nia and received course credit for their participation. The MTurk workers were recruited from Amazon’s system and Havingdeterminedthatlengthdoesnotunderlietheseduc- werepaid20centsfortheirparticipation. Anadditional50 tive allure effect, and that neuroscience information is ef- subjects completed the survey but were excluded from the fective at disguising bad explanations, Study 3 tested an- final analyses for failing an attention check (described be- otherpossiblereasonthatneuroscienceexplanationsareap- low; 22MTurkworkersand28undergraduates). Although pealingtosubjects,namelythatthisinformationtendstoin- more subjects failedthe attention check here than inStudy cludetechnicaljargon. Ifsubjectsareattractedtoscientific- 2, thedesignofthisstudywasdifferentandmayhavepre- soundingterms,thenneuroscienceinformationperseisnot sentedalessengagingtaskthanStudy2,andthesenumbers seductive;subjects’responsescanbeinfluencedbythepres- areinlinewithotherstudiesthatincludedsimilarattention enceofanyjargon. However,technicallanguageandrefer- checks(seeOppenheimeretal.,2009). encestothebrainwereconfoundedintheexplanationsused thusfar,preventingusfromdeterminingwhetheranyrefer- Design. Thisstudyuseda2(Group: MTurk,undergradu- ence to the brain would be sufficient or whether technical ate)x2(Neuroscience:SimpleNeuroscience,Neuroscience jargonisnecessary. Study3constructedalternativeversions Plus Jargon) x 2 (Quality: good, bad) design. Group and oftheseexplanationsinordertoteaseoutwhichtypeofin- Neuroscience were between-subjects variables and Quality formationisresponsiblefortheseductiveallureeffect. was a within-subjects variable.3 Subjects were assigned to This study’s design mirrored that of Study 1, in which either the Neuroscience Plus Jargon condition (42 MTurk subjectsreaddescriptionsofpsychologicalphenomenaone workersand44undergraduates)ortheSimpleNeuroscience at a time and then rated one explanation of each phe- condition(40MTurkworkersand44undergraduates). Data nomenon. InStudy3,theseexplanationscamefromoneof from the 45 MTurk workers and 45 undergraduates in the two sets: Simple Neuroscience, in which the explanations Without Neuroscience-Short condition from Study 1 were referred to brain scans and neural processes but in simple alsousedhereforcomparison. languagewithoutreferencetospecificbrainareas,andNeu- roscience Plus Jargon, in which the explanations included Materials. Toconstructthestimuli,weusedthesamefour technical terms to refer to the type of brain scan used and psychologicalphenomenaasinStudies1and2,andmodi- theindividualareasofthebrain. Inbothcases, thesestim- fiedtheWithNeuroscience-Shortexplanationstofitthenew uliwereconstructedbymodifyingtheShortversionsofthe conditions(seesupplementalmaterials).Explanationsinthe stimuliusedinStudy1,andtheShort-WithoutNeuroscience SimpleNeuroscienceconditionremovedreferencestospe- conditionfromthatstudyservesasacontrolconditionhere. cific brain-scanning techniques or brain areas and replaced Thissetofstimuliallowsustotestamongthreehypothe- themwithgenericterms(“brainscans”,“visualarea”). Ex- ses. If neuroscience information alone is responsible for planations in the Neuroscience Plus Jargon condition en- theseductiveallureeffect,weshouldexpectsimilarratings hanced existing references to include as much specific jar- for the Neuroscience Plus Jargon and the Simple Neuro- science explanations, both of which should be rated more 3AsinStudy1,therandomizationalgorithmledto11subjectsreceiving eithergoodexplanationsoneverytrialorbadexplanationsoneverytrial. highlythantheWithoutNeuroscienceexplanations. Ifneu- Theinclusionofthesesubjectsdidnotaffectanyanalyses,sotheywere roscienceinformationappealsbecauseitcontainsfancyjar- leftinthesample. JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 437 gon as possible (“fMRI scans”, “parietal lobe”). Each ex- Table4: Study3mixed-effectslinearregressionmodel planation had a good and a bad version, and this modified (∗p<.05). informationwasexactlythesameinbothversions. Predictor Estimate [95%CI] t Procedure. All subjects filled out an online survey on Intercept 0.02 [–0.19,0.24] 0.17 Qualtrics. As in Study 1, for each of the four trials, sub- jectsfirstreadadescriptionofoneofthefourpsychological Item2 0.33 [0.06,0.62] 2.19∗ phenomena, which appeared on the screen for 10 seconds Item3 –0.91 [–1.24,–0.62] –5.97∗ beforetheywereallowedtoadvance. Onthesecondscreen, Item4 0.98 [0.69,1.25] 6.44∗ this phenomenon description appeared again, followed by Neuroscience 0.27 [0.05,0.51] 2.44∗ one of the four possible explanations of the phenomenon, Group 0.15 [0.05,0.27] 2.64∗ accordingtothesubject’sassignedcondition;whetherthey saw the good or bad version of the explanation was ran- Quality 0.57 [0.35,0.78] 5.39∗ domly determined on each trial (as described above in the NeurosciencexItem2 –0.37 [–0.68,-0.07] –2.42∗ Designsection). Subjectswereaskedtoratehowsatisfying NeurosciencexItem3 –0.04 [–0.36,0.27] –0.23 theyfoundtheexplanationona–3(veryunsatisfying)to+3 NeurosciencexItem4 –0.05 [–0.38,0.28] –0.35 (verysatisfying)scale. Theywerethenaskedtojustifytheir QualityxItem2 0.12 [–0.18,0.44] 0.82 ratinginoneortwosentences. QualityxItem3 –0.39 [–0.67,–0.07] –2.62∗ As in Study 2, after the first two trials, we included an attentioncheck. Thisattentioncheckpresentedanotherde- QualityxItem4 –0.57 [–0.85,–0.25] –3.83∗ scription of a psychological phenomenon and an explana- Note:Thisregressionpredictedsubjects’ratingsofthequal- tion for it in exactly the same way as the other trials, ex- ityoftheexplanations.Theinterceptrepresentstheexplana- cept that the last sentence of the explanation told subjects tionsthatdidnotcontainneuroscience,undergraduatesub- toselect3onthescale. Asnotedabove, 50subjectsfailed jects, and bad explanations. Item is deviation coded, such thisattentioncheck(byfailingtoselect3and/orbydemon- thatthecoefficientforeachlevelrepresentsdeviationfrom stratingalackofattentivenessintheirjustificationsforthis thegrandmean;Item1isthereferencelevel. item)andarenotincludedinouranalyses. Attheendofthe survey, subjects responded to the same basic demographic questionsasinStudies1and2,reportingtheirage,gender, absent),andQuality(goodorbad)andtheirinteractions;the andhighestlevelofeducation. modelthatbestfitthedataisshowninTable4. The analysis revealed significant main effects of Group, 4.2 Results Quality, and Neuroscience (Figure 3). MTurk subjects (M = 0.33, SD = 1.90) gave higher overall ratings than under- In order to have a control condition with which to com- graduatesubjects(M=0.03,SD=1.89),andsubjectsrated pare the current subjects’ responses, the analyses for this goodexplanations(M =0.52,SD=1.86)morehighlythan studyadditionallyincludetheresponsesfromthesubjectsin bad explanations (M = –0.19, SD = 1.88), replicating the theWithoutNeuroscience-ShortconditionfromStudy1(45 results of Study 1. The significant main effect of Neuro- MTurkworkersand45undergraduates). Preliminaryanaly- science indicates that explanations were rated more highly ses revealed no effects of gender, so it was not included in inthetwoconditionsthatusedneurosciencelanguage(M= ouranalyses. 0.29, SD = 1.95) than in the Without Neuroscience condi- As in Study 1, we conducted a mixed-effects linear re- tion(M=–0.03,SD=1.79). Therewasnosignificanteffect gression analysis. The model included random intercepts ofJargon, meaningthattheNeurosciencePlusJargoncon- by subject as well as random slopes by subject for the dition(M =0.29,SD=1.91)wasnotsignificantlydifferent effect of Quality (the only within-subjects variable). We fromtheothertwocombined(M=0.12,SD=1.89). created two dummy variables to examine the effects of As in Study 1, the effects of Neuroscience and Quality neuroscience and jargon; the Neuroscience variable coded alsovariedbyitem,asindicatedbysignificantinteractions. whether neuroscience information was present (the Simple Toexaminetheseinteractions,weconductedseparatelinear Neuroscience and Neuroscience Plus Jargon conditions) or regressionsforeachitemexaminingmaineffectsofGroup, absent(theWithoutNeurosciencecondition). Similarly,the Neuroscience, Length, and Quality. The results are sum- Jargonvariablecodedwhetherjargonwaspresent(theNeu- rosciencePlusJargoncondition)orabsent(theSimpleNeu- marized in Table 5. Although the magnitudes (and there- roscience and Without Neuroscience conditions). The re- fore significance levels) of the effects varied by item, only gressiontestedeffectsofItem,Group(MTurkorundergrad- two effects were not in the predicted directions; consistent uates),Neuroscience(presentorabsent),Jargon(presentor with Study 1, Item 2 had a negative, non-significant effect JudgmentandDecisionMaking,Vol.10,No.5,September2015 Seductiveallureofneuroscienceexplanations 438 Figure3: AverageratingsofexplanationqualityinStudy3, Table5:Regressioncoefficientsforindividualitemanalysis including the Without Neuroscience condition from Study inStudy3(∗p<.05). 1. Errorbarsrepresent95%confidenceintervalsaroundthe Item1 Item2 Item3 Item4 means. Neuroscience 0.27∗ –0.09 0.24∗ 0.22∗ Study 3 Quality 0.55∗ 0.69∗ 0.17 –0.01 3 Group 0.04 0.32∗ 0.18 0.08 Good Explanations Bad Explanations 2 4.3 Discussion 1 Explanations with neuroscience information, whether pre- Rating sentedassimplyaspossiblewithoutjargonorwithreference Average 0 tiosfsypinegcitfihcanneeuxrpallatneacthionniqsuwesithanodutbnreauinroasrceiaesn,cweeirnefomrmoraetisoant-. However, there was no difference between the two neuro- −1 scienceconditions. Thissuggeststhatanyreferencetoneu- roscience is sufficient to cause the effect, and that adding −2 technicaljargondoesnotincreasesubjects’ratings. Additionally, subjects judged good explanations more −3 highlythanbadones,andundergraduatesubjectsgaveover- Without Neuroscience Simple Neuroscience Neuroscience Plus Jargon all lower ratings than MTurk workers. These effects repli- cate the findings of Study 1 and suggest two additional conclusions. First, people can generally discriminate good frombadexplanations. Second,participatinginresearchas partofone’seducationalexperienceseemstomakesubjects ofNeuroscience,andItem4hadanegative,non-significant moreskepticaloverall,butdoesnoteliminatetheseductive effectofQuality. allureeffect. Toanalyzesubjects’justifications,wesearchedforallref- These main effects were significant overall, but varied erencestoneuroscience,asinStudy2;24%ofalljustifica- somewhatbyitem. AsinStudy1,theattentionalblinkitem tionsreferencedthebrain,and58%ofthosedidsoinapos- (Item2)didnotshowaneffectofneuroscienceinformation andtheseeing/imaginingitem(Item4)didnotshowanef- itivemanner. A2(Group: MTurkworkers,undergraduates) fect of quality. In the case of the former, as noted above, x 2 (Condition: Simple Neuroscience, Neuroscience Plus the neuroscience information was contained in a separate Jargon)ANOVArevealednosignificanteffectsonsubjects’ sentenceratherthanbeingdirectlylinkedtotheexplanatory averagenumberofpositivebrain-basedjustifications. information. This may have made it easier for subjects to However, subjects’ justifications provide further insight realize that this information was irrelevant. In the case of into one of the item effects. Many of the justifications for latter, as in Studies 1 and 2, subjects seemed generally un- the attentional blink item (Item 2), which did not show abletotellthedifferencebetweenthegoodandbadversions a significant neuroscience effect, mentioned that the neu- ofthisitem. roscience information seemed unconnected with the phe- nomenon: e.g., “The explanation mentions the frontal lobe 5 General discussion areas but does not really say how the areas relate to atten- tional blink”, and “The explanation does not explain how The seductive allure effect of neuroscience, first observed frontalareasarerelatedtothetemporalrelationshipbetween by Weisberg et al. (2008), occurs when subjects judge that the two houses.” This item effect was consistent with the explanations for psychological phenomena (especially bad findings from Study 1, and justifications such as this sup- ones) that contain irrelevant neuroscience information are port our suggestion that putting the information about the betterthanexplanationsthatdonot.Thecurrentstudiespro- frontallobeinaseparatesentencemayhavemadeiteasier videnewinsightintowhythiseffecthappens. for subjects to separate this information from the body of First, although the original stimuli used to demon- theexplanation,explainingthelessereffectofneuroscience strate this effect confounded neuroscience information forthisitem. with length, our Study 1 and independent replications by

Description:
These results confirm that neuroscience information exerts a seductive effect on Keywords: explanation, neuroscience, reasoning, seductive allure.
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.