The Testimony of Forensic Identification Science

18 Pages·2008·0.31 MB·English

LawHumBehav
DOI10.1007/s10979-008-9169-1
ORIGINAL ARTICLE
The Testimony of Forensic Identification Science: What Expert
Witnesses Say and What Factfinders Hear
Dawn McQuiston-Surrett Æ Michael J. Saks
(cid:1)AmericanPsychology-LawSociety/Division41oftheAmericanPsychologicalAssociation2008
Abstract This research examined how variations in the Forensic identification (or ‘‘individualization’’) science
presentation of forensic science information affect fact- rests upon the notion that certain attributes of inanimate
finders’ judgments in a trial. Participants read a summary objects, persons, or even the residuum of certain behavior
ofamurdercase,thecriticaltestimonybeingtheresultsof (e.g., handwriting) contain so much variability that it is
a microscopic hair comparison given by a forensic expert. possibletoassociate evidence fromacrimescene with the
Across two experiments we manipulated how the expert source of that evidence. For much of the past century fin-
expressed his results, whether he gave an explicit conclu- gerprint identification was the archetypal example of
sion concerning identity of the hair, and whether the forensicidentification—thoughitwasprecededhistorically
limitations of forensic science were expressed during trial. by handwriting comparison (Risinger, Denbeaux, & Saks,
Qualitative testimony was more damaging to the defense 1989)andanthropometry(Cole,2001),andhasbynowbeen
than quantitative testimony, conclusion testimony displacedbyDNAtyping.Thesetechniquesareunderstood
increasedthedefendant’sculpabilityratingswhenfindings in popular culture, by most courts, and even by some
werepresentedquantitatively,andexpressinglimitationsof forensic practitioners, as being able to link crime scene
forensic science had no appreciable effect. Results are evidencetoitsprecisesource‘‘totheexclusionofallothers
discussed in terms of factfinders’ interpretation of forensic intheworld’’andtodosoflawlessly.Asoneappellatejudge
identification evidence. commented inregard tofingerprints:‘‘In essence, we have
adopted a cultural assumptionthat a governmentrepresen-
Keywords Forensic science (cid:1) Expert testimony (cid:1) tative’sassertionthatadefendant’sfingerprintwasfoundat
Juror and judge decision-making a crime scene is an infallible fact, and not merely the
examiner’sopinion’’(Statev.Quintana,2004).
But numerous scientists have long realized that—for
reasons physical, statistical, and psychological—such
Portionsofthesedatawerepresentedatthe‘FacesofForensics: credulousness reflects a fundamental misunderstanding of
IdentificationandBehavior’Conference,UniversityofCalifornia, these techniques (Balding, 2005; Cole, 2001; Cummins &
HastingsCollegeoftheLaw,SanFrancisco,CA,March2008,andthe
Midlo, 1943; Dror & Charlton, 2006; Dror, Charlton, &
meetingoftheSocietyforAppliedResearchinMemoryand
Peron, 2006; Faigman, Kaye, Saks, Sanders, & Cheng,
Cognition,Lewiston,Maine,July2007.
2007;Risinger,Saks,Rosenthal,&Thompson,2002;Saks
D.McQuiston-Surrett(&) & Koehler, 2005, 2008; Thompson & Cole, 2006; Zabell,
DivisionofSocialandBehavioralSciences,ArizonaState
2005). As two prominent forensic scientists (Thornton &
University,4701W.ThunderbirdRd,85306Glendale,AZ,USA
Peterson, 2007) have written: ‘‘[T]hough individualization
e-mail:[email protected]
isclearly thegoaltowardwhich forensicscience strives, it
M.J.Saks can be achieved only in a probabilistic sense, of reducing
SandraDayO’ConnorCollegeofLaw,CenterfortheStudyof
uncertainty to the smallest possible amount…. (p. 7)
Law,ScienceandTechnology,ArizonaStateUniversity,
Behindeveryopinionrenderedbyaforensicscientistthere
P.O.Box877906,85287-7906Tempe,AZ,USA
e-mail:[email protected] is a statistical basis. We may not know what that basis is,
123
D.McQuiston-Surrett,M.J.Saks
and we may have no feasible means of developing an We chose microscopic hair identification as a suitable
understandingofthatbasis, butitis futile todenythatone vehicle for exploring the effects on factfinders’ judgments
exists’’ (p. 22). of various ways that forensic examination findings could
Thus,ifanexaminationisdoneproperlyandresultsina be presented. Microscopic hair identification involves the
conclusionthatthecrimesceneevidenceandthesuspect’s visual comparison of one or several hairs recovered from
evidence appear indistinguishably similar (they ‘‘match’’), a crime scene with a sample of hairs taken from a sus-
then the suspect’s evidence along with some unknown pect. Examiners compare numerous physical features of
numberofothersinthepopulationconstituteapoolwhose the sampled hair looking for similarities and differences.
attributes are shared by each other and by the crime scene The attributes of hair vary considerably within persons as
evidence.Thesizeofthatpoolofcandidatesiscriticaltothe well as between persons. (Two hairs from the same per-
diagnosticityoftheidentification:thesmallerthepool, the son can appear to have come from different persons,
greater the probability that the suspect is the source; the while two hairs from different persons can appear to have
largerthepool,thesmallertheprobability.Togiveasimple come from the same person.) Examiners must rely on
illustration: A murderer’s blond hair left at the scene of a their perception and judgment to assess whether one
crimewouldbefarmorediagnosticinChinathanitwould group of hairs is similar enough to another group to think
be in Sweden. Alone among forensic identification tech- that they could have come from the same individual. Like
niques, DNA typing deals with this problem explicitly, most forensic identification scientists, microscopic hair
employing data on the population frequencies of the attri- examiners must rely on their intuitions and experience
butes employed tocompare crime scene DNA tosuspects’ with other case samples to judge how common or rare the
DNA, resulting in a calculated random match probability feature sets are that they are looking at and making
(RMP).TheRMPisthe probabilitythatapersonorobject judgments about. The same is true for numerous other
selected at random from the population would have the forensic identification sciences—among them fingerprints,
same attributes as the crime scene person or object. Prac- handwriting, bitemarks, tiremarks, shoeprints, toolmark
titioners of all other forensic identification sciences rely comparison, and so on. Further, because most people are
insteadonanintuitivefeelingaboutthesizeofthatpool,or unfamiliar with the methods and conclusions of hair
simply assume all such pools consist of n = 1 (and there- examination, there is little risk that any of our manipu-
fore that amatch constitutesa unique identification). lations will be at odds with our participants’ prior
Expertsandcourtswhichseektoinsurethatinformation understandings of what hair examiners do or how they
about the nature and results of forensic identification report their findings. These circumstances allow us con-
examinations are properly understood by factfinders (jud- siderable flexibility in presenting hair examination results
ges and jurors) confront a number of challenges. to judges and jurors in a variety of forms. (For further
Factfinders usually hold mistaken assumptions about details about microscopic hair examination, see Gaudette,
uniqueness. Experts in most forensic identification fields 1976, 1978; Gaudette & Keeping, 1974; Houck & Bud-
have no empirical RMPs to share, but only intuitive gu- owle, 2002; Moenssens, Starrs, Henderson, & Inbau,
estimations which they customarily express in qualitative 1995; Scheck, Neufeld, & Dwyer, 2000; Williamson v.
terms. While someforensicscienceexpertwitnessestryto Reynolds, 1995.)
tempertheimpressionsfactfinderswillderivefromhearing
thatthecrimesceneevidenceandthedefendant’sevidence
match,othersreinforcethemisunderstandingtoachievean FORM OF TESTIMONIAL PRESENTATION
exaggeratedbeliefthatthedefendantwasthesourceofthe
crime scene evidence to the exclusion of all others in the Of central interest to our research are the terms in which
world (see, e.g., United States v. Green, 2005; Williamson the expert witness’s observation of a link between the
v. Reynolds, 1995). In short, how forensic identification defendant and the crime scene evidence is presented. (a)
experts express their observations, how they express that One of the most misleading forms is to say that the sus-
opinion,howtheyexplainwhatitmeans,andwhattheysay pect’s evidence matches the crime scene evidence. The
it implies, can be expected to have important effects on term ‘‘match’’ might be expected to suggest to factfinders
what factfinders conclude from the evidence. That is the (given widespread cultural (mis)understandings) that the
focus of our research: the impact on trial factfinders of defendant, and only the defendant, is in fact the source of
differentwaysofexpressingthemeaningoftheresultsofa the crime scene evidence. (b) Another form of the same
forensicidentificationexamination.Thestudiesreportedin observation appears to be less conclusory and more
this article test for possible effects of several variables descriptive, that is, for the witness to testify that the
whichwouldseemtobecentrallyrelatedtothenatureand defendant’shairandthe crimescenehairare similarinall
impact of forensic identification testimony in trials. microscopic characteristics.
123
TheTestimonyofForensicIdentificationScience
To assess how these qualitative forms of examination rates that should have led to substantial changes in infer-
resultsmighthavedifferenteffectsthanwouldquantitative ences (e.g., Goodman, 1992; Faigman & Baglioni, 1998;
results of the kind that have become familiar in DNA Koehler, 1996; Nance & Morris, 2002, 2005; Schklar &
typing, we presented several different forms of RMP tes- Diamond,1999;Smith,Penrod,Otto,&Park,1996).Onthe
timony.(c)Themostcandidtestimonymightofferfrankly otherhand,peoplehavebeenfoundtobe(overly)sensitive
subjective probability estimates. Such testimony would totheforminwhichthematchstatisticsandotherelements
provideaquantitativeestimateofaRMP,whileatthesame of the forensic science evidence is presented (Hoffrage,
timebeinghonestaboutthefactthatsuchestimatesarethe Lindsey, Hertwig, & Gigerenzer, 2000; Koehler, 1996,
product of the examiner’s subjective judgment—a guesti- 2001; Koehler & Macchi, 2004; Lindsey, Hertwig, &
mate. (These three are currently possible testimony, Gigerenzer, 2003;Thompson &Schumann,1987).
whereasthenexttworequiredatawhichdonotyetexistin The hint that those studies offer to our own research,
the field of microscopic hair comparison, or most other which is focused on nonscientifically based forms of
forensic identification sciences (Moenssens et al., 1995; expertise and nonquantitative testimony, is that factfinders
Saks&Koehler,2005,2008;Thornton&Peterson,2007)). might not be sensitive to variations that they (logically)
We can also imagine that the necessary population data ought to be sensitive to, and might respond to differences
will some day come into being and hair examiners will be in form that ought to make no difference. Moreover, fact-
able to report objective, data-based random-match infor- finders sometimes or often intuit (something like)
mation.1 These data could be presented in a number of baserates, so that when they are given actual data they
different forms, but two that have been found to produce appear less rather than more impressed. This suggests that
widely differing effects seemed worthy of replication. the present studies might see factfinders giving unexpect-
Following Koehler (2001), we had our microscopic hair edly heavy weight to the qualitative, subjective testimony
expert present his objective probability findings either as typical of most forensic identification expert evidence.
(d) a single-probability, that is, with the target being the That expectation is further reinforced by the findings of
defendant and the data framed as a probability, or (e) a Krauss and Sales (2001), who in the context of testimony
multi-frequency, with the target being the relevant com- aboutthepredictionofdangerousnessmanipulatedthetype
munity and the frame being the number of people in that of expert testimony presented (clinical versus actuarial)
community who are expected tohave the same identifying and found that mock jurors were more influenced by the
attributes. qualitative, less-scientific version.
We are aware of no research which directly tests the
impact of qualitative forensic identification expert testi-
mony presented using variations of form such as those IMPLICIT VERSUS EXPLICIT CONCLUSIONS
described above. Several related lines of research exist
which might offer clues to such effects. The most well- A second major factor that might affect the decisions of
developed body of related research is concerned with factfinders is whether or not the expert witness asserts an
factfinders’ reactions to quantitative evidence, especially explicit conclusion regarding the identity of the defendant
probability data about baserates, usually bearing on DNA as being the source of the crime scene evidence (in our
typingorserologicalmarkers.Thisbodyoffindingscanbe experiments, the perpetrator’s hair left at a crime scene).
summed up by saying that lay factfinders do not use the One can think of this independent variable as being a
evidence to reach optimal inferences, and can be further version of implicit versus explicit conclusions in persua-
swayed by fallacious arguments by attorneys (e.g., sive communications, and look to classical findings of the
Thompson,1989;Thompson&Schumann,1987),thatthey social psychology of attitude change. Those studies have
usually give insufficient weight to the evidence, and are foundthat,exceptunderunusualcircumstances,amessage
insensitive to population baserates and laboratory error is more persuasive when it includes explicit conclusions,
rather than leaving implicit the conclusion to which it
1 The advent of these data depends upon developing adequate leads.Onecanalsothinkofthisasaproblemoftheimpact
measurementsoffeaturesandcollectingsamplesfromthepopulation of testimony on the ultimate issue in a case. When the
withwhichtodevelopdatabasesofbaseratedata.Howsoonthiswill
centraldisputedissueinatrialiswhetherthedefendanthas
happen depends on progress within each forensic identification
been correctly identified as the perpetrator, then expert
subfieldand/orexternalpressures,suchasfromcourts.Forexample,
it has already largely come about in one subfield (at least among testimony on identity is, for all practical purposes, an
European and Australian forensic scientists), voice identification, a opinion on the trial’s ultimate issue. A number of studies
fieldwhichhadbeenexcludedbymanycourtsandgrownmoribund.
have found that expert evidence has a greater impact on
Developing these databases and employed likelihood ratios restored
factfinders to the extent that the witness ties the evidence
the subfield to life (see, e.g., Gonzalez-Rodriguez, Rose, Ramos,
Toledano,&Oretega-Garcia,2007). explicitly to issues in the case (e.g., Brekke & Borgida,
123
D.McQuiston-Surrett,M.J.Saks
1988). But we have found only one study that deals with greaterontheactuarialthantheclinicalopiniontestimony.
forensic identification, and does so in relation to the ulti- Other research examining whether misleading testimony
mate issue of identity. predictingfuturedangerousnesscouldbeneutralizedfound
In a series of experiments, Wells (1992) found that that neither a strong cross-examination nor a strong cross
factfinders would adopt the approximately correct sub- plus an opposing expert were able to reduce the prosecu-
jective probability about the identity of a civil tortfeasor, tion expert’s persuasiveness or the percentage of juries
but would not use that probability estimation to find the opting for death (Diamond, Casper, Heiert, & Marshall,
identified defendant liable—except in one circumstance: 1996). Using a case involving expert testimony on the
when an expert witness was willing to add that he, the veracity of a child witness alleging sexual abuse, Kovera,
witness, ‘‘believed that the bus that ran over Mrs. Prob’s Levy, Borgida, and Penrod (1994) varied the strength of
dog was a Blue Bus Company bus.’’ With this one small thedefense’scross-examinationoftheexpert.Theirresults
addition, factfinders were willing to find against the showed that the cross did not affect participants’ percep-
defendant. Those hearing the evidence without that addi- tions of the quality of the evidence, nor did it affect their
tional statement reached the same subjective probability verdicts. This result was replicated in a study in which the
estimatesofthelikelihoodthattheBlueBusCompanywas expert testimony concerned a lawsuit about a hostile work
the offender. But verdicts against the Blue Bus Company environment, and a number of factors were varied related
were rendered only in the presence of the additional testi- tothequalityoftheresearchbeingpresentedbytheexpert
monial statement. Wells’s findings suggest that factfinders (Kovera, McAuliff, & Hebert, 1999). Even a scientifically
are more likely to accept the liability implications of sound and sophisticated cross-examination, compared to a
forensic identification testimony when the expert witness controlcondition,hadnoeffectonjurors’judgmentsofthe
makes conclusoryassertions aboutthe defendant being the validity of the expert’s evidence.
perpetrator—even when, as in Wells’s experiment, it is Generally,thesestudieshavefoundlittleornoabilityof
plain that the expert knows nothing more than the fact- cross-examination to undo the effects of an expert’s testi-
finders do about the inferential step from the evidence to monyondirect-examination,evenifthedirecttestimonyis
the ultimate conclusion of identity. fraught with weaknesses and the cross is well designed to
expose those weaknesses. The results suggest that it is
unlikely that defense cross-examination (or even opposing
LEARNING ABOUT LIMITATIONS experts, e.g., Levett & Kovera, 2008) will reduce the
OF THE EXPERTISE impact of the forensic expert witness’s direct testimony.
We therefore included a condition where the judge—pre-
Thethirdfactortestedinourexperimentswastheeffecton sumably the most credible force in the trial—instructs the
jurors of being informed of limitations in the foundations factfinders regarding the weaknesses of the expert evi-
of the expert testimony, implying that less reliance should dence. The handful of studies which have examined the
be placed on the testimony. The most common way for effects of instruction on ability to reason with forensic
jurors to learn of such limitations is through cross-exami- science statistics yield mixed results. In a DNA context,
nation of the witness. A less common means of learning Rowe(1997)showedthatjurorsdoabetterjobaggregating
about weaknesses of the expert field is through an moderatelyprobativeevidence with nonscientific evidence
instructionbythecourt(e.g.,UnitedStatesv.Starzecpyzel, when presented with a relevant judicial instruction. How-
1995). In one of our experiments we tested whether cross- ever, Smith et al. (1996) did not find that a similar type of
examination or judicial instruction weakened the strength instruction was helpful in a case that included blood and
oftheinferencesjurorsmadeaboutthelinkagebetweenthe enzyme evidence.
defendant and the crime scene evidence.
Anumberofstudieshaveexaminedtheimpactofcross-
examination, involving expert witnesses testifying about JUDGES VERSUS JURORS
matters other than forensic identification. For example,
Krauss and Sales (2001) examined clinical or actuarial Ourresearchparticipantsincludedjudgesaswellasjurors,
expert testimony regarding the future dangerousness of a all drawn from the same State. In the first of our two
capital murder defendant, with both experts then con- experiments, samples of judges and jurors were presented
fronted by opposing expert testimony and cross- with identical descriptions of trial evidence, incorporating
examinationpointingoutthehigherrorratesofpredictions identical experimental manipulations. Asking whether
of future dangerousness. Cross-examination reduced mock judges and jurors reach different conclusions on the same
juror estimates of future risk from where they were after forensic science evidence is of relevance because some
hearing the first expert’s testimony, but this influence was defendants choose to waive their right to jury trial and
123
TheTestimonyofForensicIdentificationScience
insteadtobetriedbeforeajudgealone.So,inbenchtrials, examiner’s use of the more culturally weighted term
judges are the factfinders. In addition, judges have (or ‘‘match’’? What happens if instead of a conventional
could have) more opportunity to learn about and to think qualitative conclusion the examiner were to make explicit
about the value and limitations of forensic identification theprobabilisticnatureoftheinferencesthatcanbedrawn
science—not only because they observe more trials than from the examination by explaining and offering the
jurors do, but because judges have in recent years been probability of a random match? Does it make a difference
assigned a larger role in evaluating the quality of forensic if the RMP is a subjective guestimate or an objective cal-
science for purposes of determining admissibility (see culation based on empirical evidence? We were interested
Faigman et al., 2007 [especially volume 1, chapter 1 and inwhathappenswhenafieldmovesfromnoRMPsatallto
much of volume 4]). giving RMPs. Does it matter whether those RMPs are
Most studies that have compared the fact-finding and subjective guesses rather than objective calculations based
decision-making of judges to that of jurors have found no on data?
differences. These studies have included the ability to Second, we tested the effect of a forensic hair expert
disregard inadmissible evidence, to avoid taking cognitive giving an explicit conclusion about the ultimate factual
shortcuts(heuristics)thatleadtoerrors,todrawinferences issue in the case (the identity of the perpetrator) versus
from probabilistic evidence, to analyze the substance of leavingthatjudgmentofidentitytofactfindersaftergiving
scientificevidence(centralprocessing)ratherthanfocusing themtheobservationalfindingsoftheforensicexamination.
onsuperficialaspectsoftheexpert(peripheralprocessing), After all, once the examiner properly shares his findings
to distinguish features that make research sound or with the court, the factfinders know as much as the expert
unsound, etc. In all of these, judges have been found to doesaboutthecontributionofthehairevidencetothecase;
perform very much like the jurors (see Champagne, Shu- theycanconnectittotherestoftheevidenceinthecaseand
man, & Whitaker, 1991; Guthrie, Rachlinski, & Wistrich, reach their own conclusion about identity and guilt.
2001; Kovera & McAuliff, 2000; Kovera et al., 1999;
Landsman & Rakos, 1994; Shuman, Whitaker, & Cham-
pagne, 1994; Wells, 1992). METHOD
Giventherangeoftasksinwhichjudgesandjurorshave
been found to think and perform similarly, we have no Participants
reasontoexpectthatjudgeswouldbehavedifferentlyfrom
jurors. But given the importance of judges to the expert Four hundred and twenty-five participants were tested. Of
witness task—screening expert evidence for admissibility this number, 128 were Arizona judges and other judicial
and sometimes serving as the finder of fact at trial—we officials attending a state judicial conference who volun-
thoughtit at least interesting todirectlycompare judgesto tarily participated in the experiment as a prelude to a talk
jurorsinourresearch,eveniftheresultwastoconfirmthe given by the authors. The remaining 297 were venireper-
expectation of no difference. sonscalledforjurydutybytheMaricopaCounty(Arizona)
Insum,then,ourexperimentstestedforpossibleeffects Superior Court (with 60.4% of the state’s eighteen-and-
on the judgments and decisions of jurors and judges when over population). They were recruited in the courthouse
presented with expert testimony communicating forensic just as they were released from service for the day, and
identification evidence in several different forms. In addi- paid $5 to remain and participate. This subset of partici-
tion, we test the possible effects of experts asserting a pants (56% females, M = 42.36 years old) included 73%
conclusion about the ultimate issue of identity, and of Whites,11%Hispanics,5%AfricanAmericans,3%Native
learning about limitations of the expertise through cross- Americans, 2% Asians, and 6% ‘‘other’’; this ethnic
examination or judicial instruction. breakdown of participating jurors approximately reflects
thepopulationfromwhichthissamplewastaken.Fourteen
percent indicated having served on a jury the day they
EXPERIMENT 1 participated in our study and 23% indicated they had
served on a jury on a prior occasion.
Experiment1examinedtheimpactonjudgesandjurorsof
variations in the presentation of the forensic expert’s Materials and Design
findings. First, we varied the language and concepts by
which the expert witness communicated the results of his Stimulus Case and Independent Variables
examination.Dofactfindersdrawadifferentunderstanding
from the conventional qualitative conclusion (‘‘the hairs The stimulus case was a murder trial presented in a one-
are similar in all microscopic characteristics’’) versus the page summary form. The focus of the trial was the
123
D.McQuiston-Surrett,M.J.Saks
testimony of a microscopic hair examination expert wit- thevariationsineachofthosecharacteristics.With
ness who asserted in one way or another that the that information, statistical calculations allow
defendant’s hair and the crime scene hair matched. The examiners to infer the likelihood that another
case involved a convenience store clerk who was mur- person in the population has hair that is the same
dered during the course of a robbery. No surveillance as that being tested in this case. The probability
video was available as the camera at the store was not thatapersonchosenatrandomfromthepopulation
functioning properly at the time of the robbery. A would have hair that is indistinguishable from the
bystander witnessed the crime but did not get a good look hair found in the ski mask is .001, or 0.1%. That
at the perpetrator’s face. A murder weapon was not helpsto assess the likelihood that the hairfound at
recovered. In canvassing the neighborhood, police identi- the crime scene was contributed by the defendant
fied several potential suspects who agreed to provide a or by someone else.
hairsample forcomparison with the hair recovered froma
(e) he gave an objective probability estimate framed as
ski mask dropped at the crime scene; this analysis linked
multi-frequency:
one of the suspects to the crime scene. Thus, the critical
evidence in the case was the comparison of the perpe- [I]f the two sets of hair are indistinguishably alike,
trator’s hair from the crime scene with hair sampled from then one needs to assess the significance of that
the defendant. fact. To do so, the field of hair comparison has
Our first experimental manipulation involved the expert gathereddataonthefrequencyinthepopulationof
witness on microscopic hair examination presenting his thevariationsineachofthosecharacteristics.With
findings in one of five ways: that information, statistical calculations allow
examiners to infer the likelihood that another
(a) he characterized the similarity between the crime
person in the population has hair that is the same
scene evidence and the defendant’s hair as being a
as that being tested in this case. About 1 in 1,000
match,
people have hair that is indistinguishable from the
(b) he characterized the findings as showing the hair
hair sample found in the ski mask in this case.
samplestobesimilarinallmicroscopiccharacteristics,
Since the population of the metropolitan area is
(c) he gave his subjective probability estimate of the
about3,000,000,therewouldbeabout3,000people
RMP, described as follows:
in it whose hair would also be indistinguishable
[I]f the two sets of hair are indistinguishably alike, from that found in the ski mask. That helps to
then one needs to assess the significance of that assessthelikelihoodthatthehairfoundatthecrime
fact. Since thefield ofhair comparison has nodata scene was contributed by the defendant or by
on which to base this assessment, each examiner someone else.
has to make a subjective judgment of the rarity or
For the second experimental manipulation, in half of
commonness of the hair in each case. In this case,
the conditions the forensic expert’s testimony went no
the examiner offered the estimate that hair with
further. In the other half the expert went on to offer an
these characteristics would be found in approxi-
ultimate conclusion on identity: ‘‘The examiner concluded
mately 1 in every 1000 people. By this estimation,
by offering the opinion that, based on his examination of
a person chosen at random would have a proba-
the hair in this case, that the defendant … was the
bility of .001, or a 0.1% chance, of having hair
source.’’
with the same characteristics as those of the hair
Theresultingdesignwasa2(Participants:Judgesversus
found in the ski mask. Since the population of the
Jurors) 9 5 (Findings Presented: Match versus Similar-in-
local metropolitan area is about 3,000,000, there
microscopic-characteristics versus Subjective-probability
would be about 3000 people in it whose hair
versus Objective Single-probability versus Objective
would be indistinguishable from that found in the
Multiple-frequency) 9 2(UltimateConclusion:Yesversus
ski mask. But this, the examiner noted, is nothing
No) fully randomized, between-groups factorial.
more than his best subjective estimation.
(d) he gave an objective probability estimate framed as
Dependent Measures Questionnaire
single-probability:
[I]f the twosetsofhair are indistinguishablyalike, Thequestionnaireincludeditemsconcerningthelikelihood
then one needs to assess the significance of that thatthedefendantwasthesourceofthecrimescenehairand
fact. To do so, the field of hair comparison has about their understanding of the forensic testimony (see
gathereddataonthefrequencyinthepopulationof Table 1). Two of the questions required participants to
123
TheTestimonyofForensicIdentificationScience
Table1 Dependentmeasures
*†1. What is your own best estimate of the probability (0% to 100%) that the hair which the forensic scientist analyzed came from the defendant? _____%
*2. How complex do you consider the forensic scientist’s testimony to be?
1 2 3 4 5 6 7
Not at all Extremely
complex complex
*3. Suppose this crime occurred in a city of 500,000 people. If every person in that city were tested, approximately how many people do you think would
have hair that is indistinguishably similar to the hair recovered from the crime scene? (Please provide a numerical estimate): ______
*4. How much does the microscopic hair examination evidence contribute to a finding of guilt in this case?
1 2 3 4 5 6 7
Not at all Extremely
much
*†5. How well do you understand how the forensic scientist came to his decision in the microscopic hair examination that was summarized?
1 2 3 4 5 6 7
Not very Extremely
well at all well
†6. How sure are you that the defendant is the person who committed the crime in this case?
1 2 3 4 5 6 7
Not at all Extremely
much
†7. How much does the microscopic hair examination evidence contribute to the strength of the case against the defendant?
1 2 3 4 5 6 7
Not at all Extremely
much
†8. When a forensic scientist finds a “match” between crime scene evidence and a suspect, his next step is to consider how many other people in the
population would also match the crime scene evidence. His conclusion that the suspect is the person who left the evidence at the crime scene would
be strongest when the size of the pool of others who would match is…
1 2 3 4 5 6 7
Extremely Extremely
small large
*DependentmeasuresusedinExperiment1
(cid:1) DependentmeasuresusedinExperiment2
generateanumericanswerandtheremainingthreequestions RESULTS
included7-pointLikert-typescalesforparticipantresponses.
Likelihood Defendant was the Source
Procedure
A major question of interest is what effect the different
Participants were randomly assigned to an experimental forms of presentation of the hair examination findings had
conditionandgivenabookletofmaterials.Theyfirstreada onparticipants’judgmentofthelikelihood(from0–100%)
consent form which explained the nature of the research that the defendant was the source of the crime scene hair.
andthatparticipationwasvoluntaryandanonymous(jurors Participanttypehadasignificantmaineffectonprobability
were also told they would be paid $5 for their participa- estimates, F(1,377) = 26.24, p\0.001, g2 = 0.07, with
tion).Theythenreadoverthecasesummaryandanswered jurors providing an overall higher probability that the
thedependentmeasuresquestionnaire(jurorsalsofilledout defendant was the source of the hair (M = 67.76;
a biographical questionnaire). They were later debriefed SD = 34.04) than judges (M = 48.30, SD = 38.86). The
and jurors were paid. type of presentation had a significant main effect on
123
D.McQuiston-Surrett,M.J.Saks
Table2 MeansandstandarddeviationsfordependentmeasuresforExperiment1a
Probabilityof Hairevidence Understandingofthe Complexityofthe Howmanypeoplewould
forensicmatch contributestoguilt forensictestimony forensictestimony matchcrimescenehair
Jurors(N=297)
‘‘Match’’ 77.36(27.05) 4.98(1.68) 5.17(1.56) 4.68(1.65) 48,066(96,978)
‘‘Similarinallmicroscopic 79.59(19.86) 4.80(1.71) 5.10(1.37) 4.79(1.46) 49,310(82,920)
characteristics’’
Subjectiveprobability 50.56(38.57) 4.42(1.86) 4.97(1.71) 4.31(1.58) 17,458(53,968)
Objectivesingleprobability 70.93(36.50) 5.37(1.33) 4.95(1.54) 4.27(1.84) 13,616(66,330)
Objectivemultiplefrequency 59.66(36.62) 4.51(1.73) 5.00(1.62) 4.49(1.61) 3,411(13,399)
Judges(N=128)
‘‘Match’’ 54.52(36.61) 4.79(2.04) 3.52(1.86) 3.24(1.64) 79,128(125,461)
‘‘Similarinallmicroscopic 55.53(25.77) 4.52(1.87) 3.16(1.89) 3.16(1.37) 84,805(96,868)
characteristics’’
Subjectiveprobability 30.88(35.45) 3.75(1.98) 3.75(1.67) 3.29(1.60) 547(1,101)
Objectivesingleprobability 66.18(43.69) 5.17(2.04) 4.59(1.74) 3.65(1.11) 9,441(19,384)
Objectivemultiplefrequency 35.50(43.41) 4.42(1.81) 4.73(1.82) 3.38(1.30) 1,182(1,723)
a ThedatapresentedinTable2arecollapsedacrosstheultimateconclusionvariable
probability estimates, F(4,377) = 9.41, p\0.001, type 9 presentation type interaction, F(1,404) = 4.73,
g2 = 0.07.2 Games-Howell post hoc tests revealed that p\0.05, g2 = 0.01. Follow-up analyses indicated that
findings presented by the expert in the form of match, judges’ responses concerning the evidence’s contribution
similar-in-all-microscopic-characteristics, or objective sin- toafinding ofguiltdidnotvaryasafunctionpresentation
gle-probability led to a significantly higher probability type,F(4,122) = 1.73,p[0.05,whereasjurors’responses
estimate that the hair the forensic scientist analyzed came did, F(4,292) = 3.10, p\0.02. The forensic expert’s
from the defendant than when presented in the form of a findings presented as an objective single-probability
subjective probability or objective multiple-frequency, (M = 5.37,SD = 1.33)wasmorepersuasivetojurorsthan
ps\0.033(seeTable 2formeansandstandarddeviations). findings presented as either a subjective-probability
How, in turn, does this probability judgment relate to (M = 4.42, SD = 1.86) or objective multiple-frequency
factfinders’ judgments about guilt? Rather than asking for (M = 4.51, SD = 1.73), ps\0.05.
verdicts,weaskedaslightlymorenuancedquestion:‘‘How Whether or not the expert gave an opinion on the ulti-
much does the microscopic hair examination evidence mate issue of identity did not have a significant effect on
contribute to a finding of guilt in this case?’’ on a scale the perceived probability of common source, or on the
from1to7(1 = notatall,7 = extremelymuch).Thetwo contribution of the expert evidence to a guilty verdict.
dependent measures were moderately highly correlated Moreover, there were no other significant interactions on
with each other, indicating that those who thought the these two dependent variables.
probabilityofacommonsourcewashighweremorelikely
to believe that the evidence that had been presented con- Understanding of the Testimony
tributedtoaverdictofguilt(r = 0.53,p\0.001).Typeof
presentation had a significant main effect on contribution- Another question of concern is how well participants
to-guilt estimates, F(4,404) = 4.46, p\0.01, g2 = .04, understoodthetestimony.Whenasked,‘‘Howwelldoyou
but this effect was qualified by a significant participant understand how the forensic scientist came to his decision
in the microscopic hair examination that was summa-
rized?’’onascalerangingfrom1to7(1 = notverywellat
2 Levene’s test for homogeneity of variance revealed a significant all, 7 = extremely well), the answers given varied signifi-
differencebetweenthevariancesacrossexperimentalgroupsonthis
cantly by participant type, F(1,403) = 36.84, p\0.001,
dependentmeasure,p\0.05.Visualinspectionofboxplotsindicates
that the interquartile range in both the subjective probability and g2 = 0.08, and by presentation type, F(4,403) = 2.58,
objectivemulti-frequencyconditionsisabouttwiceaslargeasinthe p\0.05, g2 = 0.03, but these main effects were qualified
threeremainingconditions.Toaddressthis,Games-Howellposthoc by a significant participant type 9 presentation type
testsforunequalvarianceswereused.
interaction,F(4,403) = 3.77,p\0.01,g2 = 0.04.Follow-
3 The difference between the objective single probability condition
up analyses showed that jurors’ responses concerning how
andtheobjectivemultiplefrequencyconditionapproachedstatistical
significance,p=0.06. well they understood the information were not affected by
123
TheTestimonyofForensicIdentificationScience
the form of presentation of the forensic evidence, bynearlyhalfoftheparticipantsinthoseconditions.Inthe
F(4,292) = 0.21, p[0.05, while judges’ responses were, objective single-probability condition, where a slightly
F(4,121) = 3.60, p\0.01. Judges reported a better more difficult calculation was needed to reach the correct
understanding of the forensic scientist’s testimony in the answer (multiplying a population of 500,000 by 0.001),
objective multiple-frequency condition (M = 4.73, only25.3% ofparticipantsgavethecorrect answerof500.
SD = 1.82) than in the similar-in-all-microscopic-charac- Participants in those three conditions gave estimates that
teristics condition (M = 3.16, SD = 1.87), p\0.02. In were a fraction of the size of the estimates given by those
general, the highest ratings of understanding tended to in the two conditions in which guessing was unavoidable.
resultfromthestrictlyquantitativepresentations,compared Accordingly, the estimates differed significantly as a
with the other three, more subjective presentations. Whe- functionoftheforminwhichthetestimonywaspresented,
ther or not the expert gave an ultimate opinion on identity F(4,363) = 11.61, p\0.001, g2 = 0.11.5 Games-Howell
had no impact on the participants’ perception of their own posthoctestsshowedthatcommunicatingthefindingsasa
understanding, nor were there any significant interactions. matchorsimilar-in-all-microscopic-characteristicsresulted
When asked, ‘‘How complex do you consider the in significantly higher estimates than when the findings
forensicscientist’stestimonytobe?’’onascalefrom1to7 werepresentedasasubjectiveprobability,objectivesingle-
(1 = not at all complex, 7 = extremely complex), jurors probability, or objective multiple-frequency, ps\0.02.
gave significantly higher ratings (M = 4.51, SD = 1.64) Results did not vary significantly based on the participant
than did judges (M = 3.34, SD = 1.42), F(1,402) = sample or whether the expert gave an ultimate opinion on
48.13, p\0.001, g2 = 0.11. No other main effects or identity, and the interactions were not significant.
interactions were statistically significant. Participants in the conditions which led to the highest
Another approach to testing understanding of the estimates that the crime scene hair came from the defen-
forensic evidence presented was to ask how many other dant paradoxically gave the highest estimates of the
people would match the hair, in addition to the defendant: incidence of the same hair traits in the reference popula-
‘‘Suppose this crime occurred in a city of 500,000 people. tion. This reinforces the inference that those two
Ifeverypersoninthatcityweretested,approximatelyhow testimonialconditionsleadtotheleastunderstandingofthe
many people do you think would have hair that is indis- basic concepts of forensic identification while leading to
tinguishably similar to the hair recovered from the crime the highest inculpatory judgments.
scene?’’.4 Understanding any differences in response
requires us to first consider the correct answer to the
question,whichis500,andtorecognizethatitwasreadily DISCUSSION
accessible toparticipants who hadbeen presentedwith the
testimonyinanyofthreeconditions:Intheobjectivemulti- Overall, jurors gave higher estimates of the source proba-
frequencyandsubjective-probabilityconditions,theexpert bilitythandidthejudges.Whenitcametovariationsinthe
stated that the number would be 3,000 in a city of presentation of the expert’s findings, judges and jurors
3,000,000,soarrivingattheproportionatenumberinacity alike were most impressed with the qualitative (match,
of 500,000 was a matter of simple arithmetic. In the similar-in-all-microscopic-characteristics) and objective
objective single-probability version of the testimony, the single-probability forms of testimonial presentation, infer-
arithmetic was slightly harder; the expert stated that the ring a higher probability that the defendant was the source
incidence rate in the population was 0.001, so the partici- of the crime scene hair in those three conditions over the
pants would need to multiply 500,000 by 0.001. Those in subjective probability and objective multiple-frequency
the match and similar-in-all-microscopic characteristics forms.Thisfindingsupportsthehypothesisthatpeopleare
conditions, like judges or jurors in real cases, would have more persuaded by qualitative information than quantita-
no recourse but a wild guess. tive information, as well as prior research demonstrating
Approximately half of the participants in two of the thatquantitativeinformationexpressedinprobabilityterms
conditions containing quantitative information (subjective ismoreinculpatorythanthatsameinformationpresentedin
probability, objective multi-frequency) gave exactly the frequency terms (e.g., Koehler, 2001).
correct answer of 500. This was true of 41.0% of those in Althoughbothjudges’andjurors’judgmentsconcerning
the subjective probability condition and 46.5% of those in source probability were influenced by the form in which
theobjectivemultiple-frequencycondition,suggestingthat
theinformationgivenintheseconditionswasgraspedwell 5 Levene’s test for homogeneity of variance revealed a significant
differencebetweenthevariancesacrossexperimentalgroupsonthis
dependentmeasure,p\0.05.Visualinspectionofboxplotsindicates
4 The data on this variable show significant positive skewness and that the interquartile range varied across all five conditions. To
kurtosis,indicatingnon-normality. addressthis,Games-Howellposthoctestswereused.
123
D.McQuiston-Surrett,M.J.Saks
the testimony was presented, these judgments did not The answer to this puzzle has something to do with the
necessarily carry over to ratings of the defendant’s guilt. way the problem is framed by the different experimental
Only the jurors’ judgments concerning the contribution of conditions, not that jurors as individuals tend to have a
the evidence to a finding of guilt varied as a function of backward understanding of the relationship between ran-
testimonial presentation: consistent with the above find- dommatchesinthepopulationandtheprobabilityofbeing
ings, jurors were more likely to think the evidence the source of the evidence. We can say this because the
supported a conviction when the expert’s findings were correlations reflect a logical understanding for participants
framedasanobjectivesingle-probabilitythanwhenframed in the match and similar-in-all-microscopic-characteristics
as either a subjective probability or objective multiple- conditions. This is true for both jurors (r = -0.38,
frequency.Judges’ratingsoftheevidence’scontributionto p\0.001)andjudges(r = -0.27,p = 0.12).Forjurorsin
guilt were not affected by the expert’s presentation of his the three probability conditions, however, confusion arises
findings. in that these two variables become uncorrelated. This is
Jurors rated their own understanding of the evidence as true for both jurors (r = 0.06, p = 0.46) and judges
being greater than the judges did, and at the same time (r = 0.14,p = 0.34).Paradoxically,bothjurorsandjudges
jurors rated the complexity of the forensic evidence as in the three probability conditions do not seem to see the
being greater than the judges did. Most likely, the judges relationshipbetweenrandommatchesinthepopulationand
either have a higher threshold for feeling that they the likelihood that the defendant is the true source of the
understand something or, because they have encountered hair,whilejurorsinthetwotraditionalconditionsdoseem
forensic identification evidence more often, it does not tosensetherelationship.Butwecansaythatwhateverthe
seem so complex (as it does to the jurors), but their explanation for the puzzle, it is one more product of the
puzzlement about it has grownmorenuanced. Overall, the experimental variable: how forensic identification testi-
highest ratings of understanding of the testimony were mony is presented.
associated with conditions containing quantitative infor- Surprisingly, whether or not the expert witness offered
mation, which is interesting since two of these three an ultimate opinion on identity affected none of the
conditions were also associated with the lowest ratings of dependent measures.
source probability. But it should not be surprising that
people find simplistic approaches (even if they are mis-
leadingly simple) easier to understand than approaches EXPERIMENT 2
that are more detailed, sophisticated, and empirically
based. Experiment 2 had two goals. First, it sought to discover
About half of the judges and jurors were able to accu- whether the understanding and impact of forensic identi-
rately calculate the estimated population frequency of fication testimony could be improved by providing
persons with hair characteristics matching those of crime information to the jurors about the limitations of such
scene hair in a city of 500,000 people in two of the three expertise. Specifically, jurors learned of the scientific lim-
conditionswhichprovidedthedatanecessarytomakesuch itations of forensic identification either through cross-
a calculation (namely, the subjective-probability and examination by the defense attorney or by an instruction
objective multi-frequency conditions), with about one- from the judge.
fourth of participants in the third quantitative condition Second, the previous experiment had failed to find any
(objective single-probability) calculating this number effectoftheforensicexpertofferinganultimateopinionor
accurately. These estimates were a fraction of the size of not. Before acceptingthissurprisingfinding,we wantedto
those given in the two conditions which provided no data make sure that the failure to find an effect was not the
(match and similar-in-all-microscopic-characteristics), and result of the manipulation being too subtle. Therefore, in
whichtherebyforcedparticipantstoguessatthepopulation this experiment, we sought to make the expert’s rendering
frequency.Thisleadstoapuzzle:whydojudgesandjurors of an ultimate opinion more pronounced.
in the match and similar-in-all-microscopic-characteristics The specific manipulations of the form of presentation
conditions, who think that so many random matches exist were limited to three: match and similar-in-all-micro-
in the reference population, arrive at such high probability scopic-characteristics are the two that are most commonly
estimates that the defendant is the source of the crime encountered under current practice, and subjective-proba-
scene hair? The logic of forensic identification is exactly bility is the only remotely possible option for the near
theopposite:wherethenumber(orprobability)ofmatches future. Calculations of objective RMPs must wait until the
in the population is greater, the probability that the day when sufficient population data on forensically rele-
defendant is the true sourceof the crime scene evidence is vant attributes are gathered so that they can be used in
smaller. routine casework.
123