Artifacts in Behavioral Research: Robert Rosenthal and Ralph L. Rosnow's Classic Books

907 Pages·2009·4.95 MB·English

by  Robert Rosenthal

Artifacts in Behavioral Research
Robert Rosenthal and Ralph L. Rosnow’s Classic Books
This page intentionally left blank
Artifacts in Behavioral
Research
Robert Rosenthal and Ralph L. Rosnow’s
Classic Books
A Re-issue of
ARTIFACT IN BEHAVIORAL RESEARCH
EXPERIMENTER EFFECTS IN BEHAVIORAL RESEARCH
and
THE VOLUNTEER SUBJECT
With a Foreword by
Alan E. Kazdin
1
2009
1
OxfordUniversityPress,Inc.,publishesworksthatfurther
OxfordUniversity’sobjectiveofexcellence
inresearch,scholarship,andeducation.
Oxford NewYork
Auckland CapeTown DaresSalaam HongKong Karachi
KualaLumpur Madrid Melbourne MexicoCity Nairobi
NewDelhi Shanghai Taipei Toronto
Withofficesin
Argentina Austria Brazil Chile CzechRepublic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
SouthKorea Switzerland Thailand Turkey Ukraine Vietnam
Copyright#2009byOxfordUniversityPress,Inc.
PublishedbyOxfordUniversityPress,Inc.
198MadisonAvenue,NewYork,NewYork10016
www.oup.com
OxfordisaregisteredtrademarkofOxfordUniversityPress
Allrightsreserved.Nopartofthispublicationmaybereproduced,
storedinaretrievalsystem,ortransmitted,inanyformorbyanymeans,
electronic,mechanical,photocopying,recording,orotherwise,
withoutthepriorpermissionofOxfordUniversityPress.
LibraryofCongressCataloging-in-PublicationData
Rosenthal,Robert,1933–
Artifactsinbehavioralresearch:RobertRosenthalandRalphL.Rosnow’sclassicbooks/RobertRosenthal,RalphL.Rosnow.
p. cm.
Newcombinationvolumeofthreebooks-in-one.
Includesbibliographicalreferencesandindex.
ISBN978-0-19-538554-0(alk.paper)
1. Psychology—Research. 2. Socialsciences—Research. I. Rosnow,RalphL. II. Title.
BF76.5.R6352009
150.72—dc22
2008038527
1 3 5 7 9 8 6 4 2
PrintedintheUnitedStatesofAmerica
onacid-freepaper
Foreword
We are extremely fortunate to have this three-in-one volume. To begin, Robert
RosenthalandRalphRosnowhavebeencollaboratingfornearly50yearswithstellar
contributions to research design, statistical evaluation, and methodology. This
volume conveys their enduring dedication to advancing and improving behavioral
research.Theyhaveaccordedspecialattentiontothestudyofartifactsinresearch,
the topic of this volume. The volume provides a convenient way to bring the
remarkablebodyofworkthathasfallenoutofprintand,evenworse,outofmind.
The volume will be extremely valuable for individuals in training as well as for
experiencedresearchers.Eachwillrelishintherichmaterialwhetherthisisnewor
rekindledknowledge.
Artifactinthecontextofresearchreferstoinfluencesthatarenotofdirectinterest
totheinvestigator.Fromamethodologicalstandpoint,considerationofandattention
to artifacts are no less critical to the design of a study than to the phenomenon of
direct interest. An uncontrolled or neglected artifact can account for, explain, and
competewiththeinterpretationtheauthorwishestoplaceonthedata.Perhapsthe
most familiar artifact in intervention research is the placebo effect. In medication
trials, individuals who receive a fake drug, that is, one with no pharmacological
properties that could influence the outcome, still improve and improve more than
thosewhoreceivenodrugatall.Becausetheinvestigatorisinterestedintheeffects
of medication, placebo effects become an influence to be controlled. Although
placebo effects are the widely recognized poster child of artifacts, there are many
otherartifactslesswellknownordiscussed.RosenthalandRosnowhavelikenoone
before codified a large set of artifacts in behavioral science but also science more
generally and showed that they in fact can exert significant impact on human and
non-humananimalbehaviorandtheresultsofinvestigations.
Insufficient attention has been paid to artifacts and their influence. Among the
reasons, the very term conveys they are not the main show or interest. A particular
phenomenonsparkstheinterestandcuriosityofinvestigatorsandperhapsevendraws
themintothefield;artifactsseemtobeallthoseotherfactorsthatmerelydousethe
flame.Afterall,unlessoneisananthropologistwhyshouldonecareaboutartifacts?
Theansweristhreefold.Firstandperhapsthemostobviousisthemethodological
part. Artifacts are influences that must be controlled or addressed to draw valid
inferences.Itisofteneasytocompletearesearchprojectthatfocusessolelyonthe
phenomenonofinterestandneglectallsortsofplausibleotherinfluences.Onecan
thendrawconclusionsabouttheresultsandplaceaperfunctorylimitationsparagraph
that gives a passing curtsy to what was not controlled. The question for any
researcher before a study is conducted is what might be an alternative explanation
of the findings if the intervention/independent variable has the predicted impact.
Thosecodifiedasartifactsaresomeofthemorenuancedanswerstothequestionand
hence readily overlooked. Although we may not be interested in a particular
v
vi Foreword
experimentalartifact,wecertainlydonotwantoneofthemtobeamoreorequally
parsimoniousinterpretationofourresults.
Second, artifacts are not artifacts. They raise critical substantive issues about
subtle factors that influence performance. Beliefs, attributions, expectancies,
demandcharacteristics,subjectroles,testsensitization, andmore,allelaboratedin
this volume, are not mere influences to be controlled. They reflect influences—
genuine and often more subtle—that need to be understood and harnessed. For
example, volunteer subjects behave differently from those who are not volunteers.
There is more to this than demographic differences. Influences such as choice,
thresholdforagreeingtoparticipate,andwillingnesstotakeriskornotseeingrisk
all are critical issues pregnant with theoretical and applied implications. For those
artifactsthatareaboutdataanddataanalysis,someofthese(e.g.,biasesinobserving
andscoring,datafabrication)relatetothepillarsofscientificresearch,wellbeyond
psychology. Understanding more about how they take place and how they can be
preventediscriticallyimportant.
Third,RosenthalandRosnowhavealertedustotherelativityofartifact.Artifacts
are what one is not interested in, but should worry about, from a methodological
standpoint. If I study placebo effects, the last thing I want is having my subjects
takingrealmedicationsthatIdonotknowabout.Placeboismyvariableofinterest,
and real medication (e.g., taking pills at home to treat their disorder, using some
alternativemedicine)istheartifact.Evenifallsubjectsaretakingdifferentmedica-
tionsorsomeareandsomearenottakingmedication,thiscaninterferewithdrawing
validconclusions.(Largevariabilitythatthesealternativetreatmentsintroducecan
readily reduce power and interfere with demonstrating an effect.) In other words,
consideraclasscalled,‘‘allvariablesthatinfluencehumanandnon-humananimal
functioning.’’Inourindividualworkweselectthoseweareinterestedinforstudy;
somesubsetoftheothersinwhichwearenotveryinterestedaretheartifacts.And,
more colloquially one person’s artifact is another person’s independent variable.
I mention this because understanding artifacts and their nuanced effects has broad
generalityinunderstandinghumanbehavior.
The present volume elaborates the issues I have raised and makes a connection
that to me represents the highest level of sophistication in our field. The volume
focusesonartifactbutasthereaderwillsee,threebroadtopicsareincluded:theory,
research design, and data analysis. By elaborating artifacts, we learn that we need
new concepts, new theory to explain the effects, new design variations, and new
methodsofdataanalysis.RosenthalandRosnowhavemovedsocialsciencestonew
heightswiththisworkbyinfluencingandintegratingthreebroadtopics.
Onamorepersonalnote,overtheyearsIhavedeeplyadmiredthecontributionsof
RosenthalandRosnow.Weknowfromanalysesofsciencethatqualityandquantity
ofpublicationsoftengohandinhand(e.g.,CharlesDarwin,AlbertEinstein,Gustav
Fechner).WecanaddRosenthalandRosnowtothislistasthisthree-inonevolume
illustrates. However, the larger context cannot be neglected by noting many other
productsoftheircollaboration,including:
• BeginningBehavioralResearch:AConceptualPrimer(RosnowandRosenthal)
• ContrastAnalysis:FocusedComparisonsintheAnalysisofVariance(Rosenthal
andRosnow)
• Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach
(Rosenthal,Rosnow,andRubin)
Foreword vii
• EssentialsofBehavioralResearch:MethodsandDataAnalysis(Rosenthaland
Rosnow)
• PeopleStudyingPeople:ArtifactsandEthicsinBehavioralResearch(Rosnow
andRosenthal)
• PrimerofMethodsfortheBehavioralSciences(RosenthalandRosnow)
• UnderstandingBehavioralScience:ResearchMethodsforResearchConsumers
(RosnowandRosenthal)
Mylist,ofcourse,omitsmanyotherworks(e.g.,books,articles,andchapters)—
workseachhascontributedindividuallyoutsideofthisrichcollaboration.
The artifacts they have detailed are rich in hypotheses and could accommodate
legions of dissertations and careers as reflected in such generic questions as, How
they work? Forwhom they work? And how their influence can be harnessed?Our
field and sciences more generally would profit from knowing the moderators,
mediators, and mechanisms of action. Moreover, relating these artifacts to other
influences(e.g.,socialnorms,priming)wouldcontributegreatlytotheory.Rosenthal
andRosnowpresentuswithrevolutionaryconcepts,insights,andchallenges.Ihave
beengreatlyinfluencedbytheirworkandamsodelightedtohavetheopportunityto
expressmygratitudetothempublicly.
I have read these volumes and urge you to as well, first selectively to suit your
tastesandresearch,butthenfullytoenrichthequalityofone’sownresearchandthe
carewithwhichoneinterpretsallresearch.Thescholarshipissuperb,theissuesare
provocative, and the artifacts still wait to be exploited to understand the range of
influences on human functioning. It would be difficult to read the volume without
shiftingalittleofone’sworktothestudyorcontroloftheinfluencestheseauthors
elaborate.
AlanE.Kazdin,Ph.D.
YaleUniversity
September2008
This page intentionally left blank
A Preface to Three Prefaces
We are as sailors who are forced torebuild their ship on the open sea, without
everbeingabletostartfreshfromthebottomup.Whereverabeamistakenaway,
immediatelyanewonemusttakeitsplace,andwhilethisisdone,therestofthe
shipisusedassupport.Inthisway,theshipmaybecompletelyrebuiltlikenew
with the help of the old beams and driftwood—but only through gradual
rebuilding.
OttoNeurath1
Otto Neurath’s brilliant simile of science captures beautifully the structure and
evolution of science. New ideas, new theories, new findings, new connections,
new methods—all of these constitute the new beams that replace the old.
Sometimestheoldbeamsweren’tsobad;it’sjustthatsomeneweroneswereabit
better.Fromtimetotimeitwasn’tnoticedthatparticularbeamswereweakeningthe
ship.Artifactsthatweakentheshipofscienceoftengounnoticed.Thethreebooks
republishedinthisvolumewereoriginallyintendedtoencouragescientiststonotice
thebeamsweakenedbyartifactsandtothinkabouthowtheweakenedbeamsmight
bereplacedbystronger,thoughstillimperfect,beams.
Inthepresentcontext,thethreebookscanserveasanintroductionandareminder.
They are an introduction to the topic of artifacts for graduate students, advanced
undergraduates,andyoungerresearchers.Theyarearemindertomoreexperienced
researchers inside and outside of academia that the problems of artifacts in beha-
vioral research, which they may have first encountered as beginning researchers,
havenotgoneaway.
For example, problems of experimenter effects have not been solved.
Experimenters still differ in the ways in which they see, interpret, and selectively
presenttheirdata.Experimentersstillobtaindifferentresponsesfromtheirresearch
participants (human or infrahuman) as a function of the experimenters’ transitory
states as well as their enduring traits of biosocial, psychosocial, and situational
origins. Similarly, experimenters’ expectations still are manifested too often as
self-fulfilling prophecies. Biomedical researchers seem to have acknowledged and
guarded against this problem better than have behavioral researchers, as many
biomedical experiments would be considered of dubious scientific value had their
experimentersnotbeenblindtoexperimentalcondition.
Problemsofparticipantorsubjecteffectshavealsonotbeeneliminated.Research
samples are, by necessity, usually drawn from a population of volunteers. These
volunteersubjectsoftendifferalongmanydimensionsfromthosenotfindingtheir
wayintotheresearch.Attimes,thosevolunteersubjectsmaydifferfromnonvolun-
teers in the degree to which they are affected by the experimental conditions.
1Neurath, O.(1921).Antispengler(T.Parzen,Trans.).Munich:Calwey,pp.75–76.
ix