Selection Bias and Covariate Imbalances in Randomized Clinical Trials

209 Pages·2005·1.289 MB·English

by  Vance W. Berger(auth.), Stephen Senn, Vic Barnett(eds.)

JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Selection Bias and
Covariate Imbalances in
Randomized Clinical Trials
Selection Bias and Covariate Imbalances in Randomized Clinical Trials V. W. Berger
(cid:1)C 2005 John Wiley & Sons, Ltd., ISBN: 0-470-86362-5 (HB)
i
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
STATISTICSINPRACTICE
AdvisoryEditor
StephenSenn
UniversityCollegeLondon,UK
FoundingEditor
VicBarnett
NottinghamTrentUniversity,UK
Statistics in Practice is an important international series of texts, which
providedetailedcoverageofstatisticalconcepts,methodsandworkedcase
studiesinspecificfieldsofinvestigationandstudy.
Withsoundmotivationandmanyworkedpracticalexamples,thebooks
showindown-to-earthtermshowtoselectanduseanappropriaterangeof
statisticaltechniquesinaparticularpracticalfieldwithineachtitle’sspecial
topicarea.
Thebooksprovidestatisticalsupportforprofessionalsandresearchwork-
ersacrossarangeofemploymentfieldsandresearchenvironments.Subject
areas covered include medicine and pharmaceutics; industry, finance and
commerce;publicservices;theearthandenvironmentalsciences,andsoon.
The books also provide support to students studying statistical courses
appliedtotheaboveareas.Thedemandforgraduatestobeequippedforthe
workenvironmenthasledtosuchcoursesbecomingincreasinglyprevalent
atuniversitiesandcolleges.
Itisouraimtopresentjudiciouslychosenandwell-writtenworkbooksto
meeteverydaypracticalneeds.Thefeedbackofviewsfromreaderswillbe
mostvaluabletomonitorthesuccessofthisaim.
Acompletelistoftitlesinthisseriesappearsattheendofthevolume.
ii
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Selection Bias and
Covariate Imbalances in
Randomized Clinical Trials
VanceW.Berger
UniversityofMaryland,BaltimoreCounty,USA
iii
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Copyright(cid:1)C 2005 JohnWiley&SonsLtd,TheAtrium,SouthernGate,
Chichester,WestSussexPO198SQ,England
Telephone [phone]
Email(forordersandcustomerserviceenquiries):[email protected]
VisitourHomePageonwww.wiley.com
AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem
ortransmittedinanyformorbyanymeans,electronic,mechanical,photocopying,recording,
scanningorotherwise,exceptunderthetermsoftheCopyright,DesignsandPatentsAct1988
orunderthetermsofalicenceissuedbytheCopyrightLicensingAgencyLtd,90Tottenham
CourtRoad,LondonW1T4LP,UK,withoutthepermissioninwritingofthePublisher.Requests
tothePublishershouldbeaddressedtothePermissionsDepartment,JohnWiley&Sons,Ltd,
TheAtrium,SouthernGate,Chichester,WestSussexPO198SQ,England,oremailedto
[email protected],orfaxedto([phone].
Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationinregardto
thesubjectmattercovered.ItissoldontheunderstandingthatthePublisherisnotengaged
inrenderingprofessionalservices.Ifprofessionaladviceorotherexpertassistanceis
required,theservicesofacompetentprofessionalshouldbesought.
OtherWileyEditorialOffices
JohnWiley&SonsInc.,111RiverStreet,Hoboken,NJ07030,USA
Jossey-Bass,989MarketStreet,SanFrancisco,CA94103-1741,USA
Wiley-VCHVerlagGmbH,Boschstr.12,D-69469Weinheim,Germany
JohnWiley&SonsAustraliaLtd,33ParkRoad,Milton,Queensland4064,Australia
JohnWiley&Sons(Asia)PteLtd,2ClementiLoop#02-01,JinXingDistripark,
Singapore129809
JohnWiley&SonsCanadaLtd,22WorcesterRoad,Etobicoke,Ontario,CanadaM9W1L1
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappears
inprintmaynotbeavailableinelectronicbooks.
LibraryofCongressCataloguing-in-PublicationData
Berger,Vance.
Selectionbiasandcovariateimbalancesinclinicaltrials/VanceW.Berger.
p. cm.
ISBN-13978-0-470-86362-6
ISBN-100-470-86362-5
1.Clinicaltrails–Statisticalmethods–Evaluation. 2.Rankingand
selection(Statistics)–Evaluation. 3.Sampling(Statistics)–Evaluation.
I.Title.
R853.C55B47 2005
610’.72(cid:2)4—dc22 2004026799
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN-13978-0-470-86362-6(HB)
ISBN-100-470-86362-5(HB)
Typesetin11/13ptPhotinabyTechBooks,NewDelhi,India
PrintedandboundinGreatBritainbyTJInternationalLtd,Padstow,Cornwall
Thisbookisprintedonacid-freepaperresponsiblymanufacturedfromsustainableforestry
inwhichatleasttwotreesareplantedforeachoneusedforpaperproduction.
iv
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Contents
Preface ix
PartI: IsThereaProblemwithReliabilityin
MedicalStudies? 1
1 AnEvolutionofComparativeMethodology 3
1.1 Single-subjectstudies 3
1.2 Caseseriesandcohortstudies 4
1.3 Historicalcontrols 5
1.4 Parallelcontrolgroups 6
1.5 Matchedstudies 6
1.6 Randomization 9
1.7 Advancerandomization 11
1.8 Allocationconcealment 12
1.9 Residualselectionbias 14
2 SusceptibilityofRandomizedTrialsto
SubversionandSelectionBias 17
2.1 Canrandomizedtrialsbesubverted? 18
2.2 Ifrandomizedtrialsaresubverted,dotheyceasetobe
randomizedtrials? 22
2.3 Whatismasking? 25
2.4 Whatisallocationconcealment? 27
2.5 Adoublestandard 28
2.6 Whatifallocationconcealmentcouldbeensured? 29
3 EvidenceofSelectionBiasinRandomizedTrials 37
3.1 Theburdenofproofregardingtheexistenceofselectionbiasin
randomizedtrials 38
3.2 Indirectpopulation-levelevidencethatselectionbiasexistsin
randomizedtrials 41
3.3 Directtrial-levelevidencethatselectionbiasexistsinrandomizedtrials 42
3.3.1 Heparinformyocardialinfarction 43
3.3.2 UniversityGroupDiabetesProgram 43
v
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
vi Contents
3.3.3 Talcandmustineforpleuraleffusions 44
3.3.4 Tonsillectomyforrecurrentthroatinfectioninchildren 45
3.3.5 Oxytocinandamniotomyforinductionoflabor 45
3.3.6 WesternWashingtonIntracoronaryStreptokinaseTrial 46
3.3.7 RSVimmuneglobulinininfantsandyoungchildrenwith
respiratorysyncytialvirus 47
3.3.8 Atrialtoassessepisiotomy 47
3.3.9 CanadianNationalBreastCancerScreeningStudy 48
3.3.10 Surgicaltrial 49
3.3.11 LifestyleHeartTrial 50
3.3.12 CoronaryArterySurgeryStudy 50
3.3.13 Etanerceptforchildrenwithjuvenilerheumatoidarthritis 51
3.3.14 EdinburghRandomizedTrialofBreast-CancerScreening 52
3.3.15 CaptoprilPreventionProject 53
3.3.16 Go¨teborg(Swedish)MammographyTrial 53
3.3.17 HIPMammographyTrial 53
3.3.18 HypertensionDetectionandFollow-UpProgram 63
3.3.19 RandomizedtrialtopreventverticaltransmissionofHIV-1 63
3.3.20 Effectivenesstrialofadiagnostictest 63
3.3.21 SouthAfricantrialofhigh-dosechemotherapyfor
metastaticbreastcancer 64
3.3.22 RandomizedstudyofaculturallysensitiveAIDS
educationprogram 65
3.3.23 RunawayYouthStudy 65
3.3.24 Clusterrandomizedtrialofpalliativecare 67
3.3.25 Randomizedtrialofmethadonewithorwithoutheroin 72
3.3.26 RandomizedNINDStrialoftissueplasminogenactivator
foracuteischemicstroke 74
3.3.27 NorwegianTimololTrial 75
3.3.28 Laparoscopicversusopenappendectomy 77
3.3.29 TheLosartanInterventionforEndpointReductionin
Hypertension(LIFE)Study 78
3.3.30 TheHeartOutcomesPreventionEvaluation(HOPE)Study 79
3.4 Insearchofbetterevidence 80
4 ImpactofSelectionBiasinRandomizedTrials 85
4.1 Quantifyingthepredictionoffutureallocations:balancedblocks 85
4.2 Quantifyingpredictionoffutureallocations:unbalancedblocks 90
4.3 Quantifyingcovariateimbalanceresultingfromselectionbias 94
4.4 Quantifyingthebiasresultingfromcovariateimbalance 100
PartII: ActionstobeTakentoImprovetheReliability
ofMedicalStudies 103
5 PreventingSelectionBiasinRandomizedTrials 105
5.1 MinimizingtheImpactofSelectionBias 106
5.2 BiasedSelectionofInvestigators 106
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Contents vii
5.3 Minimizingthepredictionoffutureallocations 107
5.3.1 Thetrade-offbetweenselectionbiasandchronologicalbias 109
5.3.2 Notation 111
5.3.3 Varyingtheblocksizes 113
5.3.4 Themaximalprocedure 115
5.3.5 Extensions 121
6 DetectingSelectionBiasinRandomizedTrials 123
6.1 BaselineImbalancesinObservedCovariates 124
6.2 Testingforselectionbiaswithoutbaselineanalyses 129
6.3 Theselectioncovariate 129
6.4 Theroleofthereversepropensityscoreinthird-orderresidual
selectionbias 130
6.5 UsingthereversepropensityscoreToTestforselectionbias:the
Berger–Exnertestandgraph 132
6.6 Usingthescreeninglogtotestforselectionbias 142
6.7 TheIvanova-Barrier-Berger(IBB)DetectionMethod 145
6.8 Interpretingnegativetestsofselectionbias 147
6.9 Whenshouldonetestforselectionbias? 148
6.10 Whoshouldtestforselectionbias? 155
7 AdjustingforSelectionBiasinRandomizedTrials 157
7.1 Methodsproposedforaddressingnon-randombaselineimbalances 159
7.2 Selectionbiasarisingfromacompletelackofallocationconcealment 161
7.3 Selectionbiasarisingfromimperfectallocationconcealment 161
7.3.1 TheRPSapproachtoadjustingforselectionbias 162
7.3.2 TheIvanova–Barrier–Berger(IBB)methodforcorrecting
selectionbias 165
8 ManagingSelectionBiasinRandomizedTrials 171
8.1 Actionpointsduringthedesignphaseofthetrial 171
8.2 Actionpointsduringtheconductofthetrial 174
8.3 Actionpointsduringtheanalysisoftrialdata 178
8.4 Actionpointsbyparty 182
References 187
AuthorIndex 199
SubjectIndex 203
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Preface
Itstandstoreasonthatimportantdecisions,suchasthoseinvolving
thetreatmentofapatientoragroupofpatients,wouldbebasedon
solidevidence,especiallywhenoneconsiderswhatthealternativeis.
Hence,evidence-basedmedicinehasbecomeamajorthemeofmedical
researchandmedicaldecision-making.Ahierarchyofevidencehas
evolved to guide users of medical research, and randomized trials
tend to gravitate towards the top of this hierarchy. In fact, it is not
uncommontohearthephrase‘randomizedevidence’usedtodenote
thestrongestofallpossibletypesofevidence.
Itisfarlesscommontofinddiscussionsofthewaysinwhichvari-
ation among randomized trials themselves might make some better
thanotherswithinthehierarchyofevidence.Attheheartoftheun-
spokenassumptionthatallrandomizedtrialsarecreatedequalisthe
beliefthatallrandomizedtrialsarefaircomparisons.Thatis,theva-
lidity of the comparison is guaranteed by virtue of the fact that the
word ‘randomized’ appears in the title. The only question, then, is
howwelltheseinternallyvalidrandomizedresultsgeneralizetothe
target population. That is, to what extent is there external validity?
Theselectionbiasmostcommonlyassociatedwithrandomizedtrials
istheonethatmightlimittheexternalvalidity,becausetheresultsof
randomizedtrialswouldtendtoapplytoonlythosepatientswillingto
berandomized.Thisisalegitimateconcern,butitisnottheconcern,
orthetypeofselectionbias,consideredinthisbook.
Before one can even ask if the results of a randomized trial gen-
eralize, one needs to ask if these results are internally valid. Upon
careful scrutiny it is fairly easy to deduce that internal validity, or a
faircomparison,isinfactnotguaranteedbyevenproperrandomiza-
tion(which,bytheway,isalsonotguaranteedmerelybyvirtueofthe
claimthatthetrialwasrandomized).Thecentralthemeofthebookis
ix
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
x Preface
theideathateveninaproperlyrandomizedtrialconfoundingcanbe
inducedtocreateacovariateimbalancethatleadstoatypeofselection
biasthatcancompromiseinternalvalidity.Thatis,estimatesoftreat-
ment effects can be biased, confidence intervals can be too narrow,
andp-valuescanbeartificiallytoolowasaresultofthisselectionbias.
Moreover,nothinginthesesummarystatisticsthemselveswillallow
areadertodiscernthatsuchselectionbiashasoccurred,sooneneeds
tolookelsewheretoconfirmthefacevalidityofthefindings.
Thisbookdealswiththeaspectsofselectionbiasthatonewouldneed
toconsiderwhendesigningarandomizedtrialoranalyzingthedata
fromarandomizedtrialorreviewingtheresultsofarandomizedtrial.
PartI(Chapters1–4)servesasastatementoftheproblem.Specifically,
Chapter 1 presents the various types of study designs and clarifies
why randomized trials are at the top of the hierarchy of evidence.
Chapter 2 presents the mechanisms by which upcoming treatment
allocationscanbepredictedeveninrandomizedtrials,andhowthis
foreknowledgeoffutureallocationscanbeexploitedtocreatethetype
of selection bias we consider. Chapter 3 presents the evidence that
thistypeofselectionbiashasactuallyoccurred,andisnotsimplya
hypothetical concern. Chapter 4 presents the impact this selection
biascanhaveontheresultsofrandomizedtrials.Itissuggestedthat
Chapter 2 be read by all readers, because it sets the stage for the
methodsproposedinsubsequentchaptersformanagingselectionbias.
Thosereaderswhoarestudyingselectionbiasaspartoftheirtraining
inclinicaltrialdesignandanalysismightwishtoalsoreadChapters
1,3,and4,butthesechaptersareoptionalonafirstreadingforthose
readerswhohaveactualtrials(anddeadlines!)inmindwhenreading
thisbook,andwishtoapplytheresultsasquicklyaspossible.These
readersshouldskipaheadtoPartIIrightafterreadingChapter2.
PartIItellsthereaderthestepsthatcanbetakentomanageselection
biasinrandomizedtrials.Chapter5dealswithpreventionbyvarious
means,butmostlybyselectingarandomizationtechniquethatdefies
prediction of future allocations even when the past allocations are
known.Ofcourse,onecouldeliminateallprediction,butatthecost
ofenablingchronologicalbias,andsothetrade-offbetweenthesetwo
must be considered. A reader who is writing a protocol should read
Chapter 5 right after Chapter 2. Chapter 6 deals with detecting se-
lectionbiasbyvariousmethods,butmostlybytheBerger–Exnertest.
JWBK019-FM JWBK019-Berger March30,2005 13:45 CharCount=0
Preface xi
Readerswhoreviewrandomizedtrials,suchasregulatoryagencies,
journaleditors,andfundingagencies,mightwishtoreadChapter6
rightafterChapter2.Chapter7dealswithmethodsrecentlyproposed
for correcting for selection bias if it is found in a randomized trial.
Thiswouldbeimportantreadingforanyreaderwhohasfoundeven
theappearanceofselectionbiasinarandomizedtrial.Finally,Chap-
ter8offersrecommendations,anddrawsfrompreviouschapters.A
moduleonselectionbiaswithintheframeworkofatrainingprogram
in clinical trial design and analysis might use just Chapters 2 and
8,withtheotherchaptersservingasreferencematerial.Chapter8is
alsoanimportantchapterforresearchersintrialdesignmethodology,
becauseitpresentssomeopenproblemsthesolutionstowhichmight
helptoimprovefuturetrialmethodology.
Thisisnotamathematicallychallengingtext.Thereaderwillnot
encounterendlessequationsandderivations,andsotherereallyisno
mathematicalprerequisite.Infact,thisbookisintendedforabroad
readershiprepresentingthefullspectrumofdisciplinescontributing
to the design, conduct, analysis, reporting, and review of random-
izedclinicaltrials.Perhapsthebestprerequisiteswouldbeintellectual
curiosityandperhapsactualstudiestoworkon.
I wish to thank Stephen Senn for discussing these ideas with me,
andforsuggestingthatIputtheseideasintobookform.Inaddition,it
ishisownwritingsonbaselineimbalancesinrandomizedtrialsthat
began my thinking about these ideas. I also thank the researchers
whojoinedmeinstudyingselectionbiasinrandomizedtrials,includ-
ingDerekExner,JeffreyBears,CostasChristophi,AnastasiaIvanova,
MariaDeloria-Knoll,andSherriWeinstein.IthankKenSchulzforbe-
ingpartofasuccessfulworkshoponthissubjectatthe2002Societyof
ClinicalTrialsMeetinginArlington,VA,alongwithCostasChristophi,
MariaDeloria-Knoll,andmyself.IalsothankDougAltman,DonCorle,
SimonDay,StevenHirschfeld,DamianMcEntegart,ThomasPermutt,
andDrummondRennieforusefuldiscussionsofselectionbias.Ithank
myformersuperiorsStephenWilson,GeorgeChi,SatyaDubey,Chuck
Anello,BobO’Neill,TonyLachenbruch,andSusanEllenberg,atthe
Food and Drug Administration, and more recently Phil Prorok and
Peter Greenwald at the National Cancer Institute, for encouraging
metocontinuemyresearch.IthankJeffreyMannforinformingme
of the randomization irregularities in the NINDS Stroke Trial, and