BAYESIAN META-ANALYSIS ON MEDICAL DEVICES: APPLICATION TO IMPLANTABLE CARDIOVERTER DEFIBRILLATORS.

2012·0.29 MB·English

by  Youn, Ji-Hee

InternationalJournalofTechnologyAssessmentinHealthCare,28:2(2012),115–124.
(cid:1)c CambridgeUniversityPress2012
doi:10.1017/S0266462312000037
Bayesian Meta-Analysis on Medical
Devices: Application to Implantable
Cardioverter Defibrillators
Ji-HeeYoun KarlaHemmingandAlanGirling
BrunelUniversity UniversityofBirmingham
email:cmp11jy@sheffield.ac.uk
MartinBuxton
JoanneLord BrunelUniversity
BrunelUniversity
Objectives:Theaimofthisstudyistodescribeandillustrateamethodtoobtainearlyestimatesoftheeffectivenessofanewversionofamedicaldevice.
Methods:Intheabsenceofempiricaldata,expertopinionmaybeelicitedontheexpecteddifferencebetweentheconventionalandmodifieddevices.BayesianMixedTreatmentComparison(MTC)
meta-analysiscanthenbeusedtocombinethisexpertopinionwithexistingtrialdataonearlierversionsofthedevice.Weillustratethisapproachforanewfour-poleimplantablecardioverter
defibrillator(ICD)comparedwithconventionalICDs,ClassIIIanti-arrhythmicdrugs,andconventionaldrugtherapyforthepreventionofsuddencardiacdeathinhighriskpatients.ExistingRCTswere
identifiedfromapublishedsystematicreview,andweelicitedopiniononthedifferencebetweenfour-poleandconventionalICDsfromexpertsrecruitedatacardiologyconference.
Results:Twelverandomizedcontrolledtrialswereidentified.SevenexpertsprovidedvalidprobabilitydistributionsforthenewICDscomparedwithcurrentdevices.TheMTCmodelresultedin
estimatedrelativerisksofmortalityof0.74(0.60–0.89)(predictiverelativerisk[RR]=0.77[0.41–1.26])and0.83(0.70–0.97)(predictiveRR=0.84[0.55–1.22])withthenewICD
therapycomparedtoClassIIIanti-arrhythmicdrugtherapyandconventionaldrugtherapy,respectively.TheseresultsshowednegligibledifferencesfromthepreliminaryresultsfortheexistingICDs.
Conclusions:Theproposedmethodincorporatingexpertopiniontoadjustforamodificationmadetoanexistingdevicemayplayausefulroleinassistingdecisionmakerstomakeearlyinformed
judgmentsontheeffectivenessoffrequentlymodifiedhealthcaretechnologies.
Keywords:Bayesiananalysis,Meta-analysis,Medicaldevices,Expertopinions,Defibrillator,Implantable
Medical devices differ from pharmaceuticals in terms of the asthearticularsurfacereplacementfromDePuy,whichhasnow
nature and quantity of evidence about their effectiveness and beenrecalledduetotheunusuallyhighrateofimplantfailure.
cost-effectiveness. The medical device industry is a dynamic The U.S. Food and Drug Administration has recently ap-
sectorwherenewtechnologiesarecontinuouslydeveloped,and provedtheuseofanimplantablecardioverterdefibrillator(ICD)
the life cycle of a particular version of a device is often less with four-pole (IS-4) lead connectors without premarket clini-
than 2 years. Frequent technological improvements are made caltesting,andinsteadrequiredapost-approvalstudy.Although
without changing major features or functionality. With such thenewIS-4deviceissaidtoconferbenefitssuchasasmaller,
rapidincrementalchange,itcanbedifficulttokeeptheevidence lower-profile pulse generator, there have been some concerns
up-to-date. that such a modification should be clinically tested before ap-
The regulatory environment for devices also differs from proval(10).
that for pharmaceuticals. Evidence required for marketing ap- This leaves potential early adopters—clinicians, patients,
proval may only describe technical performance rather than and payers—with a difficult decision. Unless a new version of
clinical safety or efficacy, let alone effectiveness or cost- a medical device is a genuine breakthrough, its effects will be
effectiveness. In the United States, new products can obtain largely predictable from information on previous generations.
accesstothemarketwithoutbeingtestedonpatientsiftheyare Butthisdoesrequireajudgmentonwhetherdemonstratedtech-
considered “substantially equivalent” to previously approved nicalimprovementsarelikelytotranslateintoclinicalgains,and
devices.Amongnumerousexamplesistheartificialhipknown ifso,whetherthesearesufficienttojustifythe(usuallyhigher)
cost. Furthermore, potential users have to judge the likelihood
ofunforeseenadverseeffects.
Theaimofthisstudyistodevelopamethodtohelpdecision
ThisstudywasproducedundertheMATCHProgramme(EPSRCGrantGR/S29874/01),
makers make an informed, early assessment of a new version
althoughtheviewsexpressedareentirelythoseoftheauthors.
ofamedicaldevice.WesuggestthatBayesianbias-adjustment
TheonlineversionofthisarticleispublishedwithinanOpenAccessenvironmentsubjecttothe
techniquesmaybeadaptedtocombineexistinginformationon
conditionsoftheCreativeCommonsAttribution-NonCommercial-ShareAlikelicence
<http://creativecommons.org/licenses/by-nc-sa/2.5/>.Thewrittenpermissionof earlier generations of the device with expert judgments about
CambridgeUniversityPressmustbeobtainedforcommercialre-use. the extent to which the new version is likely to differ. In the
115
Younetal.
following section, we explain the method in more detail. We idea). This relies on the assumption that the new device is
thenillustratethemethodusingtheexampleoffour-poleICDs. sufficiently similar to the existing one, and that its effect is
The advantages and disadvantages of this approach are then predictable from data on the earlier version. And, of course,
discussed. thesameassumptionsabouttheconsistencybetweendirectand
indirect evidence for the existing device must still hold, as for
anyMTCanalysis.
METHODSFOREARLYEVALUATIONOFAMODIFIEDTECHNOLOGY The probability distribution for the difference parameter,
The Bayesian approach resembles the human cognitive proce- denotedasβ, canbe estimatedfrom expertopinion.There are
dureofacquiringknowledgebyupdatingapriorunderstanding manywaysinwhichsuchdistributionscanbeelicitedtoreflect
as new information becomes available. Usually, the prior view individuals’degreesofbeliefandextentofuncertaintyoverthe
is supported by scant or poor-quality data, or it may even be location of an unknown parameter. In our example below, we
basedonsubjectiveopinionalone.Asmoreorbetterempirical elicitedthisinformationintheformofhistograms,withexperts
evidence emerges, priors are updated to derive posterior esti- askedtostatetheprobabilityoftheparameterlyingwithineach
mates.Insuchcases,thereisusuallylessuncertaintyaroundthe ofaseriesoffiniteintervals.
posterior,thanaroundthepriorstateofknowledge.Wepropose Therearealsomanywaysinwhichindividuals’subjective
to adapt this method for the situation where there is a strong probability estimates can be pooled. In our example, we com-
body of evidence on an existing technology, and only weak binedhistogramsfromseveralexpertsbydividingtheintervals
evidence or beliefs about how a new modified version of the on which opinion was elicited to obtain the smallest subinter-
technology differs. Here, posterior estimates of the effective- vals, and taking the arithmetic means of the probabilities for
ness of the modified technology will be less certain than the each of the subintervals across the experts. To illustrate, sup-
priorknowledgeaboutearliergenerationsofthedevice. pose that Expert 1 said there was a 50 percent probability that
Bayesianmethodsofmeta-analysispoolevidencefromclin- theparameterlayineachoftwointervals:(0,1)and(1,2).Expert
ical trials to estimate the effectiveness of health technologies. 2saidtherewasa50percentprobabilitythatitlayineachofthe
The meta-analysis requires assumptions about the heterogene- intervals (0,1) and (1,3). Then, one might say that the elicited
ityoftreatmenteffects:afixedeffectmodelassumesacommon probabilitiesforthreesubintervals,(0,1),(1,2)and(2,3),are50
true effect underlying all study results, whereas a random ef- percent,37.5percent,and12.5percent,respectively.
fects model assumes that trials draw a random sample from Appropriate scale conversion (Appendix 1) provides a
the distribution of true effects. The standard pair-wise meta- pooled discrete probability distribution for β, which can be
analysis can be extended to a Mixed Treatment Comparison usedtodirectlysamplevaluesforthisparameterintheBayesian
(MTC) meta-analysis when exploring the relative efficacy of a MCMCmodel.Alternatively,onemightfitasuitableparamet-
range of interventions (2). This integrates information from a ric distribution to reflect the pooled opinion. In our example
networkofdirectandindirectcomparativetrialswithinasingle reported below, we found that the elicited distribution was ap-
model(12).Thisdoes,ofcourse,requireassumptionsaboutthe proximately normal, so we calculated the mean of β from the
consistency of the underlying evidence. Bayesian computation discrete distribution, and used a variation on a univariate least
ispossiblewithMonteCarloMarkovchain(MCMC)methods squaresfittingmethod(19)tofinditsstandarddeviation.Sam-
usingWinBUGS,afreelyavailablestatisticalsoftwarepackage ples from the fitted Normal distribution for β were then incor-
(16). poratedintothemainMTCmodel.
TheseBayesianmeta-analysistechniquesmaybeextended Inothercircumstances,otherparametricdistributionsmight
to adjust for suspected “biases” in the evidence base. For ex- provide a better fit for the elicited data: for a more complete
ample, Welton and colleagues (25) included additive bias pa- discussionofdistributionselection,seeO’Haganetal.(19).
rameterstoadjusttheresultsofstudiesthatlackedappropriate
allocationconcealment,estimatingtheextentof thisbiasfrom
a large meta-epidemiological study. And Turner et al. (23) in-
ILLUSTRATIVEEXAMPLE:ICDS
corporatedawiderangeofpossiblebiases,fromselectionbias
to health outcome bias, using expert judgment to identify and TheReviewQuestion
quantifythem. OurexampleconcernstheuseofICDsinthepreventionofsud-
We borrow this idea, introducing an adjustment parameter dencardiacdeathinhighriskpatientswithalowleftventricular
to reflect beliefs about the difference in effectiveness between ejectionfraction(LVEF<35–40percent).Thesepatientsoften
anewandanexistingversionofadevice.Thiscanbeaddedto haveahistoryofmyocardialinfarctionand/orcongestiveheart
anestimateoftreatmenteffectfromarandom-effectsBayesian failure. We included interventions targeted at individuals who
MTC meta-analysis of trial evidence for the existing device, have not yet experienced a major arrhythmic event (primary
to obtain an estimate of effectiveness for the new device (see prevention), and those targeted at people who have survived
Appendix1foramoreformalmathematicaldescriptionofthis suchanevent(secondaryprevention)(9).
INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012 116
Bayesianmeta-analysisonmedicaldevices
Mixed Treatment Comparison Meta analysis
Trial data (Likelihood)
Diffuse
A 1 studies * B
priors No an(cid:1)- (n=1692) An(cid:1)- Posterior
arrhythmic arrhythmic distribu(cid:1)on
(on rela(cid:2)ve therapy therapy for
effec(cid:2)vene
ss between 7 studies* 6 studies* Treatment D
A,B, and C) (n=6256) C (n=3839)
Conven(cid:1)on
al ICD
0studies
Elicited informa(cid:1)on
D
on the difference Four-pole
between C & D ICD
* One three-arm trial (SCD-HeFT) was included.
Figure1. Illustrationofanalyticalapproach.
WedistinguishfourbroadtypesofTreatment:(A)Conven- (treatmentC)comparedwithnon-ICDtherapies(treatmentsA
tional drug therapy (which may include beta-blockers or ACE andB).Ahierarchicalrandomeffectsmodelwasused,assum-
inhibitors); (B) Conventional drug therapy and Class III Anti- ing study-leveltreatment effects to be a random sample drawn
arrhythmic drug therapy (such as amiodarone or sotalol); (C) from a probability distribution for the overall treatment effect
Conventional drug therapy and conventional ICD therapy; and governedbyasetofcommonparameters.Themodelassumesa
(D) Conventional drug therapy and four-pole connector ICD lognormaldistributionfortherelativerisks,andappliesdiffuse
therapy. priors(thisisacommonMTCmodel)(12).Conventionaldrug
“Conventional” drug therapy was defined to include any therapy (treatment A) was taken as the reference treatment for
drugs other than class III anti-arrhythmics, provided they are themeta-analysis.
alsoprescribedforpatientsintheotherarmsofthetrial.Inter- ThedistributionfortherelativeeffectivenessofthenewICD
ventions might also include other surgical procedures such as (treatment D), for which there is not yet any RCT evidence,
coronaryarterybypassgraft,providedthesewerealsocommon is assumed to be the same as that for C, except its mean is
“underlying”treatmentsacrossallarmswithinthesameRCT. shifted by an adjustment parameter β, which is also assumed
Allfourinterventionscanbeusedinprimaryorsecondary to be normally distributed. The mean and variance of β was
prevention, although conventional drug therapy alone is rela- estimatedfromelicitedexpertopinion.
tivelyunusualinsecondaryprevention. The meta-analysis model and adjustment for the ICD ex-
ampleisillustratedinFigure1,andtheBUGScodeisprovided
ReviewofEvidenceonCurrentICDs
inAppendix2.
Clinical trial data on existing ICDs was obtained from a
recently-published systematic review by Ezekowitz and col-
ElicitationMethod
leagues (9). They report details about the search strategy, the
Preliminary discussions with two practicing cardiologists led
characteristicsoftheincludedstudiesandtheextentofhetero-
us to focus on the practicality of our elicitation methods, so
geneity. Although this review included both randomized and
we adopted a rather simpler approach than that advocated in
observational studies, we focused only on RCT data for illus-
the literature (15). Our paper-based questionnaire was refined
trativepurposes.
after these discussions, to focus and standardize the elicitation
Twoauthors(J.Y.andK.H.)extracteddatafromtheoriginal
process(seeAppendix3).
papers cited in the Ezekowitz review, and also checked other
Data were collected through face-to-face interviews with
papersrelatedtoeachtrialifinformationgiveninthecitedones
individuals recruited at a cardiology conference (after a ses-
wasinsufficient.
sion devoted to discussion on advanced device treatments for
Meta-analysisModel arrhythmias at the Heart Rhythm Congress 2009). This pro-
We used a Bayesian MTC meta-analysis to estimate the rela- vided an opportunity to clarify the questions if required. Par-
tive risk of death for patients under the existing ICD therapy ticipants were first asked about their clinical expertise and
117 INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012
Younetal.
Treatment A vs. Treatment C ICD Others RR (mean, 95% CrI)
(r/n) (r/n)
SCD-HeFT(5) 182/829 244/847 0.79 (0.67-0.91)
DINAMIT (12) 62/332 58/342 0.93 (0.74-1.24)
DEFINITE (14) 28/229 40/229 0.79 (0.58-1.00)
MADIT-II (19) 105/742 97/490 0.77 (0.62-0.92)
CABG-PATCH (6) 101/446 95/454 0.95 (0.77-1.22)
COMPANION (7) 105/595 131/617 0.83 (0.68-1.00)
CAT (4) 13/50 17/54 0.82 (0.59-1.12)
Pooled effect (A vs. C) 0.83 (0.70-0.97)
Predic(cid:1)ve effect (A vs. C) 0.84 (0.55-1.25)
Vs. Treatment D
Predic(cid:1)ve effect (A vs. D) 0.84 (0.55-1.22)
Treatment B vs. Treatment C
SCD-HeFT(5) 182/829 240/845 0.77 (0.66-0.89)
AVID (22) 80/507 122/509 0.70 (0.56-0.86)
CIDS (8) 83/328 98/331 0.81 (0.66-1.00)
MADIT-I (18) 15/95 39/101 0.64 (0.41-0.85)
AMIOVIRT (21) 6/51 7/52 0.77 (0.51-1.09)
CASH (15) 36/99 40/92 0.78 (0.61-1.01)
Pooled effect (B vs. C) 0.75 (0.59-0.90)
Predic(cid:1)ve effect
0.77 (0.42-1.29)
(B vs. C)
Vs. Treatment D
Predic(cid:1)ve effect 0.77 (0.41-1.26)
(B vs. D)
0.0 0.5 1.0 1.5
Figure2. MixedtreatmentcomparisonoftheconventionalICD(TreatmentC)andothertherapies(TreatmentsA&B),andthepredictednewICDeffect.TreatmentA:Conventionaldrugtherapy;TreatmentB:Anti-arrhythmic
drugtherapy;TreatmentC:ConventionalICDtherapy;TreatmentD:Four-poleconnectorICDtherapy.Dashedlines:thepredictiveposteriorsestimatedfromtheMTCmodelwithandwithoutadjustmentforβ.
familiaritywithICDimplantation.Theywerethengivensome RESULTS
simple summary information about the 5-year mortality rate
with “conventional” ICDs, based on the results of two, large LiteratureReview
trials. The experts were then asked to estimate the most likely The systematic review by Ezekowitz and colleagues (9) iden-
value,andlowerandupperlimits,fortherelativemortalityrate tified twelve RCTs that reported all-cause deaths during the
with the four-pole IS-4 ICD. Finally, they were asked to dis- follow-upperiodandthenumberofstudyparticipantsforeach
tribute 100 points along a line showing different numbers of treatmentarm(3–7;11;13;14;17;18;20;21).
lives gained or lost with the four-pole device in comparison Ezekowitzetal.madepair-wisecomparisonsbetweenICD
with a “conventional” ICD to reflect their subjective beliefs. and non-ICD treatments. However, for our MTC analysis,
The comparison against “conventional” ICD was intended to we further classified the non-ICD treatments as conventional
simplifyelicitationandanalysis,althoughitdoesneglectpossi- drug therapy alone (A) and conventional drug therapy com-
bledifferencesbetweencurrentICDdevices.Weusedstatistical bined with a Class III Anti-arrhythmic drug therapy (B). One
tests of heterogeneity on the trial data to look for evidence of study, the CASH trial (14), reported data on three random-
suchdifferences. ized groups: (i) metoprolol (a beta blocker), (ii) amiodarone
Asthemodelestimatesthelogarithmofrelativemortality, (a Class III anti-arrhythmic), and (iii) conventional ICD. We
elicited absolute mortality values were converted to an appro- excluded the metoprolol arm, because this did not fit our defi-
priatescale(seeAppendix1).MicrosoftExcelandRsoftware nition of “conventional drug therapy” (treatment A), because
toolswere thenusedtocalculatea combineddiscretedistribu- it was not also co-prescribed in the other treatment arms.
tionforthedifference-adjustmentparameter,andtotestandfit The final list of data included in our analysis is shown in
anormaldistribution,asdescribedabove. Figure2.
INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012 118
Bayesianmeta-analysisonmedicaldevices
Table1. ThePosteriorMean(95%CredibleInterval)ofSelectedRelativeEffectivenessMeasuresRegardingMortality(τ2:Between-StudyVariance,
i.e.,HeterogeneityParameter)
ICDsvs.TreatmentB ICDsvs.TreatmentA
a)ExistingICDs(TreatmentC) Primaryandsecondarypreventiontrialscombined(12trials)(n=9266)
LogRR −0.30 LogRR −0.20
(−0.52,−0.11) (−0.36,−0.03)
RR 0.75 RR 0.83
(0.59,0.90) (0.70,0.97)
PredictiveRR 0.77 PredictiveRR 0.84
(0.42,1.29) (0.55,1.25)
τ2=0.031, Posteriormeanofresidualdeviance=26.08
b)Four-poleconnectorICDs(TreatmentD) Primaryandsecondarypreventiontrialscombined(12trials)
withβ adjustment(n=9266)
LogRR −0.30 LogRR −0.19
(−0.52,−0.12) (−0.36,−0.03)
RR 0.74 RR 0.83
(0.60,0.89) (0.70,0.97)
PredictiveRR 0.77 PredictiveRR 0.84
(0.41,1.26) (0.55,1.22)
τ2=0.031, Posteriormeanofresidualdeviance=26.55
Note.Treatment A: Conventional drug therapy; Treatment B: Anti-arrhythmic drug therapy; Treatment C: Conventional ICD therapy; Treatment D:
Four-poleconnectorICDtherapy.
PrimaryversusSecondaryPrevention (τ2 =0.69)thanbetweenthesecondarypreventionstudies(τ2
Ezekowitz et al. (9) reported similar estimates for the relative = 0.27), pooling all these trials resulted in a much lower level
risk of mortality in the primary prevention trials as in the sec- of heterogeneity (τ2 = 0.03). This led us to conclude that the
ondary prevention trials. They noted that the extent of hetero- use of ICD technology results in similar mortality benefits in
geneityacrossallprimaryandsecondarypreventiontrialswas primaryandsecondaryprevention,and,therefore,wecombined
moderate(I2 =44.4percent),althoughtheprimaryprevention theseresults.
trialsshowedanintrinsicallywidervariance.
WeinitiallyconductedseparateBayesianmeta-analysesfor Meta-analysisResultsforExistingICDs
primary and secondary prevention trials to test whether they OurmainMTCmodelincluded12trials:nineprimarypreven-
aresufficientlyhomogeneoustobecombined.Forthisanalysis, tiontrials(3–6;11;13;17;18;20)andthreesecondaryprevention
we excluded trials for which the control arm was Treatment A trials(7;14;21).
becausenosecondarypreventiontrialsincludedthistreatment. The forest plot of treatment effects for all comparisons
TheresultsofthisC-Bcomparisonshowedlittledifferencebe- involvingtheexistingICDtherapy(TreatmentC)andthepooled
tweentheestimatedeffectsforprimaryprevention(relativerisk estimates of mortality benefits from the use of existing ICDs
[RR]0.72(0.22–1.81),basedon3trials(4;17;20)N=1973)and are shown in Figure 2 and Table 1, respectively. Note that, in
thoseforsecondaryprevention(RR0.81(0.41–1.46),basedon Figure 2, the results for some studies (particularly DINAMIT,
3trials(7;14;21),N=1866).(Notethatthedifferencesbetween CABG-PATCH, and MADIT-I) appear rather different to what
theseresultsandthosereportedin(9)areduetodifferencesin one might expect from the raw data, because of a “shrinkage
the meta-analysis models (24), and in the data included – we effect” (1). The relative risk estimate for the C-B comparison
excluded three primary prevention trials with a Treatment A (mean=0.75[95percentCrI0.59–0.90])islowerthanthatfor
control arm, and the metoprolol arm from the CASH study, as theC-Acomparison(mean=0.83[95percentCrI0.70–0.97]),
mentionedabove.) indicating a higher mortality risk associated with the Class III
Although this analysis suggested a relatively higher level anti-arrhythmicdrugtherapy(B)thanthatforconventionaldrug
of heterogeneity between the primary prevention studies therapy alone (A) (mean = 1.12 [95 percent CrI 0.89–1.42]).
119 INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012
Younetal.
(a)Combinedsubjec(cid:2)vedistribu(cid:2)on (b)NormalCDFvs.subjec(cid:2)veCDF
1.0
0.8
0.6
y
0.4
0.2
0.0
-0.03 -0.02 -0.01 0.00 0.01 0.02 0.03
β
Dots:cumula(cid:2)veprobabili(cid:2)esformid-intervalvaluesfrom
thecombinedhistogram
Line:Fi(cid:4)edNormaldistribu(cid:2)onCDFvalues
β=log(mortalityonoldICD/mortalityonnewICD);Mortality
onnewICD=(Livessavedorlost/1000)+mortalityonoldICD
Figure3. Combinedexperts’subjectivedistributionsforthedifferenceinmortalitywiththenewfour-poleICDcomparedwithconventionalICDandthefittedNormaldistribution.
(Similar findings were also reported in Tung et al. (22).) The AdjustedMeta-analysis:PredictionforNewICD
predictive posteriors for a future trial on conventional ICDs The fitted Normal distribution for β was incorporated into the
showwiderintervals:0.77(0.42–1.29)fortheC-Bcomparison meta-analysistoestimatetheeffectivenessofthefour-polecon-
and 0.84 (0.55–1.25) for the C-A comparison. Between-study nectorICDs.TheadjustedMTCresultsareprovidedinTable1.
variabilitywasmoderate(τ2 =0.031). TheestimatesinTable1showextremelysmalldifferences
The model predictions were considered to offer adequate between the estimated mortality effects of the old and new
goodness of fit when the posterior mean residual deviance of ICDs: RR of 0.75 (0.59–0.90) for C versus B and 0.74 (0.60–
26.08 was compared to the number of data points, 25. For all 0.89) for D versus B. This was expected due to the near-zero
meta-analyses,standardchecksformodelconvergenceledtothe meanandstandarddeviationestimatesofβ.Also,thepredictive
use of 10,000 iterations as a burn-in, and the results presented distributionsforfuturetrialsofthenewICDwereverysimilarto
wereobtainedbasedon20,000iterations. thoseforexistingICDs(Figure2).Thebetween-trialvariance,
τ2, was estimated as having a mean of 0.031(on the log RR
ElicitationofExpertOpinion scale).TheMonteCarlostandarderrorsofthemeanestimates,
Seven experts provided valid probability distributions for the
wereassmallas0.003fortheD-Bcomparison.
mortalitydifferencewiththefour-poleICD.Wecannotcalculate
a response rate, as the invitation to participate was made in an
openconferencesessionandtwooftheauthors(K.H.andJ.Y.) DISCUSSION
approachedindividualsafterthesession.However,mostofthose We have presented a Bayesian, random-effects, MTC meta-
approached did agree to offer their opinion and complete the analysisincorporatingpertinentexternalinformation.Thiscan
elicitationform. be used for the evaluation of new versions of medical devices,
The mean and standard deviation of the fitted distribution forwhichtrialdataisnotyetavailable.Itadaptsanestablished
for β were both close to zero, i.e., Normal (µβ = 0.0017, technique of plugging a functional relationship between a tar-
σβ = 0.0060), reflecting expert opinion that the mortality im- get parameter and an adjustment parameter into an evidence
pactofthenewICDswouldbesimilartothatofcurrentdevices. synthesis model (1). Based on the current literature on bias
However, the positive mean value signifies the belief that the modeling(23;25),anintuitivelysimpleadditiveparameterwas
newdevicemightslightlyreducemortality. used to capture expected departures of the effectiveness of the
The pooled cumulative subjective probabilities from the newdevicefromtheoldone.
expertswerecomparedagainstthecumulativedensityfunction Weillustratedthisapproach,obtainingmortalityestimates
oftheNormaldistribution(Figure3).Althoughaweakdegreeof foranewICDdevicebasedonamixtureofexpertopinionand
skewness is observed in the empirical probability distribution, empiricalevidenceonexistingICDs.Thisexamplewaschosen
the assumptionthat β follows a normal distributionappears to asthereisrelativelylargeexistingevidencebaseforICDs,while
bereasonable. further data collection on a new variant may be expensive and
INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012 120
Bayesianmeta-analysisonmedicaldevices
difficultduetotheinvasivenatureofthetechnology.Inthisex- addresssuchheterogeneityproblems(8). As inanysystematic
ample,theincorporationofexpertopiniononthenewtechnol- review and meta-analysis, it is also important to consider the
ogymadeverylittledifference.However,theproposedmethod potentialforotherformsofbias;suchaspublicationbias.
could potentially change a technology adoption decision, de- Wealsonotethat,whereasBayesianincorporationofexpert
pendingonhowpositiveandstrongtheexperts’beliefsare. opinion can help consolidate existing opinion and data, these
Whetherthesuggestedmethodshouldhelptoinformman- methods cannot substitute for long term evaluation of rare ad-
ufacturers’ decisions about development, or payers’ decisions verse events. Post-marketing follow-up studies are essential in
about reimbursement, depends on the intrinsic reliability of ensuringthatnewvariantsofadevicedonotimposeadditional
expert opinion—are they right in their expectations about the harms.Expertopinionsonrareadverseeventshavelimiteduse.
worth of a new version of a device more often than they are Because of the rarity of these events, not all experts will have
wrong? It also depends on the robustness of methods used to encounteredthem.Furthermore,theissueofwhenadifference
elicitexpertopinionandtocombineitwithpriorevidence. inanewvariantofadeviceissufficientlylargetowarrantpri-
Even well-designed elicitation processes can be subject to mary randomized controlled trials is fundamental. In relation
cognitiveorbehavioralbias,andtherearefurtheruncertainties tothis,theproposedmethodhasanotherpotentialuse:iffuture
associated with the selection of experts and methods of syn- research generates evidence on the new variants, the estimates
thesizingeliciteddata.Thereisnotcurrentlyaprescriptionfor obtained by this method can also serve as prior information in
how best to elicit and pool expert opinion. In our example, we thesequentialuseofBayesianevidencesynthesis.
recruitedexpertsfromarelevantconference.Theseindividuals
might not reflect the spectrum of beliefs of cardiac specialists,
CONCLUSIONS
which could explain the relatively small within- and between-
expertvariation.Theuseofindividualinterviews,ratherthanan Despite these caveats, the suggested method may play a use-
actualornominalgroupapproachmightalsohaveinfluencedthe ful role in assisting decision makers to make early informed
results;withfeedbackand/orachancefordiscussion,opinions judgments about evolving healthcare technologies. Before any
might have evolved (19). The specific design and wording of formalclinicalevidenceisgathered,quantifiedopinioncanpro-
ourquestionnairemightalsohavehadanimpact.Forexample, vide some indication of expected benefits from early adoption
we informed participants that there is a fixed 5-year mortality andfurtherevidencecollection.Thismethodprovidesameans
rate of 35 percent with existing version of ICDs, which may for formally including experts’ opinion into an evidence syn-
underestimateuncertaintyoverβ. thesismodel,whichcanbeupdatedasmoreevidenceemerges.
The chosen effectiveness parameter in our analysis was
all-cause mortality. However, the dominant opinion of the in-
CONTACTINFORMATION
terviewed experts was that the change in the lead connector is
Ji-Hee Youn, MSc, Research Fellow, Joanne Lord, PhD,
morelikelytobeassociatedwithothercomplicationrates,asthe
Reader,HealthEconomicsResearchGroup,BrunelUniversity,
modified lead may be related to higher rates of lead dislodge-
Uxbridge,UnitedKingdom
mentandinsulationdefects.Ifothereffectivenessmeasureshad
KarlaHemming,PhD,SeniorLecturer,AlanGirling,MA,Se-
been chosen, it is possible the experts might have anticipated
nior Research Fellow, Department of Public Health, Epidemi-
a “substantial departure from current ICD connector systems”
ologyandBiostatistics,UniversityofBirmingham,Edgbaston,
(10).Thissuggeststhatpreliminary,qualitativediscussionwith
Birmingham,UnitedKingdom
expertsisveryimportanttoidentifythekeyparametersofinter-
Martin Buxton, BA, Professor, Health Economics Research
est. It would also be relatively straightforward to elicit a range
Group,BrunelUniversity,Uxbridge,UnitedKingdom
ofoutcomes,includingresourceuseorcostestimatesrequired
foranearlyassessmentofcost-effectiveness.However,elicita-
tion of probability distributions is time-consuming, so care is
CONFLICTSOFINTEREST
needednottooverburdenrespondents.
All authors have received a grant to their institutes from the
Validity of the meta-analysis model must also be consid-
Engineering and Physical Sciences Research Council funded
ered. Various assumptions on the model structure could have
MATCH Project. Martin Buxton has also received honoraria
beenmade.Anoverarchingassumptionisthatheterogeneityin
from Medtronics International for advisory board meetings on
thedataassociatedwiththeearlierversionsofICDsislessthan
HTAissues.
thedifferencebetweenthenewIS-4deviceandtheearlierver-
sions.Althougharound44percentofthetotalvariationintrial
estimates that is due to heterogeneity is considered reasonable APPENDIX1:MATHEMATICALREPRESENTATIONOFTHEMTC
inthecurrentstudy,thereismuchvariationinthelevelsofdoses
MODELWITHADJUSTMENTFORTHENEWTECHNOLOGY
of anti-arrhythmic drugs and other adjuvant therapies such as
Foreachtriali,theincludedtreatmentearliestinthealphabet(labeledwiththe
theuseofbetablockersacrossstudies.Thereareseveralwaysto indexbbelow)wastakenasthebaselinetreatment(e.g.,intrialscomparing
121 INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012
Younetal.
A with B, b = A). The Bayesian MTC model, with adjustment for a new, delta.u[i]<-min(delta[i],-mu[s[i]])
modifieddevice,isshowninEq.1. delta[i]∼dnorm(md[i],prec)
md[i]<-d.study[i]+beta[s[i]]∗equals(t[i],3)
rk,i ∼Bino(cid:1)mial(pk,i,nk,i) #betatobeincludedonlywhenthetreatmentisICD(trt=3)
lδoiUkgb(p=k,mi)i=n{δikµb,ib−µ+µibiδbiUk=b −iflbokg=a(plpAbh,i,a)Bb}e,tCic.ailflyka=ftebrb d}.study[i]<-d[t[i]]-d[b[i]]
δikb ∼Norma(cid:2)l(dkb+(cid:3)βi,τ2)∼Normal((dAk−dAb)+βi,τ2) (Eq.1) #Informationondistributionofbeta
βi ∼Normal µβ,σβ2 mu.beta<-0.001680164
prec.beta<-1/pow(0.006045638,2)
Priors:−µib ∼Gamma(.001,.001), #Priorsandadjustmentatthestudylevel(Nstud=12)
dAk ∼Normal(0,10000),wherek=B,D,
τ ∼Uniform(0,2) for(jin1:NStud)
{
where rk,i are the number of deaths out of the total number of patients at
risk nk,i for each treatment k in study i; pk,i is the probability of death in mu[j]<-0-mu.minus[j]
studyiundertreatmentk;µib isthelogarithmofbaselinemortalityriskfor mu.minus[j]∼dgamma(0.001,0.001)
triali.Theparameterδikb representsthelogrelativerisk(LRR)oftreatment beta[j]∼dnorm(mu.beta,prec.beta)
k relative to baseline treatment b for study i. The nuisance parameter δiUkb }
is included to constrain the range of probabilities of an event below 1 (see
#Treatmentlevelprior(NTrt=3)
(40)).Thesetrial-specificLRRaredrawnfromacommondistribution,and
thepooledLRR,dbk,isdefinedwithrespecttothereferencetreatmentA(e.g. d[1]<-0
d =d −d ).Theinclusionofβ signifiesthemeanmortalityeffectof for(kin2:NTrt)
BC AC AB
the conventional device estimated from observed data is deviated from the {
effectofthenewdevice,dkbbythemagnitudeofβ. d[k]∼dnorm(0,0.0001)
Finally,aflatGammapriorwasspecifiedforthenegativeofµibtocon- }
straintherangeofµibbelow0,andNormalpriorsforthe‘basicparameters’
#Predictivedistribution
whose comparators are the reference treatment A. The between-study vari-
ance(heterogeneityparameter),τ2,wasassignedaUniformprior.Thepoten- d.pred[1]<-0
tialcorrelationbetweenrelativeeffectivenessparametersfortrialsinvolving for(kin2:NTrt)
morethantwoarmswerenotconsideredinthisstudyforsimplicity. {d.pred[k]∼dnorm(d[k],prec)}
Expert opinions form the basis of estimating µβand σβ. Although βis #PairwiseLRRandRR:newICD
representthedifferencebetweentwologrelativerisks,actualquestionsasked
for(cin1:2){
expertstodirectlycomparetheproportionofpatientswhowoulddieunderthe
modified device therapy with that under the conventional device treatment. for(kin(c+1):3){
This was justified by the relationshipE(δiCb) = dDb +E(βi) = dDb +µβ, lrr[c,k]<-d[k]-d[c]
wherebothδandddenotethelogrelativerisks,butδcomparesthebaseline log(rr[c,k])<-lrr[c,k]
treatmentwithtreatmentCwhileddoeswithtreatmentD.Takingawaythe lrr.pred[c,k]<-d.pred[k]-d.pred[c]
logarithmfromtheaboverelationshipgives: log(rr.pred[c,k])<-lrr.pred[c,k]
(cid:2) (cid:3) }
E RRC P /P P
eβi = i ≈ C Control = C }
RRD P /P P
D Control D prec<-1/tau.sq
tau.sq<-tau∗tau
PisariskofdeathandRRarelativerisk.Themortalityriskundercontrol
treatment,P ,cancelsout,makingpossiblethedirectcomparisonbetween tau∼dunif(0,2)
Control
treatmentsCandD,i.e.,PC andPD. resdev<-sum(dev[])#Residualdeviance
}
APPENDIX2:FULLBUGSCODEFORMTCMODELWITH
ADJUSTMENTPARAMETERINCORPORATED
#Data
list(Narm=25,NStud=12,NTrt=3)
model{ s[] t[] r[] n[] b[]
#Modeldefinedattreatmentarmlevel(Narm=25) 1 1 244 847 1
for(iin1:Narm) 1 2 240 845 1
{
1 3 182 829 1
log(p[i])<-mu[s[i]]+delta.u[i]∗(1-equals(t[i],b[i])) 2 1 58 342 1
r[i]∼dbin(p[i],n[i]) 2 3 62 332 1
rhat[i]<-p[i]∗n[i] 3 1 40 229 1
dev[i]<-2∗(r[i]∗(log(r[i])-log(rhat[i]))+(n[i]-r[i])∗(log(n[i]-r[i]) 3 3 28 229 1
-log(n[i]-rhat[i]))) 4 1 97 490 1
INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012 122
Bayesianmeta-analysisonmedicaldevices
4 3 105 742 1 BerryDA,eds.Meta-analysisinmedicineandhealthpolicy.NewYork:
5 1 95 454 1 MarcelDekker;2000:29-63.
2. Ades AE, Welton NJ, Caldwell D, et al. Multiparameter evidence syn-
5 3 101 446 1
thesisinepidemiologyandmedicaldecision-making.JHealthServRes
6 1 131 617 1
Policy.2008;13(Suppl3):12-22.
6 3 105 595 1
3. BanschD,AntzM,BoczorS,etal.Primarypreventionofsuddencardiac
7 1 17 54 1 death in idiopathic dilated cardiomyopathy: The cardiomyopathy trial
7 3 13 50 1 (CAT).Circulation.2002;105:1453-1458.
8 2 122 509 2 4. Bardy GH, Lee KL, Mark DB, et al. Amiodarone or an implantable
cardioverter-defibrillator for congestive heart failure. N Engl J Med.
8 3 80 507 2
2005;352:225-237.
9 2 98 331 2
5. BiggerJTJr.Prophylacticuseofimplantedcardiacdefibrillatorsinpa-
9 3 83 328 2
tientsathighriskforventriculararrhythmiasaftercoronary-arterybypass
10 2 39 101 2 graftsurgery.coronaryarterybypassgraft(CABG)patchtrialinvestiga-
10 3 15 95 2 tors.NEnglJMed.1997;337:1569-1575.
11 2 7 52 2 6. Bristow MR, Saxon LA, Boehmer J, et al. Cardiac-resynchronization
therapywithorwithoutanimplantabledefibrillatorinadvancedchronic
11 3 6 51 2
heartfailure.NEnglJMed.2004;350:2140-2150.
12 2 40 92 2
7. ConnollySJ,GentM,RobertsRS,etal.Canadianimplantabledefibril-
12 3 36 99 2
lator study (CIDS): A randomized trial of the implantable cardioverter
END defibrillatoragainstamiodarone.Circulation.2000;101:1297-1302.
8. Cooper NJ, Sutton AJ, Morris D, Ades AE, Welton NJ. Addressing
between-studyheterogeneityandinconsistencyinmixedtreatmentcom-
parisons:Applicationtostrokepreventiontreatmentsinindividualswith
APPENDIX3:SAMPLEQUESTIONNAIREUSEDTOASSISTVERBAL non-rheumaticatrialfibrillation.StatMed.2009;28:1861-1881.
INTERVIEWS 9. Ezekowitz JA, Rowe BH, Dryden DM, et al. Systematic review: Im-
What is your opinion of the likely impact of the four-pole connector on plantablecardioverterdefibrillatorsforadultswithleftventricularsystolic
mortality? dysfunction.AnnInternMed.2007;147:251-262.
AccordingtotheSCD-HeFT∗ andAVID∗∗ studies,onaveragearound35% 10. HauserRG,AlmquistAK.Learningfromourmistakes?TestingnewICD
ofICDpatients(bothprimaryandsecondaryprevention)currentlydiewithin technology.NEnglJMed.2008;359:2517-2519.
5yearsduetocardiacandnon-cardiaccauses. 11. Hohnloser SH, Kuck KH, Dorian P, et al. Prophylactic use of an im-
Doyouthinkitislikelythat5-yearmortalitywouldimproveasaresultof plantable cardioverter-defibrillator after acute myocardial infarction. N
usingafour-poleconnector,comparedtothecurrentconnectorsystem? EnglJMed.2004;351:2481-2488.
12. Jansen JP, Crawford B, Bergman G, Stam W. Bayesian meta-analysis
Yes No Don’tknow ofmultipletreatmentcomparisons:Anintroductiontomixedtreatment
comparisons.ValueHealth.2008;11:956-964.
13. KadishA,DyerA,DaubertJP,etal.Prophylacticdefibrillatorimplanta-
Supposethat1,000patientsweretohaveanICDimplanted.Howmanydeaths,
tioninpatientswithnonischemicdilatedcardiomyopathy.NEnglJMed.
onaverage,doyouthinkmightbeaverted/causedovera5yearperiodifa
2004;350:2151-2158.
four-poleconnectorwasusedinsteadofaconventionalICD?
14. KuckKH,CappatoR,SiebelsJ,RuppelR.Randomizedcomparisonof
antiarrhythmicdrugtherapywithimplantabledefibrillatorsinpatientsre-
suscitatedfromcardiacarrest:Thecardiacarreststudyhamburg(CASH).
Minimum YourBestGuess Maximum
Circulation.2000;102:748-754.
[number]more/ [number]more/ [number]more/ 15. Leal J, Wordsworth S, Legood R, Blair E. Eliciting expert opinion for
less(tick)deaths less(tick)deaths less(tick)deaths economicmodels:Anappliedexample.ValueHealth.2007;10:195-203.
16. LunnDJ,ThomasA,BestN,SpiegelhalterD.WinBUGS-Abayesian
modellingframework:Concepts,structure,andextensibility.StatCom-
Pleasetrytorepresentyouropiniononthescalebelowbydistributingatotal put.2000;10:325-337.
of100pointsbetweenthelinestoreflectyoursubjectivebeliefsabouteach 17. Moss AJ, Hall WJ, Cannom DS, et al. Improved survival with an im-
rangeoccurring. planted defibrillator in patients with coronary disease at high risk for
ventricular arrhythmia. multicenter automatic defibrillator implantation
trialinvestigators.NEnglJMed.1996;335:1933-1940.
| | | | | | | | | | | 18. MossAJ,ZarebaW,HallWJ,etal.Prophylacticimplantationofade-
−25 −20 −15 −10 −5 0 +5 +10 +15 +20 +25 fibrillator in patients with myocardial infarction and reduced ejection
fraction.NEnglJMed.2002;346:877-883.
Numberofliveslostper1,000patients Numberoflivessavedper1,000patients
19. O’Hagan A, Buck EC, Daneshkhah A, et al. Uncertain Judgements:
ElicitingExperts’Probabilities.NewYork:JohnWiley&Sons;2006.
20. Strickberger SA, Hummel JD, Bartlett TG, et al. Amiodarone versus
implantablecardioverter-defibrillator:Randomizedtrialinpatientswith
REFERENCES nonischemic dilated cardiomyopathy and asymptomatic nonsustained
1. Abrams RK, Lambert CP, Shaw C, Sanso B. Meta-analysis of hetero- ventriculartachycardia–AMIOVIRT.JAmCollCardiol.2003;41:1707-
geneouslyreportedstudyresults:ABayesianapproach.In:StanglDK, 1712.
123 INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012
Younetal.
21. TheInvestigatorsoftheAntiarrhythmicversusImplantableDefibrillators 23. TurnerRM,SpiegelhalterDJ,SmithGC,ThompsonSG.Biasmodelling
(AVID) Trial. A comparison of antiarrhythmic-drug therapy with im- inevidencesynthesis.JRStatSocSerAStatSoc.2009;172:21-47.
plantabledefibrillatorsinpatientsresuscitatedfromnear-fatalventricular 24. Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects
arrhythmias.theantiarrhythmicsversusimplantabledefibrillators(AVID) meta-analysis oftrialswithbinaryoutcomes:Methodsfortheabsolute
investigators.NEnglJMed.1997;337:1576-1583. riskdifferenceandrelativeriskscales.StatMed.2002;21:1601-1623.
22. TungR,ZimetbaumP,JosephsonME.Acriticalappraisalofimplantable 25. WeltonNJ,AdesAE,CarlinJB,AltmanDG,SterneJAC.Modelsforpo-
cardioverter-defibrillator therapy for the prevention of sudden cardiac tentiallybiasedevidenceinmeta-analysisusingempiricallybasedpriors.
death.JAmCollCardiol.2008;52:1111-1121. JRStatSocSerAStatSoc.2009;172:119-136.
INTL.J.OFTECHNOLOGYASSESSMENTINHEALTHCARE28:2,2012 124