Prediction models in reproductive medicine

16 Pages·2009·0.9 MB·English

HumanReproductionUpdate,Vol.15,No.5pp.537–552,2009
AdvancedAccesspublicationonMay12,2009 doi:10.1093/humupd/dmp013
Prediction models in reproductive
medicine: a critical appraisal†
Esther Leushuis1,2,5, Jan Willem van der Steeg2, Pieternel Steures2,
Patrick M.M. Bossuyt3, Marinus J.C. Eijkemans4, Fulco van der Veen2,
D
Ben W.J. Mol2,3, and Peter G.A. Hompes1 o
w
n
1DepartmentofObstetricsandGynecology,VrijeUniversiteitMedicalCenter,Amsterdam,TheNetherlands2DepartmentofObstetrics/ loa
Gynaecology,CenterforReproductiveMedicine,AcademicMedicalCenter,RoomH4-238,Meibergdreef9,1105AZ,Amsterdam,The de
NofePthuebrlilcanHdesa3lDthe,pEarratsmmeunstMofCC,lUinniciavleErspitidyeMmeiodliocgaylCanedntBeiroRstoatttisetricdsa,mA,cRadoettmericdaMme,dTichaelCNeenttheer,rlAanmdssterdam,TheNetherlands4Department d fro
m
5Correspondenceaddress.Tel:þ31-20-5663456;Fax:þ31-20-6963489;E-mail:[email protected] http
s
table of contents ://ac
........................................................................................................................... a
d
e
† Introduction m
ic
† Methods .o
u
† Results p.c
o
† Discussion m
† Supplementary data /hu
m
u
p
d
/a
background: rtic
Prediction models have been developed in reproductive medicine to help assess the chances of a treatment- le
(in)dependent pregnancy.Carefulevaluationis neededbefore these models can beimplemented in clinical practice. -ab
s
methods:Wesystematicallysearchedtheliteratureforpapersreportingpredictionmodelsinreproductivemedicineforthreestrategies: tra
c
expectantmanagement,intrauterineinsemination(IUI)orinvitrofertilization(IVF).Weevaluatedwhichphasesofdevelopmentthesemodels t/1
5
hadpassed,distinguishingbetween(i)modelderivation,(ii)internaland/orexternalvalidation,and(iii)impactanalysis.Wesummarizedtheir /5
/5
performance at external validation in termsof discriminationand calibration. 3
7
results: /6
Weidentified36papersreportingon29predictionmodels.Therewere9modelsforthepredictionoftreatment-independent 3
4
4
pregnancy,3forthepredictionofpregnancyafterIUIand17forthepredictionofpregnancyafterIVF.Allofthemodelshadcompletedthe 8
8
phaseofmodelderivation.Forsixmodels,thevalidityofthemodelwasassessedonlyinthepopulationinwhichitwasdeveloped(internal b
y
validation).Foreightmodels,thevaliditywasassessedinpopulationsotherthantheoneinwhichthemodelwasdeveloped(externalvali- g
u
e
dation),and onlythreeof theseshowed good performance.Onemodel had reached thephase of impactanalysis. s
conclusions: Currently,therearethreemodelswithgoodpredictiveperformance.Thesemodelscanbeusedreliablyasaguidefor t on 1
makingdecisionsaboutfertilitytreatment,inpatientssimilartothedevelopmentpopulation.Theeffectsofusingthesemodelsinpatientcare 0 A
have tobe furtherinvestigated. pril 2
0
Keywords: prediction model/ fertility / pregnancy/ spontaneouspregnancy / ART(IUI/IVF) 1
9
Introduction diseaseandinvitrofertilization(IVF)withassistedfertilizationaftersur-
gical sperm retrieval in couples with azoospermia. In many couples,
Until recently, the emphasis in reproductive medicine has been on such causal factors cannot be found. These couples are classified as
finding causal diagnoses of subfertility followed by treatment of the having unexplained subfertility, mild male subfertility, cervical factor
diagnosedcondition.Examplesareovulationinductioninwomendiag- subfertility, mild endometriosis or one-sided tubal pathology; and
nosed with anovulation, tubal surgery in women with bilateral tubal assisted reproductive techniques such as intrauterine insemination
†Abstractpresentedatthe24thAnnualMeetingoftheEuropeanSocietyofHumanReproductionandEmbryology(ESHRE)inBarcelona,Spain(6–9July2008).
&TheAuthor2009.PublishedbyOxfordUniversityPressonbehalfoftheEuropeanSocietyofHumanReproductionandEmbryology.Allrightsreserved.
ForPermissions,pleaseemail:[email protected]
538 Leushuisetal.
Figure 1 Phasesofmodeldevelopment.
D
o
w
n
lo
a
d
e
d
fro
m
h
ttp
s
://a
c
a
d
e
m
ic
.o
u
p
.c
o
m
/h
u
m
u
p
d
/a
rtic
le
-a
b
s
tra
c
t/1
5
/5
/5
3
7
Figure2 (a)TypicalROCcurveofapredictionmodelinreproductivemedicine(AUC0.56)and(b)calibrationplotwithcalculatedprobabilityon /6
3
theX-axisandobservedproportiononY-axis,showinggoodcalibration. 4
4
8
8
b
y
(IUI)orIVFarethenconsidered.Astheseinterventionsareexpensive impactanalysis(McGinnetal.,2000;ReillyandEvans2006;Steyerberg, gu
e
andnotwithoutsideeffects,theyshouldbeofferedtoacoupleonlyif 2008).Wealsosummarizedtheirperformance. s
t o
the expected success rate with treatment substantially exceeds the n
1
probability of a spontaneous pregnancy (Wasson et al., 1985; 0
Methods A
teVeldeand Cohlen, 1999). p
Itisaclinicalchallengeforgynaecologiststomakesuchacomparison. ril 2
Gynaecologistsareknowntodifferwidelyintheirestimationsoftheprob- Search strategy and study selection 019
abilityofachievingapregnancyforsubfertilecouples(vanderSteegetal., WeperformedastructuredpredefinedliteraturesearchusingMEDLINE,
2006).Tohelpgynaecologistsinassessingthechancesofpregnancy,pre- EMBASEandtheCochraneLibraryfrominceptiontoOctober2008.An
dictionmodelshavebeendeveloped.Withthesemodels,onecancalcu- informationspecialistperformedtheelectronicsearchusingthefollowing
latetheprobabilityofatreatment-independentpregnancyaswellasthe terms: pregnancy, live birth, conception, infertility/subfertility/fertility,
intrauterine insemination, in vitrofertilization, predictionmodels and vali-
probabilityofsuccesswithIUIandIVF.
dation. We checked cross-references of eligible papers to identify other
Carefulevaluationisneededbeforethesemodelscanbeimplemented
papers not captured by electronic searches. No restrictions were held
inclinicalpractice.Theuseofpoor-qualitypredictionmodelscouldhavea
concerningpublicationyearorlanguage.AReferenceManager11.0data-
negativeeffectondecisionmaking,byintroducingtheillusionofobjective
basewasestablishedtoincorporateresultsofallcitations.
improvementoverclinicaljudgment.Wesystematicallyreviewedthelit-
Tworeviewers(J.W.S.andE.L.)evaluatedpotentiallyeligiblepapersina
eratureontheavailablepredictionmodelsinreproductivemedicine.We two-stage process. First, papers identified in the search were indepen-
appraised the prediction models according to a published evaluation dently screened for eligibility by reading the title and abstract. If there
scheme,distinguishingbetweenmodelderivation,modelvalidationand wereany doubts abouteligibilityafterreading the title, wescreenedthe
Predictionmodelsinreproductivemedicine 539
D
o
w
n
lo
a
d
e
d
fro
m
h
ttp
s
://a
c
a
d
e
m
ic
.o
u
p
.c
o
m
/h
u
m
u
p
d
/a
rtic
le
-a
b
s
tra
c
t/1
5
/5
/5
3
7
/6
3
4
4
8
8
b
y
g
u
e
s
t o
n
1
Figure3 Processfrominitialsearchtofinalinclusionforpapersonpredictionmodelsinreproductivemedicine. 0
A
p
ril 2
0
1
9
Assessment of study quality
abstractandthefulltexttomakesurenopapersweremissed.Wethen
obtained full-text versions of all papers selected by at least one of the For each included paper, we identified the study characteristics and
reviewersinthefirststage.Paperswereincludediftheyreportedonapre- assessed the study quality on the basis of the following items for all
diction model for treatment-independent pregnancy, pregnancy after IUI models: whether the patient selection was consecutive, whether the
or IVF. If the paper reported on a model for embryo transfer only, it data had been collected prospectively, whether the variables and preg-
wasexcluded. nancy (or live birth) were described in sufficient detail and whether
In this review, a prediction model was defined as a model that missing data were reported and/or imputed (‘filled in’). For papers that
expressed pregnancy as a function of one or more predictor variables. reported on treatment-independent pregnancy, the basic fertility
Suchamodelcanbebasedonamultivariableregressionmodel,suchas work-uphadtobeclearlydescribed,notreatmentbetweenbasicfertility
alinear,logisticorCoxproportionalhazardsregressionsmodel.Tobeeli- work-up and time to pregnancy should have been applied and the
gible,thereportedpredictionmodelhadtobepresentedasascorechart, follow-up duration had to be at least 1 year. We also verified whether
apredictionruleorasasetofregressioncoefficientswithbaselineinter- papers that reported on treatment-independent models had been
cept,sufficienttomakepredictionsforindividualcases. derivedfromCoxproportionalhazardsanalysiswithorwithoutright-hand
540 Leushuisetal.
TableI Studycharacteristicsofthepapersthatreportonpredictionmodelsfortreatment-independentpregnancy
Firstauthor Patients Inclusionandexclusioncriteria n Study Outcomec
(year) designb
.............................................................................................................................................................................................
Jedrzejczaketal. Menfrominfertilecoupleswithoutfemale Exclusion(cases): 242 ccstudy preg.
(2008) infertilityfactor(cases)matchedwith -azoospermia
healthyfertilespermdonors(controls) -totallackofspermmotility
Exclusion(controls):
-historyofpastinfertility
-historyofinflammationorsurgery
ofthereproductiveorgans,1year
vanderSteegetal. Subfertilecouplesnotevaluatedpreviouslyfor Exclusion: 3021 pros.CH ong.preg.
D
(2007) subfertilityreferredbyageneralpractitioner -ovulationdisorder o
ExternalvalidationofHunaultetal.(2004) -TMC,3(cid:2)106 wn
-one-ortwo-sidedtubalpathology lo
a
d
Hunault Couplesfromtwouniversityhospitals Inclusion: 302 pros.CH livebirth e
d
(2005) withsubfertilityduetomildmale, -woman’sage,40years fro
cervicalorunexplainedsubfertility Exclusion: m
ExternalvalidationofHunaultetal.(2004) --oazvouolastpioenrmdiiasorder https
-one-sided/two-sidedtubaldefect ://a
c
-endocrinedisorder a
d
Hunaultetal. PatientsfromSnicketal.(1997), Exclusion: a pros.CH livebirth em
(2004) Collinsetal.(1995)andEimersetal.1994a -ovulationdisorder ic
.o
ExternalvalidationofSnicketal.(1997), -tubalpathology u
p
Collinsetal.(1995)andEimersetal.(1994) -azoopermia .c
o
m
Hunault Firstvisitofsubfertilecouplesatan Exclusion: 1061 pros.CH livebirth
/h
(2002a) universityfertilityclinic -ovulationdisorder u
m
Externalvalidationofalternatemodelof -azoospermia u
p
Eimersetal.(1994) -one-sided/two-sidedtubaldefect d
/a
Snicketal. Subfertilecouplesfromasecondarycare Inclusion: 726 pros.CH livebirth rtic
(1997) fertilitycentre -childwish le
-.1yearnopregnancy -ab
s
Collinsetal. Firstvisitofsubfertilecouplesatan Noexclusioncriteriareported 2198 pros.CH livebirth tra
(1995) universityfertilityclinic ct/1
5
Bahamondesetal. Subfertilecouplesconsultinginfertility Exclusion: 559 ret.CH preg. /5
(1994) clinicwith3yearsoffollow-upor -divorcedduringstudy /5
3
pregnancy -historyoftuballigationorhabitual 7
/6
abortion 3
4
-azoospermia 4
8
8
Wichmannetal. Subfertilemen,referredtoandrological Exclusion: 907 pros.CH preg. b
y
(1994) laboratoryforsubfertilityproblemswith –abstinenceperiod,3days g
u
registereddurationofsubfertility -incompletesample e
s
-azoospermia t o
-donorinsemination n
1
0
Eimersetal. Subfertilecouplesfromauniversity Inclusion: 996 Pros.CH preg. A
(1994) fertilitycentre -cycleof23–35days p
-biphasicBTC ril 2
0
-noazoospermia 1
9
-noabnormalHSGorlaparoscopy
Bostofteetal. Subfertilecouplesinvestigatedfor Noexclusioncriteriareported 321 pros.CH preg.
(1993) subfertilityatauniversityhospital
Bostofte Menwithsemenanalysisforsubfertility Exclusion: 765 ret.CH preg.
(1987) whorespondedtoaquestionnaire -azoospermia
-invalidnameandbirth
-death/emigration
-nottraceableinofficialregisters
aFordetails,seeSnicketal.(1997),Collinsetal.(1995)andEimersetal.(1994).
bStudydesign:ccstudy¼casecontrolstudy;pros.CH¼prospectivecohortstudy;ret.CH¼retrospectivecohortstudy.
cOutcome:preg.¼pregnancy;ong.preg.¼ongoingpregnancy.
Predictionmodelsinreproductivemedicine 541
TableII StudycharacteristicsofthepapersthatreportonpredictionmodelsforpregnancyafterIUI
Firstauthor Patients Inclusionandexclusioncriteria n Study Outcomeb
(year) designa
.............................................................................................................................................................................................
Erdemetal.(2008) Patientswithunexplained,mildmale Inclusion: 456 ret.CH livebirth
infertilitywithregularmenstrualcycles -midlutealprogesterone.10ng/ml
-confirmedbilateraltubalpatency
-normalsemenanalysis(WHO,1992)
Exclusion:
-PCOS
-previousovariansurgery
-totalmotilespermcount(TMC)
,1(cid:2)106/mlpost-wash D
o
w
Custersetal.(2007) CouplestreatedwithIUI Inclusion: 1079 pros.CH ong.preg. n
ExternalvalidationofSteuresetal.(2004) -confirmedovulatorycycle lo
a
-atleastonepatenttube de
d
Steuresetal.(2004) CouplestreatedwithIUI Inclusion: 3371 ret.CH livebirth fro
allwomenwithIUIforreasonsof: m
--mcearlveicfaalcftaocrtor https
-unexplainedsubfertility ://a
c
Tomlinsonetal.(1996) CouplestreatedwithIUI Inclusion: 260 ret.CH preg. ad
e
allwomenwithIUIforreasonsof: m
-unexplainedsubfertility ic
.o
-mildspermdysfunction u
p
-anovulation .c
o
-cervicalmucushostility m
/h
u
aStudydesign:pros.CH¼prospectivecohortstudy;ret.CH¼retrospectivecohortstudy. m
bOutcome:preg.¼pregnancy;ong.preg.¼ongoingpregnancy. up
d
/a
rtic
le
censoring.Weaddedtwoitemsforpapersthatreportedonthepredic- (phase 3a) or in varied settings (phase 3b), preferably in a randomized -a
tionoftreatment-dependentpregnancy(IUIorIVF):whetherthediagnosis controlledtrial. bs
beforetreatmentwasdescribedinsufficientdetailandwhethertheproto- tra
c
coloftreatmentwasdescribedinsufficientdetail.Wereportstudyquality Assessment of model performance t/1
5
separately for treatment-independent and treatment-dependent models, /5
i.e.IUIandIVF. Forthemodelsthatwereevaluatedinanexternalvalidation,wequantified /53
theperformanceofthepredictionmodelsbyassessingdiscriminationand 7
/6
calibration.Discriminationreferstotheabilitytodistinguishcoupleswho 3
Assessment of model development 4
willconceivefromthosewhowillnot.Iftherearemultiplescoresorprob- 48
8
Weassessedthedevelopmentofthepredictionmodelswithapublished abilities, the sensitivity–specificity pairs for each cut-off value can be b
y
evaluation scheme, which distinguishes three phases: model derivation, plotted in a receiver operating characteristic (ROC) curve (Fig. 2a) g
u
model validation and impact analysis (Fig. 1) (McGinn et al., 2000; Reilly (HanleyandMcNeil,1982).Inthatcase,discriminationcanbeexpressed e
s
andEvans2006;Steyerberg,2008).Inthemodelderivationphase,predic- astheareaunderthisROCcurve(AUC)orthec-statistic(Tostesonetal., t o
n
torsareidentified,basedonpriorknowledge,andtheweightofeachpre- 1994). An AUC of 1 implies perfect discrimination, whereas an AUC of 1
0
dictor(regressioncoefficient)iscalculated.Inthesecondphase,onecan 0.5meansthatthetestdoesnotdiscriminateatall(HanleyandMcNeil, A
p
ddiasttiionngu(isphhbaseetw2ebe)n.tWheitihnteinrtnearlnvaallidvaaltiidoanti(opnhaosfea2ap)raenddictthioenexmteordneall,vtahlie- 1if9t8h2e).AFoUrCthliisesrebveietww,eaenm0o.d5e0liasncdon0s.i7d0e.reAdntoAhUaCvebpeotowreepnerf0o.r7m0aanncde ril 20
1
model’s ability to predict outcome in the group of patients in which it 0.80 represents fair performance, and an AUC between 0.80 and 0.90 9
wasdevelopedisevaluated,sometimeswithdatacollectedinaseparate representsgoodperformance.
group of patients evaluated in the same setting (Altman and Royston, Calibration refers to the level of correspondence between the calcu-
2000). As internal validation systematically gives a too optimistic lated pregnancy chances and the observed proportion of pregnancies.
impression about the quality of the predictions, external validation is a Calibration can be evaluated by several techniques of which we will
vital next step in assessing the performance of the model (Harrell et al., describe the three techniques that are most commonly used. The first
1996).Withexternalvaliditionofapredictionmodel,themodel’sability technique relies on a goodness-of-fit test for the model for predicting
to predict outcome in populations other than the population in which pregnancy (Hosmer, 2000). The second technique uses the coefficients
themodelwasdeveloped,alsocalled‘generalizability’or‘transportability’, of the linear regression line through the prediction–observation pairs in
isevaluated.Thethirdandfinalphaseconsistsofimpactanalysis,whichis acalibrationplottoevaluatetheperformanceofamodel.Ifthecalibration
theevaluationoftheimplementationofpredictionmodelswithdocumen- isperfect,thelinewillbeonthediagonal,withinterceptzeroandslope
ted validity. Impact analysis establishes whether the prediction model unity(Cox,1958).Formodelswithaslopebelow1,high-probabilitypre-
improves doctors’ decisions byevaluating the effect on patient outcome dictions are too high and low-probability predictions are too low. If the
(Reilly and Evans 2006; Steyerberg, 2008). This can be evaluated in one slope exceeds 1, the bias is the other way around (Steyerberg et al.,
542 Leushuisetal.
TableIII StudycharacteristicsofthepapersthatreportonpredictionmodelsforpregnancyafterIVF
Firstauthor(year) Patients Inclusionandexclusioncriteria n Studydesignd Outcomee
.............................................................................................................................................................................................
vanWeertetal. Allcoupleswithmalesubfertility Inclusion: 275ptnc ret.CH ong.preg.
(2008) undergoingIVFtreatment -2semenanalysesthatdidnotmeet
WHOcriteria
-oocyteretrieval
Hunaultetal.(2007) Patientsfromauniversityhospital Inclusion:-transferoftwoembryos 642ptn ret.CH ong.preg.
intheirfirstIVFcycle Exclusion:
ExternalvalidationofHunault(2002b) -ICSItreatment
-oocytedonation
-cryopreservedembryos
D
Lintsenetal.(2007) CoupleseligibleforIVFandICSIa Exclusion: 4928ptn pros.CH ong.preg. o
w
-norecordoffollow-updates n
lo
-nostartoftreatmentforknown a
d
reasons e
d
Verbergetal.(2007) Infertilepatientswitharegular Inclusion: 201ptn pros.CH ong.preg. fro
m
indicationforIVForICSIatan -menstrualcycle25–35days
h
universityhospital -BMI18–28kg/m2 ttp
E-xpcrleuvsiioouns:IVF s://a
c
-unhealthychildafterIVF ad
e
-frozenembryostransfer m
ic
Carrera-Rotllanetal. Patientswithprimaryinfertilitydue Inclusion: 110ptn pros.CH preg. .o
(2007) toatubalfactorwithnormalsemen -age,38years up
parametersattheirfirstIVFattempt -menstrualcycle24–32days .co
m
-normalFSH/LH/E2/prolactin/
/h
TSH/BMI u
m
Exclusion: u
p
-age(cid:3)38years d
/a
-lohsisstoorryproefimgepnlaentitcatriiosknsg/epnreetgincancy rticle
diagnosis -a
b
s
Ottosenetal.(2007) IVFandICSItreatmentcyclesfrom Exclusion: 2193cyc. ret.CH preg. tra
apublicfertilityclinic -frozenembryoreplacement c
-singleembryotransfer t/15
/5
Ferlitschetal.(2004) WomenreferredforIVFtoa Exclusion: 170ptn ret.CH preg. /5
universityhospitalofknownheight -severeendometriosis 37
andweightattheirinitialIVFcycle -asingleovarywithapossible /6
3
normalovarianresponse 44
8
-anyovariancystmeasuring 8
.10mmindiameterona by
baselineday gu
e
Hunault(2002b) cWycolemenundergoingtheirfirstIVF E-xscinlugsleioenm: bryotransfer(ET) 642ptn ret.CH ong.preg. st on
-oocytedonation 10
-cryothawedembryocycles A
p
--IcCycSlIesnotresultinginET ril 20
1
Smeenketal.(2000) CoupleswhostartedtheirfirstIVF Exclusion: 1253ptn pros.CH ong.preg. 9
cycleinauniversityhospital -ICSIcycles
ExternalvalidationofTempletonetal. -donorgametes
(1996) -frozenembryos
Stolwijketal.(2000) Coupleswhounderwenttheirfirst Inclusion: 1315ptn pros.CH ong.preg.
IVForICSItreatmentatauniversity -(cid:4)41yrandFSH,20IU/L
fertilitycentre Exclusion:
-donorsemen
-MESAorTESEb
-donoroocytes
Continued
Predictionmodelsinreproductivemedicine 543
TableIII Continued
Firstauthor(year) Patients Inclusionandexclusioncriteria n Studydesignd Outcomee
.............................................................................................................................................................................................
Bancsietal.(2000) Womenundergoingtheirfirst Inclusion: 435ptn ret.CH ong.preg.
stimulatedIVFcycleatanacademic -regularmenstrualcycle
fertilitycentre -bFSHlevelonday1–4
-noendocrinedisorder
-nooocytedonation
-nounstimulatedcycles
Stolwijketal.(1998) CompleteIVFcycleswithhormone Exclusion: 757cyc. pros.CH ong.preg.
ovulationinduction
-ICSI 432cyc.
ExternalvalidationofStolwijketal. -donoroocytes 428cyc. D
o
(1996) -donorspermatozoa 1424cyc. w
n
-IVFforunexplainedsubfertility loa
d
Minaretzisetal.(1998) ConsecutiveIVFcycles Inclusion: 544ptn pros.CH livebirth ed
-atleastoneembryotransfer fro
m
Commenges-Ducos ConsecutiveIVF-embryotransfer Exclusion: 923cyc. ret.CH ong.preg. h
etal.(1998) cycles -hyperandrogenism ttp
--udtieetrhinyelstmilbaolfeosrtmroaltisoynndrome s://ac
a
-age(cid:3)40yearswithabnormal d
e
ovariantestreserve m
-cryo-ordonationoocytes ic.o
u
Templetonetal. AllIVFtreatmentcylcesinanational Exclusion: 36961cyc. ret.CH livebirth p
.c
(1996) database -sperm,oocyteorembryodonation o
m
-frozenembryotransfer /h
-microassistedfertilization u
m
-unstimulatedcycle(naturalIVF) u
p
d
Stolwijketal.(1996) Coupleswhounderwenttheirfirst Exclusion: 757cyc. ret.CH ong.preg. /a
IVFcycle -donoroocytes rtic
-ICSI le
-a
Bouckaertetal.(1994) PatientstreatedforIVF Noexclusioncriteriareported 591ptn ret.CH preg. bs
Haanetal.(1991) Allregulartreatmentcyclesfromfive Noexclusioncriteriareported 3092cyc. pros.CH ong.preg. trac
IVFcentres t/1
5
Hughesetal.(1989) ConsecutiveIVFcycles Noexclusioncriteriareported 716cyc. pros.CH ong.preg. /5
/5
Nayuduetal.(1989) IVFpatientswithfollicularaspirate Exclusion: 222ptn ret.CH ong.preg. 37
-ectopicpregnancy /63
-post13weeksabortion 44
8
-follicularfluidnotpresentfor 8
technicalreasons by
g
u
aICSI¼intracytoplasmaticsperminjection. es
bMESA¼microepididymalspermaspiration;TESE¼testicularspermextraction. t o
cptn¼patients;cyc.¼cycles. n 1
dStudydesign:pros.CH¼prospectivecohortstudy;ret.CH¼retrospectivecohortstudy. 0
eOutcome:preg.¼pregnancy;ong.preg.¼ongoingpregnancy. Ap
ril 2
0
1
9
2001).Athirdtechniqueforassessingthecalibrationisbasedonavisual performed, we evaluated the correspondence between the calculated
interpretationof the calibration figureplot (Fig. 2b).A calibration plot is probabilitiesandtheobservedpercentageofpregnanciesaftertheintro-
constructed by comparing the mean predicted probability (X-axis) with ductionofthepredictionmodels.
the observed proportion of pregnancies (Y-axis). For example, patients
can be allocated to one of 10 groups of equal size on the basis of the
Results
decilesofthecalculatedprobabilities.Foreachgroup,themeanpredicted
probability iscalculated, aswell asthe observedproportion iscalculated
Oursearchretrieved1082citationsfromMEDLINEandEMBASE,and
by Kaplan–Meier analysis. In case of perfect calibration, the prediction–
observation pairs are on the main diagonal and confidence intervals are nonefromtheCochraneLibrary.Theprocessofselectionofpapersis
not overlapping. Points below the diagonal represent overestimation of summarizedinFig.3.Weretrievedfourpapersfromcross-references.
theprobabilityofpregnancy,andpointsaboverepresentunderestimation After screening titles, abstracts and cross-references, we selected
(Custersetal.,2007;vanderSteegetal.,2007).Whenimpactanalysiswas 70 papers for further reading. Exclusion criteria are shown in Fig. 3.
544 Leushuisetal.
TableIV Overviewoftheparametersofthepredictionmodelsfortreatment-independentpregnancy(expressedasHRs
orORs)
D
o
w
n
lo
a
d
e
d
fro
m
h
ttp
s
://a
c
a
d
e
m
ic
.o
u
p
.c
o
m
/h
u
m
u
p
d
/a
rtic
le
-a
b
s
tra
c
t/1
5
/5
/5
3
7
/6
LR¼logisticregressionanalysis;CR¼Coxproportionalhazardregressionanalysis. 34
aPeryearofage(cid:4)31years;forafemaleage.31years,anHRof0.92hastobecalculatedforthenumberofyearsover31years,inadditiontotheHRfor(cid:4)31years. 48
bValidifdurationofsubfertility,24months. 8 b
cValidifdurationofsubfertility,36months. y
dValidiffemaleage(cid:4)30years. gu
eTotalmotility%combinedwithorwithoutqualityofmotility(value¼1whentotalmotility%(cid:4)20andmotilityquality,2;0otherwise). es
fSpermmorphology(cid:4)70%. t o
gValidifspermmotility.85%. n 1
hValidifspermmorphology,40or(cid:3)90%. 0 A
p
ril 2
0
A total of 36 papers were included in our critical appraisal. Some of treatment-independent pregnancy were generally subfertile 1
9
papers discussed an existing model rather than a newly derived couples, evaluated at a secondary or tertiary centre. Anovulation,
model and therefore the number of included models is lower than azoospermia and tubal pathology were the most common exclusion
thenumberofincludedpapers.Therewere12paperswhichreported criteria. The participants in the studies for the models of pregnancy
on the prediction of treatment-independent pregnancy. In these after IUI or IVF mostly concerned couples within their first cycle,
papers, nine different prediction models were described. The and in case of IVF, with or without assisted fertilization. A summary
4papersonmodelsforthepredictionofpregnancyafterIUIreported of the predictor variables and an estimate of the contribution made
on 3 different models, and the 20 papers for the prediction of by each parameter to the prediction for the different models are
pregnancyafterIVFaccounted for 17differentmodels. shown in Tables IV–VI.
The characteristics of the studies in these papers are summarized
Study quality
for the different interventions in Tables I–III. The majority of studies
were designed as a prospective cohort study. The inclusion criteria An overviewof the quality items per intervention is shownin Fig. 4a
for the patients in the studies on the models for the prediction and b. Patient selection was consecutive in 7 (78%) models for of
Predictionmodelsinreproductivemedicine 545
TableV OverviewoftheparametersofthepredictionmodelsforpregnancyafterIUI(expressedasHRsorORs)
D
o
w
n
lo
a
d
e
d
fro
m
h
ttp
s
://a
c
a
d
e
m
ic
.o
u
p
LR¼logisticregressionanalysis;CR¼Coxproportionalhazardregressionanalysis. .c
o
m
/h
u
the treatment-independent pregnancy and in 18 (90%) of the afterIVF(Stolwijketal.,1996;Templetonetal.,1996;Hunaultetal., m
u
treatment-dependent (IUIandIVF)models.Datacollectionwaspro- 2002a).Theonlymodelthatreachedthephaseofimpactanalysiswas p
d
smpoedcteilvseoinn8tre(4at0m%e)not-finthdeeptreenadtmenetnpt-rdeegnpaenncdye.nDteasncdriipnti6on(6o7f%th)eofvathrie- ttrheeatmmenotd-ienldepoefndeHntunpareugltnanecty. al. for the prediction of /article
ables for treatment was sufficient in 15 models for pregnancy after -ab
s
treatment (75%) and in 5 (56%) for the treatment-independent Model performance tra
c
mnuomdbeelsr.sThoefdsetsucdriiepst.ionMoisfsipnrgegonrancimypwuatsatgioivnenoinf amlmisossintgcodmaptaarawbales The performance of the eight models that were externally validated t/15/5
reported for only a few models. Of all models for (Eimersetal.,1994;Collinsetal.,1995;Stolwijketal.,1996;Temple- /5
3
tonetal.,1996;Snicketal.,1997;Hunaultetal.,2002b;Hunaultetal., 7
treatment-independent pregnancy, seven (78%) stated that they /6
2004; Steures et al., 2004) is presented in Table VII. One model for 3
usedCoxproportionalhazardsanalysis,butonlytwo(22%)described 4
4
the prediction of treatment-independent pregnancy (Hunault et al., 8
censoring. The amount of interventions between basic fertility 8
2004) had a poor discrimination (AUC 0.59), but good calibration. b
work-up and time to pregnancy varied substantially between these y
models; the basic fertility work-up was clearly described in 67% of Theothermodels forthepredictionof treatment-independentpreg- gue
nancy(Eimersetal.,1994;Collinsetal.,1995;Snicketal.,1997)also s
studies, and the follow-up duration was adequate in almost all of t o
had a poor discrimination (AUC ranging from 0.59 to 0.67) and did n
the studies. Of the treatment-dependent models, diagnosis before 1
not performwellatcalibration. 0
treatment wasdescribedin 14 (70%), and theprotocol of treatment A
TheoneexternallyvalidatedmodelforpregnancyafterIUI(Steures p
wasdescribedin 18 (90%). etal.,2004)hadpoordiscrimination(AUC0.59),butgoodcalibration; ril 2
0
itcoulddistinguishbetweenagroupwithpoorchancesofpregnancy 19
Phases of development
(0–5%) and a group with good chances of pregnancy (8–11%)
Thephasesofdevelopmentthatthepredictionmodelshadpassedare (Custers et al., 2007). Three models for the prediction of pregnancy
shown in Table VII. All models had passed development phase 1, as afterIVFhadbeenexternallyvalidated(Stolwijketal.,1996;Temple-
this was a criterion for inclusion in our review. Of the 29 models tonetal.,1996;Hunaultetal.,2002a).ThemodelofTempletonetal.
forpredictionofpregnancy,6modelshadbeenvalidatedonlyintern- had apoor discrimination with ac-statistic of0.63, butdifferentiated
ally, and only 8 other models had passed the phase of external vali- reliablybetweenwomenwithalowandarelativelyhighprobabilityof
dation. Onemodel had reached thephase ofimpactanalysis. successwithIVF(Smeenketal.,2000)andwasthereforetobecon-
Oftheeightexternallyvalidatedmodels,fourmodelsdealtwiththe sideredofgoodcalibration.ThemodelofStolwijketal.hadpoordis-
prediction of treatment-independent pregnancy (Eimers et al., 1994; crimination,withc-statisticsrangingfrom0.50to0.56.Calibrationwas
Collins et al., 1995; Snick et al., 1997; Hunault et al., 2004), one alsopoor,becausethemodelcouldnotidentifywomenwitha(very)
model dealt with the prediction of pregnancy after IUI (Steures lowprobabilityofongoingpregnancyafterIVF(Stolwijketal.,1998).In
etal.,2004)andthreemodelsdealtwiththepredictionofpregnancy the most recent validation of the model of Hunault et al. for the
5
4
6
TableVI OverviewoftheparametersofthepredictionmodelsforpregnancyafterIVF(expressedasHRsorORs)
D
o
w
n
lo
a
d
e
d
fro
m
h
ttp
s
://a
c
a
d
e
m
ic
.o
u
p
.c
o
m
/h
u
m
u
p
d
/a
rtic
le
-a
b
s
tra
c
t/1
5
/5
/5
3
7
/6
3
4
4
8
8
b
y
g
u
e
s
t o
n
1
0
A
p
ril 2
0
1
9
L
e
u
s
h
u
is
e
t
a
l.