Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study.

0.53 MB·English

by  Zhang, Zhongheng

Zhangetal.CriticalCare2013,17:R2
http://ccforum.com/content/17/1/R2
RESEARCH Open Access
Small studies may overestimate the effect sizes in
critical care meta-analyses: a meta-
epidemiological study
Zhongheng Zhang*, Xiao Xu and Hongying Ni
Abstract
Introduction: Small-study effects refer to the fact that trials with limited sample sizes are more likely to report
larger beneficial effects than large trials. However, this has never been investigated in critical care medicine. Thus,
the present study aimed to examine the presence and extent of small-study effects in critical care medicine.
Methods: Critical care meta-analyses involving randomized controlled trials and reported mortality as an outcome
measure were considered eligible for the study. Component trials were classified as large (≥100 patients per arm)
and small (<100 patients per arm) according to their sample sizes. Ratio of odds ratio (ROR) was calculated for
each meta-analysis and then RORs were combined using a meta-analytic approach. ROR<1 indicated larger
beneficial effect in small trials. Small and large trials were compared in methodological qualities including
sequence generating, blinding, allocation concealment, intention to treat and sample size calculation.
Results: A total of 27 critical care meta-analyses involving 317 trials were included. Of them, five meta-analyses
showed statistically significant RORs <1, and other meta-analyses did not reach a statistical significance. Overall, the
pooled ROR was 0.60 (95% CI: 0.53 to 0.68); the heterogeneity was moderate with an I2 of 50.3% (chi-squared =
52.30; P = 0.002). Large trials showed significantly better reporting quality than small trials in terms of sequence
generating, allocation concealment, blinding, intention to treat, sample size calculation and incomplete follow-up
data.
Conclusions: Small trials are more likely to report larger beneficial effects than large trials in critical care medicine,
which could be partly explained by the lower methodological quality in small trials. Caution should be practiced in
the interpretation of meta-analyses involving small trials.
Introduction topoftheplot,makingittheshapeofaninvertedfunnel
Small-studyeffectsrefertothepatternthatsmallstudies plot. If a funnel plot appears asymmetrical, publication
aremorelikelytoreportbeneficialeffectintheinterven- biasisassumedtobepresent.
tion arm, which was first described by Sterne et al. [1]. Incriticalcaremedicine,studiesareconductedininten-
Thiseffectcanbeexplained,atleastpartly,bythecombi- sivecareunits(ICU)wherethenumberofbedsislimited.
nation of lower methodological quality of small studies Duetothe natureofpopulation andthe care setting,the
and publication bias [2,3]. Typically, such small-study studiesincriticalcarefrequentlyhaveasmallsamplesize.
effects can be evaluated by funnel plot. Funnel plot Meta-analysis is considered to be an important tool to
depicts the effect size against the precision of the effect combinetheeffectsizesofsmalltrials,allowingmoresta-
size. Small studies with effect sizes of wider standard tistical power to detect the beneficial effects of a new
deviationsshouldwidely andsymmetricallydistributeat intervention.However,accordingtometa-epidemiological
thebottomoftheplot,andlargestudiesshouldclusterat studies conducted in other biomedical fields, interpreta-
tion of meta-analyses of small trials should be cautious,
andsuch meta-analyses may overestimate the true effect
*Correspondence:[email protected] of an intervention [3,4]. Small-study effect has been
DepartmentofCriticalCareMedicine,JinhuaMunicipalCentralHospital,351
observed when examining meta-analysis with binary [3]
MingyueStreet,JinhuaCity,Zhejiang321004,PRChina
©2013Zhangetal.;licenseeBioMedCentralLtd.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommons
AttributionLicense(http://creativecommons.org/licenses/by/2.0),whichpermitsunrestricteduse,distribution,andreproductionin
anymedium,providedtheoriginalworkisproperlycited.
Zhangetal.CriticalCare2013,17:R2 Page2of9
http://ccforum.com/content/17/1/R2
and continuous outcomes [4]. In critical care medicine, sample size of 200 patients allowed an 80% statistical
small-studyeffectshaveneverbeenquantitativelyassessed. power to detect an absolute difference of 20% for binary
Thus,weconductedthissystematicreviewofcriticalcare outcomes (assuming that the proportion in the control
meta-analysesinanattempttoexaminethepresenceand group was 50%) at two-sideda = 0.05. Another reason
extentofsmall-studyeffectsincriticalcaremedicine. for this definition was that critical care trials were
usually small, and a greater cutoff point would signifi-
Materials and methods cantly reduce the number of meta-analyses that were
Search strategy and study selection eligible for the analysis.
Medline and Embase databases were searched from
inception to August 2012. There was no language Statistical analysis
restriction. The core search terms consisted of critical Treatment effectswere expressed asodds ratio(OR)for
care, mortality and meta-analysis (detailed search strat- mortality. The number of events and total number of
egy is shown in Additional file 1). Inclusion criteria patientsineacharmwereextractedforeachcomponent
were as follows: critical care meta-analyses involving trial. AnOR <1 indicated beneficial effect in the experi-
randomized controlled trial; the end points should mental arm. A standard logistic regression model was
include mortality; at least one component trial had usedtoexaminewhetherestimatedtreatmenteffectsdif-
more than 100 subjects per arm on average. Exclusion fer according to whether a trial is large or small [6,7].
criteria were systematic reviews without meta-analysis; Ratio of OR (ROR) was estimated from the regression
all component trials were exclusively large (sample sizes model.ROR<1indicateslargereffectsizeinsmallerstu-
≥100 per arm) or small trials (sample sizes <100 per diesandROR>1indicateslargereffectsizeinlargetrials.
arm); meta-analyses included duplicated component ROR was calculated separately for each meta-analysis.
trials. If there were several meta-analyses addressing the These RORs were then combined using a meta-analytic
same clinical issue, we included the most updated one. approach. Inverse variance weighting and either fixed
Two reviewers (XX and ZZ) independently assessed the effect or random effects models were used to pool these
literature and disagreement was settled by a third opi- RORs.Meta-analysesinvolvingexclusivelylargeorsmall
nion (HN). trialswere notincludedinouranalysisand thusdid not
contribute to the analysis. Heterogeneity between trials
Data extraction wasestimatedusingI2. A roughguide to the interpreta-
The following data were extracted from eligible meta- tionofI2canbeasfollows:0to40%representsunimpor-
analyses:theleadauthorofthestudy,yearofpublication, tant heterogeneity; 30% to 60% represents moderate
number of trials, treatment strategy in the experimental heterogeneity;50%to90%representssubstantialhetero-
arm, proportion of large trials in each meta-analysis, geneityand75%to100%representsconsiderablehetero-
effect size and corresponding 95% confidence interval geneity [8]. To account for the difference of estimated
(CI),heterogeneityasrepresentedbyI2.Foreachcompo- effects between large and small trials, the qualities of
nenttrial,weextractedthefollowingdata:sequencegen- study reportingincludingsequencegenerating,blinding,
erating, allocation concealment, blinding, incomplete allocationconcealment,incompletefollow-updata,sam-
follow-up data, intention-to-treat analysis, sample size plesizecalculationandintentiontotreatwerecompared
calculation,andyearofpublication.Sequencegenerating between large and small trials. The proportions of large
was considered adequate when the trial reported the andsmalltrialswerecomparedbasedontheyearofpub-
method to generate the randomization sequence (for licationbefore and after 2002. This was defined arbitra-
examplecomputer,randomizationtable).Allocationcon- rily. However, we feel that multicenter large trials have
cealmentwasconsideredadequatewhentheinvestigator increased rapidly in the last 10 years. We analyzed the
responsible for patient selection was unable to predict association between sample size and treatment effects,
allocationofthenextpatient.Thecommonlyusedtech- stratified according to the significance of effect size and
niques included the use of central randomization or heterogeneity within each meta-analysis. All statistical
sequentially numbered, opaque and sealed envelopes. analyses were performed using Stata software version
Blinding was considered adequate if the experimental 11.0(StataCorpLP,CollegeStation,TX,USA).Statistical
and control interventions were described as indistin- significancewasconsideredattwo-sidedP<0.05.
guishablebypatientsorinvestigators[5].
Small and large trials were distinguished by a cutoff of Results
an average of 100 subjects per arm. For example, if a Study selection and characteristics
two-arm trial had 113 patients in one arm and 87 Our initial search identified 371 citations. Of them, 329
patients in the other, it was considered a large trial. were excluded by reviewing the title and abstract
This definition was somewhat arbitrary. However, a because they were duplicate meta-analyses, included
Zhangetal.CriticalCare2013,17:R2 Page3of9
http://ccforum.com/content/17/1/R2
non-randomized trials, did not report data on mortality, only counted those that reported mortality as an end
and other irrelevant articles. Full text of the remaining point. Of note, because one meta-analysis [34] included
42 citations was reviewed, of which 15 citations were seven trials conducted by Joachim Boldt that had been
excluded. In these excluded 15 citations, eight did not retracted [36], Boldt’strialswere excluded fromanalysis.
include large trials, study end point was not mortality Eight meta-analyses [10,13,17,18,25,28,31,34] included
in four meta-analyses, and three were duplicated meta- onlyonelargetrial,andthemeta-analysesbyZhongheng
analyses. A total of 27 meta-analyses [9-35] involving [35] included mostly large trials (83.3%). Most meta-
randomized controlled trials were finally included in our analysesreportednon-significant effectsize,andonlysix
analysis (Figure 1). meta-analyses [19,20,22,25,30,31] reported statistically
Characteristics of included meta-analyses are shownin significanteffectsizes(forexamplemortality).Eightmeta-
Table 1. All meta-analyses were published after the year analyses[12,15,16,20,21,30,32,35]reportedhighheteroge-
2005. These meta-analyses covered all subspecialties of neity and the remaining 19 meta-analyses showed no
critical care medicine, including mechanical ventilation, significantheterogeneityamongcomponenttrials.Seven-
sedation, fluid resuscitation, prevention of nosocomial teenmeta-analyses[9-11,14,17,18,22-29,31,33,34]reported
infection, nutrition and critical care nephrology. For the an I2 of 0%, most of which were meta-analyses without
number of component trials in each meta-analysis, we significanteffectsizes.
Figure1Flowchartshowingtheselectionofstudies.
hZ
ttpha
://cng
c e
forum.talC
Table1Characteristics ofincluded meta-analyses. .com ritica
Intervention Nstou.doiefsII Egvroeunptr(a%te) incontrol L(na,rg%e)trials E(9ff5e%ctCsIi)zeofoddsratio Heterogeneity(I2) /conte lCare
n2
AAbfsrhoaurgiAF22000171[[190]] PArnotnitehrvoemnbtiliantiIoIIn 270 3397..93 13((54)2.9) 00..9961((00..8795,,11..013)) 00%% t/17/1013,1
AfshariA2011[11] InhalednitricoxideforARDS 14 38.6 3(21.4) 1.06(0.93,1.22) 0% /R27:R2
AnnaneD2009[12] Corticosteroids 20 36.4 4(20) 0.87(0.74,1.01) 44%
AugustesR2011[13] Sedationinterruption 5 37.2 1(20) 0.84(0.58,1.21) 19.4%
BarkunAN2012[14] PPIforstress-relatedmucosalbleeding 8 18.7 3(37.5) 1.19(0.84,1.68) 0%
BlackwoodB2011[15] Weaningprotocol 10 14.7 5(50) 0.98(0.48,2.02) 57%
BurnsKE2011[16] Volume-limitedventilation 10 42.5 2(20) 0.84(0.7,1.0) 43.1%
DelaneyAP2011[17] Albuminforresuscitation 17 33.8 1(5.9) 0.82(0.67,1.0) 0%
KopteridesP2010[18] Procalcitonin-guided 6 22.6 1(16.7) 0.93(0.69,1.26) 0%
LandoniG2010[19] Levosimendan 27 22.4 3(11.1) 0.74(0.62,0.89) 11.3%
LauplandKB2007[20] Polyclonalintravenousimmunoglobulin 14 41.8 2(14.3) 0.66(0.53,0.83) 53.8%
MarikPE2008[21] Immunonutrition 24 25.8 5(20.8) 0.85(0.64,1.13) 44%
PhoenixSI2009[22] HighPEEP 6 38.5 3(50) 0.87(0.78,0.96) 0%
PileggiC2011[23] Selectivedecontaminationofthedigestiveor 11 NR 6(54.5) 1.1(0.98,1.24) 0%
respiratorytract
PuskarichMA2009[24] Glucose-insulin-potassiuminfusion 23 9 5(21.7) 1.02(0.93,1.11) 0%
SerpaNetoA2012[25] Vasopressin/terlipressin 9 50.3 1(11.1) 0.87(0.79,0.96) 0%
ShahMR2005[26] Pulmonaryarterycatheter 13 32.7 4(30.8) 1.04(0.9,1.2) 0%
ShanL2011[27] Intensiveinsulintherapy 5 34.7 2(40) 0.98(0.82,1.16) 0%
SiemposII2010[28] Probiotics 4 19.5 1(25) 0.75(0.47,1.21) 0%
TanJA2010[29] Dexmedetomidine 16 8.7 3(18.8) 0.85(0.64,1.17) 0%
VasuTS2012[30] Dopamine/norepinephrine 6 54.0 2(33.3) 0.43(0.26,0.69) 65.4%
VisserJ2011[31] Micronutrientsupplementation 10 28.8 1(10) 0.78(0.67,0.90) 0%
WangF2011[32] Timingoftracheotomy 7 33.0 2(28.6) 0.86(0.65,1.13) 45%
WangF2012[33] Subglotticsecretiondrainage 9 22.1 3(33.3) 0.97(0.84,1.12) 0%
ZarychanskiR2009[34] Hydroxyethylstarchresuscitation 10 31.1 1(10) 1.07(0.85,1.34) 0%
ZhonghengZ2010[35] DoseofCRRT 6 45.4 5(83.3) 0.91(0.77,1.08) 75%
IIOnlystudiesthatreportedmortalitywereincluded.
P
a
g
e
4
o
f
9
Zhangetal.CriticalCare2013,17:R2 Page5of9
http://ccforum.com/content/17/1/R2
RORs were estimated by logistic regression model = 52.30; P = 0.002). Meta-regression was performed to
separately for each meta-analysis (Figure 2). The RORs test whether the observed RORs were dependent on the
and relevant 95% CI are shown in the left column of quality of meta-analyses. Covariates including conceal-
Figure 2. Five meta-analyses [12,19-22] showed statisti- ment (P = 0.88), blinding (P = 0.82), intention to treat
cally significant RORs, indicating significantly larger (P = 0.72), sequence generating (P = 0.48) and sample
beneficial effects in small studies; nine meta-analyses size calculation (P = 0.57) cannot explain the heteroge-
[10,11,13,15,16,26,27,31,33] showed RORs >1, but with- neity between meta-analyses.
out statistical significance; the remaining 13 meta-ana- To explore possible explanations for the difference of
lyses showed RORs <1, again without statistical effect sizes between large and small trials, we compared
significance. RORs were combined using a meta-analytic reporting qualitiesoflarge andsmalltrials(Table 2).As
approach with inverse variance weighting. A fixed effect expected, the large trials showed significantly better
model was used to combine the RORs. Overall, the reportingqualitythansmalltrials.Morelargetrialswere
pooled ROR was 0.60 (95% CI: 0.53 to 0.68); the hetero- well conducted than small trials in terms of sequence
geneity was moderate with an I2 of 50.3% (chi-squared generating,allocationconcealment,blinding,intentionto
Figure2Forestplotshowingtheratioofoddsratiobetweensmallandlargetrials.
Zhangetal.CriticalCare2013,17:R2 Page6of9
http://ccforum.com/content/17/1/R2
Table 2Comparison ofqualities between small andlarge trials in meta-analyses in critical caremedicine.
No.ofpatientsincomponenttrials Pvalue
Largetrials(>100perarm,n=73) Smalltrials(<100perarm,n=244)
Sequencegenerating 0.031
Adequate 29(59%) 50(41%)
Inadequate/unclear 20(41%) 72(59%)
Allocationconcealment <0.001
Adequate 56(82%) 114(51%)
Inadequate/unclear 12(18%) 111(49%)
Blinding <0.001
Adequate 49(67%) 99(41%)
Inadequate/unclear 24(33%) 145(59%)
Incompletefollow-updataaddressed <0.001
Adequate 36(86%) 56(48%)
Inadequate/unclear 6(14%) 61(52%)
Intentiontotreat <0.001
Yes 38(83%) 79(52%)
No/unclear 8(17%) 72(48%)
Samplesizecalculation <0.001
Yes 29(88%) 32(44%)
No/unclear 4(12%) 40(56%)
Yearofpublication <0.001
Before2002 55(75%) 127(52%)
After2002 18(25%) 117(48%)
treat, sample size calculation and incomplete follow-up thepooledRORwas0.69(95%:0.59to0.79).Stratifiedby
data. For instance, 82% oflarge trials explicitly reported heterogeneity, eight meta-analyses [12,15,16,20,21,30,
allocation concealment, while only 51% of small trials 32,35] reported highheterogeneity,theircombinedROR
reported this (P <0.001). Intention-to-treat analysis was was0.46(95%:0.36to 0.55);theremaining19meta-ana-
used in 83% of the large trials, while only 52% of the lyses showed a ROR of 0.79 (95% CI: 0.68 to 0.90). The
smalltrialsusedthisanalysis(P<0.001).Samplesizecal- resultindicatedthatsmall-studyeffectsweremorepromi-
culationwasperformedaprioriin88%oflargetrialsand nentinmeta-analysesofhighheterogeneity.
only 44% ofsmall trialsreportedsample size calculation
(P<0.001).Some75%oflargetrialswerepublishedafter Discussion
the year 2002, while 52% of small trials were published Thisisthefirstmeta-epidemiologicalstudyconductedin
after2002(P<0.001). thefieldofcriticalcare medicine todemonstratesmaller
WeestimatedRORbetweenlargeandsmalltrialsstrati- trialsmayoverestimatetreatmenteffectsize.Inthisstudy,
fied according to the characteristics of individual meta- we included 27 meta-analyses of 317 randomized con-
analysis (Table 3). Eight meta-analyses [16,17,19,20, trolled trials covering all subspecialties of critical care
22,25,30,31] reported significant effect sizes, and the medicine.Theresultsshowedthatsmalltrialsweremore
pooled ROR was0.49 (95%:0.38to0.60).The remaining likely to report larger estimated treatment effects com-
19 meta-analyses reported insignificant effect sizes, and pared with large trials, and this was more prominent in
Table 3Ratioofodds ratiobetween large andsmall trials stratified by characteristics ofmeta-analyses.
Comparison No.ofmeta-analyses No.oftrials Ratioofoddsratio(95%CI) Heterogeneity(I2)
Overall 27 317 0.60(0.53-0.68) 50.3%
Significanceofeffectsize
Significant 8 99 0.49(0.38-0.60) 62.9%
Non-significant 19 218 0.69(0.59-0.79) 28.8%
Heterogeneity
Low 19 220 0.79(0.68-0.90) 0%
High 8 97 0.46(0.36-0.55) 57.2%
Zhangetal.CriticalCare2013,17:R2 Page7of9
http://ccforum.com/content/17/1/R2
meta-analysesinvolvinghighlyheterogeneouscomponent analysisfocusedonthefieldofcriticalcaremedicineand
trials.Furthermore,thesmalltrials wereoflowqualityin showed,forthefirsttime,apossibleassociationofmetho-
methodologycomparedwithlargetrials,whichmaypartly dologicalqualitywitheffectsizes.
accountforthesmall-studyeffects. Thereareseverallimitationsinthepresentstudy.First,
Inameta-epidemiologicalstudyinosteoarthritis[4],the ourstudyaimstoinvestigatethesmall-studyeffectincri-
authors employed the difference in effect sizes between ticalcaremedicine.However,criticalcareisanextremely
largeandsmalltrialstoexplorethesmall-studyeffects.In heterogeneous subspecialty that involves all varieties of
line with the present study, they concluded that small diseases. Such heterogeneous nature may potentially
trialsweremorelikelytoreportlargerbeneficialtreatment impair the quality of the meta-epidemiological study.
effectsthanlargetrials.However,inthatstudy,thesmall Second, the heterogeneity cannot be fully accounted for
trials were not statistically different from those of large inthepresentanalysis.Wetriedtoexplorethesourcesof
trialsintermsofblinding,intention-to-treatanalysis,thus, heterogeneity by incorporating factors related to the
thesmall-studyeffectscannotbefullyexplainedbymeth- quality of study design in meta-regression model, but
odologicalquality.Inosteoarthritistrialsblindingisprob- failedtoidentifyasignificantcovariate.Weproposethat
ably much more easily achieved because drugs can be theexplainablefactorsmaybethosethatcannotberead-
madeindistinguishableinappearance.Thus,smallstudies, ilyaccessible.Studieswithnegativeresultsarelesslikely
usually with limited financial support, can also achieve to be published than studies with positive results, parti-
good methodological quality. In contrast, in critical care cularly if such studies are small in sample size. As a
medicine, blinding is sometimes impossible or complex result,smallstudieswithnegativeresultarelesslikelyto
due to the nature of intervention. Such interventions be accepted into publication. If this is the case, it is not
includedpulmonaryarterycatheter,intensityofcontinu- surprising that small studies are more likely to report
ousrenalreplacementtherapy,proneventilation,andsub- beneficial effect. However, such kind of publication bias
glottic secretion drainage. In these situations, blinding cannot be systematically investigated. Another explana-
may be difficult to achieve or only large trials with more tionforthesmall-studyeffectmaybethatmorerigorous
planningandmethodologicalsupportcanmakethispossi- implementationofinterventionsisperformedin smaller
ble.Thus,smalltrialsincriticalcaremedicineareoflim- studies.
itedqualityindesigncomparedwithlargetrials.
Possible explanations for the small-study effects have Conclusions
beenexplored.Kjaergardandcolleagues[3]demonstrated In conclusion, our study included 27 critical care meta-
thatsmallstudieswithlowerqualitysignificantlyexagger- analysesinvolvingallsubspecialtiesinthefieldofcritical
atedtheinterventioneffectcomparedtolargetrials,while care medicine. The result showed that small studies
smalltrialswithadequatesequencegenerating,allocation tendedtoreportlargerbeneficialeffectsthanlargetrials.
concealmentandblindingdidnotdifferfromlarge trials. Interpretationof meta-analyses of small trials should be
Thisisconsistentwithourfindingsthatsmallstudieshad cautiousandsometimesdefinitiveconclusionscannotbe
lower methodological quality compared withlarge trials. madeuntilalargemulticentertrialisconducted.
However, the impact of methodological quality such as
allocation concealment and blinding varies according to Key messages
different outcomes. A meta-epidemiological study invol- (cid:129) Interpretation of critical care meta-analyses involving
ving 146 meta-analyses demonstrated that in trials with small studies should be cautious due to the small-study
subjectiveoutcomestheestimatedeffectsizeswereexag- effects.
geratedwhentherewasinadequateconcealmentorblind- (cid:129) Small-study effects may be attributable to the poor
ing, while in trials with objective outcomes such as methodological quality of the small studies.
mortality, there was little evidence that inadequate con-
cealmentandlackofblindingwoulddistorttheestimated Additional material
effect sizes [37]. This is in contrast with our findings,
becauseweusedmortalityasanendpoint,buttheresult Additionalfile1:Searchstrategy.Theadditionalfileshowsthe
indicated that lackof blinding and inadequate allocation detailedsearchstrategyperformedinourstudy.
concealment might contribute to the exaggerated effect
sizesinsmalltrials.However,thisisanunsettledquestion
andthereareotherstudiessupportingourfinding[38,39]. Abbreviations
Mostprobably,individualqualitymeasuressuchasblind- CI:confidenceinterval;ICU:intensivecareunit;ROR:ratioofoddsratio.
ingandallocationconcealmentarenotconsistentlyasso-
Authors’contributions
ciated with the effect sizes across study areas, and each ZZconceivedtheidea,collecteddataanddraftedthemanuscript;XX
medicalarea should bespecificallyinvestigated[40].Our helpedcollectdataandanalyses;HNhelpedcollectdata,andanalyzeand
Zhangetal.CriticalCare2013,17:R2 Page8of9
http://ccforum.com/content/17/1/R2
interprettheresults.Allauthorshavereadandapprovedthemanuscriptfor 19. LandoniG,MizziA,Biondi-ZoccaiG,BignamiE,PratiP,AjelloV,MarinoG,
publication. GuarracinoF,ZangrilloA:Levosimendanreducesmortalityincriticallyill
patients.Ameta-analysisofrandomizedcontrolledstudies.Minerva
Competinginterests Anestesiol2010,76:276-286.
Theauthorsdeclarethattheyhavenocompetinginterests. 20. LauplandKB,KirkpatrickAW,DelaneyA:Polyclonalintravenous
immunoglobulinforthetreatmentofseveresepsisandsepticshockin
Received:29October2012 Revised:18December2012 criticallyilladults:asystematicreviewandmeta-analysis.CritCareMed
Accepted:7January2013 Published:9January2013 2007,35:2686-2692.
21. MarikPE,ZalogaGP:Immunonutritionincriticallyillpatients:a
References systematicreviewandanalysisoftheliterature.IntensiveCareMed2008,
1. SterneJA,EggerM:Funnelplotsfordetectingbiasinmeta-analysis: 34:1980-1990.
guidelinesonchoiceofaxis.JClinEpidemiol2001,54:1046-1055. 22. PhoenixSI,ParavastuS,ColumbM,VincentJL,NirmalanM:Doesahigher
2. ChanAW,AltmanDG:Identifyingoutcomereportingbiasinrandomised positiveendexpiratorypressuredecreasemortalityinacuterespiratory
trialsonPubMed:reviewofpublicationsandsurveyofauthors.BMJ distresssyndrome?Asystematicreviewandmeta-analysis.Anesthesiology
2005,330:753. 2009,110:1098-1105.
3. KjaergardLL,VillumsenJ,GluudC:Reportedmethodologicqualityand 23. PileggiC,BiancoA,FlottaD,NobileCG,PaviaM:Preventionofventilator-
discrepanciesbetweenlargeandsmallrandomizedtrialsinmeta- associatedpneumonia,mortalityandallintensivecareunitacquired
analyses.AnnInternMed2001,135:982-989. infectionsbytopicallyappliedantimicrobialorantisepticagents:a
4. NüeschE,TrelleS,ReichenbachS,RutjesAW,TschannenB,AltmanDG, meta-analysisofrandomizedcontrolledtrialsinintensivecareunits.Crit
EggerM,JüniP:Smallstudyeffectsinmeta-analysesofosteoarthritis Care2011,15:R155.
trials:meta-epidemiologicalstudy.BMJ2010,341:c3515. 24. PuskarichMA,RunyonMS,TrzeciakS,KlineJA,JonesAE:Effectofglucose-
5. SchulzKF,AltmanDG,MoherD,CONSORTGroup:CONSORT2010 insulin-potassiuminfusiononmortalityincriticalcaresettings:a
statement:updatedguidelinesforreportingparallelgrouprandomised systematicreviewandmeta-analysis.JClinPharmacol2009,49:758-767.
trials.BMJ2010,340:c332. 25. SerpaNetoA,NassarJuniorAP,CardosoSO,ManetttaJA,PereiraVG,
6. SterneJA,JüniP,SchulzKF,AltmanDG,BartlettC,EggerM:Statistical EspositoDC,DamascenoMC,RussellJA:Vasopressinandterlipressinin
methodsforassessingtheinfluenceofstudycharacteristicson adultvasodilatoryshock:asystematicreviewandmeta-analysisofnine
treatmenteffectsin‘meta-epidemiological’research.StatMed2002, randomizedcontrolledtrials.CritCare2012,16:R154.
21:1513-1524. 26. ShahMR,HasselbladV,StevensonLW,BinanayC,O’ConnorCM,SopkoG,
7. SiersmaV,Als-NielsenB,ChenW,HildenJ,GluudLL,GluudC:Multivariable CaliffRM:Impactofthepulmonaryarterycatheterincriticallyill
modellingformeta-epidemiologicalassessmentoftheassociation patients:meta-analysisofrandomizedclinicaltrials.JAMA2005,
betweentrialqualityandtreatmenteffectsestimatedinrandomized 294:1664-1670.
clinicaltrials.StatMed2007,26:2745-2758. 27. ShanL,HaoPP,ChenYG:Efficacyandsafetyofintensiveinsulintherapy
8. HigginsJPT,GreenS,(editors):CochraneHandbookforSystematic forcriticallyillneurologicpatients:ameta-analysis.JTrauma2011,
ReviewsofInterventionsVersion5.1.0(updatedMarch2011).The 71:1460-1464.
CochraneCollaboration2011[http://www.cochrane-handbook.org]. 28. SiemposII,NtaidouTK,FalagasME:Impactoftheadministrationof
9. AbrougF,Ouanes-BesbesL,DachraouiF,OuanesI,BrochardL:Anupdated probioticsontheincidenceofventilator-associatedpneumonia:ameta-
study-levelmeta-analysisofrandomisedcontrolledtrialsonproningin analysisofrandomizedcontrolledtrials.CritCareMed2010,38:954-962.
ARDSandacutelunginjury.CritCare2011,15:R6. 29. TanJA,HoKM:Useofdexmedetomidineasasedativeandanalgesic
10. AfshariA,WetterslevJ,BrokJ,MøllerA:AntithrombinIIIincriticallyill agentincriticallyilladultpatients:ameta-analysis.IntensiveCareMed
patients:systematicreviewwithmeta-analysisandtrialsequential 2010,36:926-939.
analysis.BMJ2007,335:1248-1251. 30. VasuTS,CavallazziR,HiraniA,KaplanG,LeibyB,MarikPE:Norepinephrine
11. AfshariA,BrokJ,MøllerAM,WetterslevJ:Inhalednitricoxideforacute ordopamineforsepticshock:systematicreviewofrandomizedclinical
respiratorydistresssyndromeandacutelunginjuryinadultsand trials.JIntensiveCareMed2012,27:172-178.
children:asystematicreviewwithmeta-analysisandtrialsequential 31. VisserJ,LabadariosD,BlaauwR:Micronutrientsupplementationfor
analysis.AnesthAnalg2011,112:1411-1421. criticallyilladults:asystematicreviewandmeta-analysis.Nutrition2011,
12. AnnaneD,BellissantE,BollaertPE,BriegelJ,ConfalonieriM,DeGaudioR, 27:745-758.
KehD,KupferY,OppertM,MeduriGU:Corticosteroidsinthetreatmentof 32. WangF,WuY,BoL,LouJ,ZhuJ,ChenF,LiJ,DengX:Thetimingof
severesepsisandsepticshockinadults:asystematicreview.JAMA2009, tracheotomyincriticallyillpatientsundergoingmechanicalventilation:
301:2362-2375. asystematicreviewandmeta-analysisofrandomizedcontrolledtrials.
13. AugustesR,HoKM:Meta-analysisofrandomisedcontrolledtrialson Chest2011,140:1456-1465.
dailysedationinterruptionforcriticallyilladultpatients.AnaesthIntensive 33. WangF,BoL,TangL,LouJ,WuY,ChenF,LiJ,DengX:Subglottic
Care2011,39:401-409. secretiondrainageforpreventingventilator-associatedpneumonia:an
14. BarkunAN,BardouM,PhamCQ,MartelM:Protonpumpinhibitorsvs. updatedmeta-analysisofrandomizedcontrolledtrials.JTraumaAcute
histamine2receptorantagonistsforstress-relatedmucosalbleeding CareSurg2012,72:1276-1285.
prophylaxisincriticallyillpatients:ameta-analysis.AmJGastroenterol 34. ZarychanskiR,TurgeonAF,FergussonDA,CookDJ,HébertP,BagshawSM,
2012,107:507-520. MonsourD,McIntyreL:Renaloutcomesandmortalityfollowing
15. BlackwoodB,AlderdiceF,BurnsK,CardwellC,LaveryG,O’HalloranP:Use hydroxyethylstarchresuscitationofcriticallyillpatients:systematic
ofweaningprotocolsforreducingdurationofmechanicalventilationin reviewandmeta-analysisofrandomizedtrials.OpenMed2009,3:
criticallyilladultpatients:Cochranesystematicreviewandmeta- e196-209.
analysis.BMJ2011,342:c7237. 35. ZhonghengZ,XiaoX,HongyangZ:Intensive-vsless-intensive-dose
16. BurnsKE,AdhikariNK,SlutskyAS,GuyattGH,VillarJ,ZhangH,ZhouQ, continuousrenalreplacementtherapyfortheintensivecareunit-related
CookDJ,StewartTE,MeadeMO:Pressureandvolumelimitedventilation acutekidneyinjury:ameta-analysisandsystematicreview.JCritCare
fortheventilatorymanagementofpatientswithacutelunginjury:a 2010,25:595-600.
systematicreviewandmeta-analysis.PLoSOne2011,6:e14623. 36. ReillyC:Retraction.NoticeofformalretractionofarticlesbyDr.Joachim
17. DelaneyAP,DanA,McCaffreyJ,FinferS:Theroleofalbuminasa Boldt.BrJAnaesth2011,107:116-117.
resuscitationfluidforpatientswithsepsis:asystematicreviewand 37. WoodL,EggerM,GluudLL,SchulzKF,JüniP,AltmanDG,GluudC,
meta-analysis.CritCareMed2011,39:386-391. MartinRM,WoodAJ,SterneJA:Empiricalevidenceofbiasintreatment
18. KopteridesP,SiemposII,TsangarisI,TsantesA,ArmaganidisA: effectestimatesincontrolledtrialswithdifferentinterventionsand
Procalcitonin-guidedalgorithmsofantibiotictherapyintheintensive outcomes:meta-epidemiologicalstudy.BMJ2008,336:601-605.
careunit:asystematicreviewandmeta-analysisofrandomized 38. MoherD,PhamB,JonesA,CookDJ,JadadAR,MoherM,TugwellP,
controlledtrials.CritCareMed2010,38:2229-2241. KlassenTP:Doesqualityofreportsofrandomisedtrialsaffectestimates
Zhangetal.CriticalCare2013,17:R2 Page9of9
http://ccforum.com/content/17/1/R2
ofinterventionefficacyreportedinmeta-analyses?Lancet1998,
352:609-913.
39. EggerM,JuniP,BartlettC,HolensteinF,SterneJ:Howimportantare
comprehensiveliteraturesearchesandtheassessmentoftrialqualityin
systematicreviews?Empiricalstudy.HealthTechnolAssess2003,7:1-76.
40. BalkEM,BonisPA,MoskowitzH,SchmidCH,IoannidisJP,WangC,LauJ:
Correlationofqualitymeasureswithestimatesoftreatmenteffectin
meta-analysesofrandomizedcontrolledtrials.JAMA2002,287:2973-2982.
doi:10.1186/cc11919
Citethisarticleas:Zhangetal.:Smallstudiesmayoverestimatethe
effectsizesincriticalcaremeta-analyses:ameta-epidemiologicalstudy.
CriticalCare201317:R2.
Submit your next manuscript to BioMed Central
and take full advantage of:
• Convenient online submission
• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
Submit your manuscript at
www.biomedcentral.com/submit