Developing a Reporting Guideline for Social and Psychological Intervention Trials.

2013·0.14 MB·English

by  Grant, Sean

Invited Article
ResearchonSocialWorkPractice
23(6)595-602
Developing a Reporting Guideline for Social ªTheAuthor(s)2013
Reprintsandpermission:
and Psychological Intervention Trials sagepub.com/journalsPermissions.nav
DOI:10.1177/1049731513498118
rsw.sagepub.com
Sean Grant1, Paul Montgomery1, Sally Hopewell2,
Geraldine Macdonald3, David Moher4, and Evan Mayo-Wilson5
Abstract
Socialandpsychologicalinterventionsareoftencomplex.Understandingrandomizedcontrolledtrials(RCTs)ofthesecomplex
interventionsrequiresadetaileddescriptionoftheinterventionstestedandthemethodsusedtoevaluatethem;however,RCT
reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of
research,wastesresources,andfailstomeetethicalobligationstoresearchparticipantsandconsumers.Inthisarticle,weexplain
how reporting guidelines have improved the quality of reports in medicine and describe the ongoing development of a new
reportingguidelineforRCTs:ConsolidatedStandardsofReportingTrials-SPI(anextensionforsocialandpsychologicalinterven-
tions). We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed
consensus on theseguidelines(http://tinyurl.com/CONSORT-study).
Keywords
randomized controlled trial, RCT, CONSORT-SPI, reporting guideline, reporting standards
Introduction andcommunitysettings.Otherexamplesofsocialandpsycholo-
gical interventions may be found in reviews by the Cochrane
Socialandpsychologicalinterventionsaimtoimprovephysical
Collaboration(2013;e.g.,theDevelopmental,Psychosocial,and
health,mentalhealth,andassociatedsocialoutcomes.Theyare
Learning Problems Group; the Cochrane Public Health Group)
oftencomplexandtypicallyinvolvemultiple,interactinginter-
andtheCampbellCollaboration(2013).
ventioncomponents(e.g.,severalbehaviorchangetechniques)
Tounderstandtheireffectsandtokeepservicesuptodate,
that may act and target outcomes on several levels (e.g., indi-
academics, policy makers, journalists, clinicians, and consu-
vidual, family, and community; Medical Research Council
mers rely on research reports of intervention studies in scien-
[MRC], 2008). Moreover, these interventions may be contex-
tific journals. Such reports should explain the methods,
tually dependent upon the hard-to-control environments in
includingthedesign,delivery,uptake,andcontextofinterven-
whichtheyaredelivered(e.g.,healthcaresettingsandcorrec-
tions, as well as subsequent results. Accurate, complete, and
tionalfacilities;Bonell,2002;Pawson,Greenhalgh,Harvey,&
transparent reporting is essential for readers to make best use
Walshe,2004).Thefunctionsandprocessesoftheseinterven-
of new evidence, to achieve returns on research investment,
tions may be designed to accommodate particular individuals
tomeet ethical obligationstoresearch participants andconsu-
or contexts, taking on different forms while still aiming to
mers of interventions, and to minimize waste in research.
achievethesameobjective(Bonell,Fletcher,Morton,Lorenc,
&Moore, 2012;Hawe, Shiell,&Riley, 2004).
Complex interventions are common in public health, 1CentreforEvidence-BasedIntervention,UniversityofOxford,Oxford,UK
psychology, education, social work, criminology, and related 2CentreforStatisticsinMedicine,UniversityofOxford,Oxford,UK
disciplines. For example, multisystemic therapy (MST) is an 3InstituteofChildCareResearch,Queen’sUniversityBelfast,Belfast,UK
intensive intervention for juvenile offenders. Based on social 4ClinicalEpidemiologyProgram,OttawaHospitalResearchInstitute,Centre
forPractice-ChangingResearch(CPCR),TheOttawaHospital,Ottawa,ON,
ecological and family system theories, MST providers target a
Canada
variety of individual, family, school, peer, neighborhood, and 5ResearchDepartmentofClinical,Educational&HealthPsychology,Centre
communityinfluencesonpsychosocialandbehavioralproblems for Outcomes Research and Effectiveness, University College London,
(Henggeler, Schoenwald, Rowland, & Cunningham, 2002). London,UK
Treatment teams of professional therapists and caseworkers
CorrespondingAuthor:
work with individuals, their families, and their peer groups to
Sean Grant, Centre for Evidence-Based Intervention, University of Oxford,
provide tailored services (Littell, Campbell, Green, & Toews,
BarnettHouse,32WellingtonSquare,OxfordOX12ER,UK.
2009). These services may be delivered in homes, social care, Email:[email protected]
596 Research on Social Work Practice 23(6)
However, reports of randomized controlled trials (RCTs) are 2012). It has been endorsed by over 600 journals (Moher,
often poorly reported within and across disciplines including Altman,Schulz,&Elbourne,2004),anditissupportedbythe
criminology (Perry, Weisburd, & Hewitt, 2010), social work Institute of Educational Sciences (Torgerson et al., 2005).
(Naleppa & Cagle, 2010), education (Torgerson, Torgerson, CONSORT is the only guideline for reporting RCTs that has
Birks, & Porthouse, 2005), psychology (Michie et al., 2011; beendevelopedwithsuchrigor,andithasremainedmorepro-
Stinson, McGrath, & Yamada, 2003), and public health minentthatany otherguideline for over 15 years; for greatest
(Semaanetal.,2002).Biomedicalresearchershavedeveloped impact,anyfurtherreportingguidelinesrelatedtoRCTsshould
guidelines to improve the reporting of RCTs of health-related bedevelopedincollaboration with theCONSORT Group.
interventions (Schulz, Altman, & Moher, for the CONSORT
Group,2010).However,manysocialandbehavioralscientists
Limitations of Previous Reporting Guidelines for Social
have not fully adopted these guidelines, which may not be
and Psychological Interventions
wholly adequate for social and psychological interventions in
their current form (Bonell, Oakley, Hargreaves, Strange, & Researchers and journal editors in the social and behavioral
Rees, 2006; Davidson et al., 2003; Perry et al., 2010; Stinson sciences are generally aware of CONSORT but often object
etal.,2003).Becauseoftheuniquefeaturesoftheseinterven- thatitisnotfullyappropriateforsocialandpsychologicalinter-
tions,updated reporting guidance is needed. ventions(Bonelletal.,2006;Davidsonetal.,2003;Perryetal.,
This articledescribes thedevelopment ofa reporting guide- 2010; Stinson et al., 2003). As a result, uptake of CONSORT
linethataimstoimprovethequalityofreportsofRCTsofsocial guidelinesinthesedisciplinesislow.Whilesomecriticismsare
and psychological interventions. We explain how reporting duetoinaccurateperceptionsaboutcommonfeaturesofRCTs
guidelineshaveimprovedthequalityofreportsinmedicine,and acrossdisciplines,manyrelatetoreallimitationsforsocialand
why guidelineshavenot yet improved the quality ofreports in psychological interventions (Mayo-Wilson, 2007). For exam-
other disciplines. We then introduce a plan to develop a new ple, CONSORT is most relevant to RCTs in medical disci-
reporting guideline for RCTs—Consolidated Standards of plines; it was developed by biostatisticians and medical
Reporting Trials (CONSORT)-SPI(an extensionfor socialand researchers with minimal input from experts in other disci-
psychological interventions)—which will be written using plines.Journaleditors,aswellassocialandbehavioralscience
recommendtechniquesforguidelinedevelopmentanddissemi- researchers, believe there is a need to include appropriate sta-
nation (Moher, Schulz, Simera, & Altman, 2010). Wide stake- keholders in developing a new, targeted guideline to improve
holder involvement and consensus are needed to create a uptake in their disciplines (Gill, 2011; Torgerson et al.,
useful, acceptable, and evidence-based guideline, so we hope 2005). The CONSORT Group has produced extensions of the
torecruitstakeholdersfrommultipledisciplinesandprofessions. originalCONSORTStatementrelevanttosocialandpsycholo-
Randomized trials are not the only rigorous method for gical interventions, such as additional checklists for cluster
evaluating interventions; many alternatives exist when RCTs (Campbell, Elbourne, & Altman, 2004), nonpharmacological
arenotpossibleorappropriate duetoscientific,practical,and (Boutron et al., 2008a), pragmatic (Zwarenstein et al., 2008),
ethical concerns (Bonell et al., 2011). Nonetheless, RCTs are and quality of life RCTs (Calvert, Blazeby, Revicki, Moher,
important to policy makers, practitioners, scientists, and ser- & Brundage, 2011). These extensions provide important
viceusers,astheyaregenerallyconsideredthemostvalidand insights, but complex social and psychological interventions,
reliable research method for estimating the effectiveness of for example, include multiple, interacting components at sev-
interventions(Chalmers,2003).Moreover,manyofthe issues eral levels, with various outcomes. These RCTs require use
facedinreportingRCTsalsorelatetootherevaluationdesigns. of several extensions at once, creating a barrier to guideline
Asaresult,thisprojectwillfocusonstandardsforRCTs,which uptake; increasing intervention complexity also gives rise to
couldthenalsoinformthedevelopmentoffutureguidelinesfor newissuesthatarenotincludedinexistingguidelines.There-
otherevaluation designs. fore,simplydisseminatingCONSORTguidelinesastheystand
is insufficient, as this would not address the need for editors
and authors to ‘‘buy-in’’ to this process. To improve uptake
Impact of CONSORT Guidelines
inthesedisciplines,CONSORTguidelinesneedtobeextended
Reportingguidelineslist(intheformofachecklist)themini- to specifically address the important features of social and
mum information required to understand the methods and psychological interventions.
resultsofstudies.Theydonotprescriberesearch conduct,but Social and behavioral scientists have developed other
facilitate the writing of transparent reports by authors and reportingguidelines,includingtheWorkgroupforIntervention
appraisal of reports by research consumers. For example, the DevelopmentandEvaluationResearch(WIDER)Recommen-
CONSORT Statement 2010 is an evidence-based guideline; dationsfor behavioral change interventions(Abraham, for the
to identify items, the developers reviewed evidence of trial WIDER,2009;Michieetal.,2011),theAmericanEducational
designandconductthatcouldcontributetobias.Usingconsen- ResearchAssociation’s(AERA,2006)StandardsforReporting
susmethods,theydevelopedachecklistof25itemsandaflow Research, the REPOrting of Studies in Education (REPOSE)
diagram (Schulz et al., 2010). CONSORT has improved the guidelines for primary research in education (Newman &
reporting of thousands of medical experiments (Turner et al., Elbourne, 2004), and the Journal Article Reporting Standards
Grant et al. 597
(JARS) of the American Psychological Association (APA) Some aspects of internal validity, although included in
Publications and Communications Board Working Group on CONSORT, remain poorly reported—even in the least com-
JARS (2008). While they address issues not covered by the plex social and psychological intervention studies. Reports
CONSORT Statement and its extensions, these guidelines ofRCTsshoulddescribeproceduresforminimizingselection
(except for JARS; APA Publications and Communications bias, but reports often omit information about random
BoardWorkingGrouponJARS,2008)donotprovidespecific sequence generation and allocation concealment (Ladd,
guidanceforRCTs.Moreover,comparedwiththeCONSORT McCrady, Manuel, & Campbell, 2010; Perry & Johnson,
Statement and its official extensions, guidelines in the social 2008),andpsychologicaljournalsreportmethodsofsequence
andbehavioralscienceshavenotconsistentlyfollowedoptimal generation less frequently than medical journals (Stinson
techniques for guideline development and dissemination that etal.,2003).Areviewofeducationalreportsfoundnostudies
arerecommendedbyinternationalleadersintheadvancement that adequately reported allocation concealment (Torgerson
of reporting guidelines (Moher, Schulz, et al., 2010), such as et al., 2005), and reports in criminology often lack informa-
the use of systematic literature reviews and formal consensus tion about randomization procedures (Gill, 2011; Perry
methods to select reporting standards (Grant, Montgomery, & et al., 2010). RCTs of social and psychological interventions
Mayo-Wilson, 2012). Researchers in public health, psychol- may also use nontraditional randomization techniques, such
ogy, education, social work, and criminology have noted that as stepped wedge or natural allocation (MRC, 2011), which
these guidelines could be more ‘‘user-friendly,’’ and dissemi- needtobethoroughlydescribed.Inaddition,reportsofsocial
nation could benefit from up-to-date knowledge transfer tech- and psychological intervention trials often fail to include
niques (Abraham, 2009; Armstrong et al., 2008; Davidson details about trial registration, protocols, and adverse events
et al., 2003; Naleppa & Cagle, 2010; Perry & Johnson, 2008; (Ladd et al., 2010; Perry & Johnson, 2008), which may
Stinson et al., 2003;Torgerson et al., 2005). includeimportantnegativeconsequencesatindividual,famil-
For example, JARS—a notable and valuable guideline for ial, and community levels.
empirical psychological research—is endorsed by few journals Other aspects of CONSORT may require greater emphasis
outside of the APA, whereas CONSORT is endorsed by ormodificationforRCTsofsocialandpsychologicalinterven-
hundreds of journals internationally. According to ISI Web of tions. In developing this CONSORT extension, we expect to
KnowledgeandGoogleScholarcitations,JARSiscitedapprox- identifynewitemsandtoadaptexistingitemsthatrelatetothe
imatelyadozentimesannually,whileCONSORTguidelinesare internalvalidity.Thesemayincludeitemsdiscussedduringthe
cited hundredsoftimesper year. Moreover, the APA commis- development of previous CONSORT extensions or other
sioned a select group of APA journal editors and reviewers to guidelines, as well as items suggested by participants in this
developJARS,andthegroupbasedmostoftheirworkonexis- project. For example, it may not be possible to blind partici-
tentCONSORTguidelines;bycomparison,officialCONSORT pants and providers of interventions, but blinding of outcome
extensionshavebeendevelopedusingrigorousconsensusmeth- assessorsisoftenpossiblebutrarelyreported,andfewstudies
ods, have involved various international stakeholders in guide- explainifblindingwasmaintainedorhowlackofblindingwas
linedevelopmentanddissemination,andupdatecontentonthe handled (Davidson et al., 2003; Ladd et al., 2010; Perry &
most recent scientific literature. Nonetheless, no current CON- Johnson,2008).Insocialandpsychologicalinterventionstud-
SORT guideline adequately addresses the unique features of ies, outcome measures are often subjective, variables may
social and psychological interventions. This new CONSORT relate to latent constructs, and information may come from
extension will incorporate lessons from previous extensions, multiple sources (e.g., participants and providers). While an
reportingguidelines,andtheresearchliteraturetoaidthecritical issue in other areas of research, the influence on RCT results
appraisal,replication,anduptakeofthisresearch. of the quality of subjective outcome measures in social and
psychologicalinterventionresearchhaslongbeenhighlighted,
giventheirprevalenceinsocialandpsychologicalintervention
Aspects of Internal Validity
research (Marshall et al., 2000). Descriptions of the validity,
Internal validity is the extent to which the results of a study reliability, and psychometric properties of such measures are
maybeinfluencedbybias.Likeotherstudydesigns,thevalidity thereforeparticularlyusefulforsocialandpsychologicalinter-
ofRCTsdependsonhigh-qualityexecution.Poorlyconducted ventiontrials,especiallywhentheyarenotwidelyavailableor
RCTs can produce more biased results than well-conducted discussed in the research literature (Campbell et al., 2004;
RCTs and well-conducted nonrandomized studies (Pildal Fraser,Galinsky,Richman,&Day,2009).Moreover,multiple
etal.,2007;Prescottetal.,1999).Forexample,evidenceindi- measuresmaybeanalyzedinseveralways,soauthorsneedto
catesthatRCTsthatdonotadequatelyconcealtherandomiza- transparently report which procedures were performed and to
tion sequence can exaggerate effect estimates by up to 30% explaintheir rationale.
(Schulz, Chalmers, Hayes, & Altman, 1995), while low-
quality reports of these RCTs are associated with effect esti-
Aspects of External Validity
mates exaggerated by up to 35% (Moher et al., 1999). Social
and psychological intervention RCTs are susceptible to these External validity is the extent to which a study’s results are
risksofbias aswell. applicable in other settings or populations. Currently, given
598 Research on Social Work Practice 23(6)
thatRCTsareprimarilydesignedtoincreasetheinternalvalid- Well-described RCT reports will include the characteristics
ityofstudyfindings,theCONSORTStatementgivesrelatively of all participants (volunteers, those who enrolled, and those
littleattentiontoexternalvalidity.Whilehighinternalvalidity who completed) in sufficient detail for readers to assess the
is an important precondition for any discussion of an RCT’s comparabilityofthestudysampletopopulationsandinevery-
externalvalidity,updatingtheCONSORTStatementtoinclude day services (AERA, 2006; APA Publications and Communi-
moreinformationaboutexternalvalidityiscriticalfortherele- cations Board Working Group on JARS, 2008; Evans &
vance and uptake of a CONSORT extension for social and Brown,2003)
psychologicalinterventions.Theseinterventionsmaybeinflu- Finally, given that these interventions often occur in social
enced by context, as different underlying social, institutional, environments,reportsshoulddescribefactorsoftheRCTcon-
psychological, and physical structures may yield different textthatarebelievedtosupport,attenuate,orfrustrateobserved
causal and probabilistic relations between interventions and effects (Moore, 2002). Interventions may differ across groups
observed outcomes. Contextual information is necessary to of different social or socioeconomic positions, and equity
compare the effectiveness of an intervention across time and considerations should be addressed explicitly (Tugwell et al.,
place (Cartwright & Munro, 2010). Lack of information rele- 2010;Welchetal.,2012).Severalaspectsofsettingandimple-
vant to external validity may prevent practitioners or policy mentationmaybeimportanttoconsider,suchasadministrative
makers from using evidence appropriately to inform decision support, staff training and supervision, organizational
making;yet,existingguidelinesdonotadequatelyexplainhow resources, the wider service system, and concurrent political
authors should describe (a) how interventions work, (b) for or social events (Bonell et al., 2012; Fixsen, Naoom, Blase,
whom,and(c)underwhatconditions(Moore&Moore,2011). Friedman, & Wallace, 2005; Shepperd et al., 2009; Wang,
First,itisusefulforauthorstoexplainthekeycomponents Moss,&Hiller,2006).Reportingprocessevaluationsmayhelp
of interventions, how those components could be delivered, understand mechanisms andoutcomes.
and how they relate to the outcomes selected. At present,
authorscanfollowcurrentstandardsforreportinginterventions
Developing a New CONSORT Extension
without providing adequate details about complex interven-
tions (Shepperd et al., 2009). Many reports neither contain ThisnewreportingguidelineforRCTsofsocialandpsycholo-
sufficientinformationabouttheinterventionstestednorrefer- gical interventions will be an official extension of the
encetreatmentmanuals(Glasziou,Meats,Heneghan,&Shep- CONSORT Statement. Optimally, it will help improve the
perd,2008).Providinglogicmodels—asdescribedintheMRC reportingofthesestudies.LikeotherofficialCONSORTexten-
FrameworkforComplexInterventions(Craigetal.,2008)—or sions (Boutron et al., 2008a; Campbell et al., 2004; Hopewell
presentingtheoriesofchangecanhelpelucidatelinksincausal et al., 2008; Zwarenstein et al., 2008), this guideline will be
chains that can be tested, identify important mediators and integratedwith theCONSORT Statement andpreviousexten-
moderators, and facilitate syntheses in reviews (Ivers et al., sions, and updates of the CONSORT Statement may incorpo-
2012).Moreover,interventionsarerarelyimplementedexactly rate references tothisextension.
asdesigned,andcomplexinterventionsmaybedesignedtobe Theprojectisbeingledbyaninternationalcollaborationof
implemented with some flexibility, in order to accommodate researchers, methodologists, guideline developers, funders,
differences across participants (Hawe et al., 2004), so it is service providers, journal editors, and consumer advocacy
important to report how interventions were actually delivered groups.We willbe recruiting participants ina mannersimilar
by providers and actually received by participants (Hardeman to other reporting guideline initiatives—identifying stake-
etal.,2008).Particularlyforsocialandpsychologicalinterven- holders through literature reviews, the project’s International
tions,theintegrityofimplementingtheintendedfunctionsand AdvisoryGroup,andstakeholder-initiatedinterestintheproj-
processesoftheinterventionareessentialtounderstand(Hawe ect(Michieetal.,2011;Schulzetal.,2010).Wehopetorecruit
etal.,2004).AsRCTsofaparticularinterventioncanyielddif- stakeholders with expertise from all related disciplines and
ferent relative effects depending on the nature of the control regionsoftheworld,includinglow-andmiddle-incomecoun-
groups, information about delivery and uptake should be pro- tries. Methodologists will identify items that relate to known
videdforalltrialarms(McGrath,Stinson,&Davidson,2003). sourcesofbias,andtheywillidentifyitemsthatfacilitatesys-
Second, reports should describe recruitment processes and tematic reviews and research synthesis. Funders will consider
representativeness of samples. Participants in RCTs of social howtheguidelinecanaidtheassessmentofgrantapplications
and psychological intervention are often recruited outside of forRCTsandmethodologicalinnovationsininterventioneva-
routine practice settings via processes that differ from routine luation. Practitioners will identify information that can aid
services (AERA, 2006). An intervention that works for one decision making. Journal editors will identify practical steps
group of people may not work for people living in different toimplementthe guideline andtoensure uptake.
culturesorphysicalspaces,oritmaynotworkforpeoplewith We will use consensus techniques to reduce bias in group
slightly different problems and comorbidities. Enrolling in an decision making and to promote widespread guideline uptake
RCT can be a complex process that affects the measured and and knowledge translation activities upon project completion
unmeasured characteristics of participants, and recruitment (Murphy et al., 1998). Following rigorous reviews of existing
may differ from how users normally access interventions. guidelines and current reporting quality, we will conduct an
Grant et al. 599
online Delphiprocesstoidentifyaprioritizedlist ofreporting hopethisguidelinewillreducetheeffortandtimerequiredfor
items to consider for the extension. That is, we will invite a authorstowrite reports ofRCTs.
group of experts to electronically answer questions about RCTsarenottheonlyvalidmethodforevaluatinginterven-
reportingitemsandtosuggestfurtherquestions.Wewillcircu- tions(Bonelletal.,2011)noraretheytheonlytypeofresearch
latetheirfeedbacktothegroupandaskasecondroundofques- thatwouldbenefitfrombetterreporting(Goldbeck&Vitiello,
tions.TheDelphiprocesswillcaptureavarietyofinternational 2011). Colleagues have identified the importance of reporting
perspectives and allow participants to share their views anon- standards for other types of research, including observational
ymously.FollowingtheDelphiprocess,wewillhostaconsen- (vonElmetal.,2007),quasi-experimental(DesJarlais,Lyles,
sus meeting to review the findings and to generate a list of Crepaz, & the TREND Group, 2004), and qualitative studies
minimalreportingstandards,mirroringthedevelopmentofpre- (Tong, Sainsbury, & Craig, 2007). This guideline is the first
vious CONSORT guidelines (Boutron et al., 2008b; Schulz step toward improving reports of many designs for evaluating
et al., 2010;Zwarenstein etal., 2008). social and psychological interventions, which we hope will
Together,participantsinthisprocesswillcreateachecklist beaddressedbythisandfutureprojects.Weinvitestakeholders
ofreportingitemsandaflowchartforreportingsocialandpsy- fromdisciplinesthatfrequentlyresearchtheseinterventionsto
chologicalinterventionRCTs.Inaddition,wewilldevelopan jointhisimportanteffortandparticipateinguidelinedevelop-
Explanation and Elaboration (E&E) document to explain the mentbyvisiting ourwebsite, where theycan find moreinfor-
scientific rationale for each recommendation and to provide mation about the project, updates on its progress, and sign up
examplesofclearreporting;asimilardocumentwasdeveloped tobeinvolved(http://tinyurl.com/CONSORT-study).
by the CONSORT group to help disseminate a better under-
standing for each included checklist item (Moher, Hopewell,
Acknowledgments
etal.,2010).Thisdocumentwillhelppersuadeeditors,authors,
We thank the Centre for Evidence-Based Intervention (Oxford
and funders of the importance of the guideline. It will be a
University), the Centre for Outcomes Research and Effectiveness
useful pedagogical tool, helping students and researchers
(UniversityCollegeLondon),andtheNationalCollaboratingCentre
understandthemethodsforconductingRCTsofsocialandpsy- forMentalHealth(NCCMH)fortheirsupport.
chological interventions, and it will help authors meet the
guideline requirements (Moher,Schulz, et al.,2010).
Authors’ Note
Thesuccessofthisprojectdependsonwidespreadinvolve-
TheCONSORT-SPI(socialandpsychologicalinterventions)Interna-
ment and agreement among key international stakeholders in
tionalAdvisoryGroupincludesJ.LawrenceAber,distinguishedpro-
research, policy, and practice. For example, previous develo-
fessorofappliedpsychologyandpublicpolicy,SteinhardtSchoolof
pers have obtained guideline endorsement by journal editors
Culture,Education,andHumanDevelopment,NewYorkUniversity;
who require authors and peer reviewers to use the guideline
Chris Bonell, professor of sociology and social intervention, Centre
duringarticlesubmissionandwhomustenforcejournalarticle for Evidence-Based Intervention, University of Oxford; David M.
word limits (Michie, Fixsen, Grimshaw, & Eccles, 2009). Clark,chairofpsychology,DepartmentofExperimentalPsychology,
Many journal editors have already agreed to participate, and UniversityofOxford;FrancesGardner,professorofchildandfamily
wehopeotherresearchersandstakeholderswillvolunteertheir psychology, Centre for Evidence-Based Intervention, University of
time andexpertise. Oxford;StevenHollon,AmericanPsychological AssociationGuide-
linesCommittee(Chair),GertrudeConawayProfessorofPsychology,
DepartmentofPsychology,VanderbiltUniversity;JimMcCambridge,
Conclusion seniorlecturerinBehaviourChange,DepartmentofSocialandEnvi-
ronmentalHealthResearch,LondonSchoolofHygieneandTropical
Reporting guidelines help us use scarce resources efficiently
Medicine;SusanMichie,professorofhealthpsychology,Department
andethically.RCTsareexpensive,andthepublichavearight
of Clinical, Educational & Health Psychology, University College
to expect returns on their investments through transparent,
London; Laurence Moore, professor of public health improvement,
usablereports.WhenRCTreportscannotbeused(forwhatever CardiffSchoolofSocialSciences,CardiffUniversity;MarkPetticrew,
reason),resourcesarewasted.Participantscontributetheirtime professorofpublichealthevaluation,DepartmentSocialandEnviron-
and put themselves at risk of harm to generate evidence that mental Health Research, London School of Hygiene and Tropical
willhelpothers,andresearchersshoulddisseminatethatinfor- Medicine; Lawrence Sherman, Wolfson Professor of Criminology,
mationeffectively(Davidsonetal.,2003).Policymakersben- Cambridge Institute of Criminology, Cambridge University; Steve
efit from research when developing effective, affordable Pilling, director, Centre for Outcomes Research and Effectiveness,
UniversityCollegeLondon;JamesThomas,associatedirectorEPPI-
standards of practice and choosing which programs and ser-
Centre, reader in social policy, Institute of Education, University of
vices to fund. Administrators and managers are required to
London; Elizabeth Waters, Jack Brockhoff Chair of Child Public
makecontextuallyappropriatedecisions.Transparentreporting
Health, McCaughey VicHealth Centre for Community Wellbeing,
ofprimarystudiesisessentialfortheirinclusioninsystematic
MelbourneSchoolofPopulation&GlobalHealth,UniversityofMel-
reviews that inform these activities. For example, there is the
bourne,Australia;DavidWeisburd,directorandWalterE.MeyerPro-
needtodetermine ifprimarystudiesarecomparable,examine fessorofLawandCriminalJustice,InstituteofCriminology,Hebrew
biases within included studies, assess the generalizability of UniversityFacultyofLaw,Jerusalem;JoanneYaffe,associateprofes-
results, and implement effective interventions. Finally, we sor,CollegeofSocialWork,UniversityofUtah.PM,EMW,andSG
600 Research on Social Work Practice 23(6)
conceivedoftheideafortheproject.Allauthors helpedtodraftthe Boutron,I.,Moher,D.,Altman,D.G.,Schulz,K.,&Ravaud,P.,for
article,andallhavereadandapprovedthefinalarticle. the CONSORT group. (2008b). Methods and processes of the
CONSORT group: Example of an extension for trials assessing
Declaration ofConflicting Interests nonpharmacologic treatments. Annals of Internal Medicine, 148,
W60–W67.
Theauthorsdeclarednopotentialconflictsofinterestwithrespectto
Calvert, M., Blazeby, J., Revicki, D., Moher, D., & Brundage, M.
theresearch,authorship,and/orpublicationofthisarticle.
(2011). Reporting quality of life in clinical trials: A CONSORT
extension.TheLancet,378,1684–1685.
Funding
Campbell Collaboration. (2013). Retrieved April 26, 2013, from
Theauthorsdisclosedreceiptofthefollowingfinancialsupportfor http://campbellcollaboration.org/
theresearch,authorship,and/orpublicationofthisarticle:Thisproj- Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004). CON-
ect is funded by the U.K. Economic and Social Research Council SORT statement: Extension to cluster randomised trials. British
(ES/K00087X/1). S.G. is supported by a linked Clarendon Fund-
MedicalJournal,328,702–708.
GreenTempletonCollegeAnnualFundScholarshipforhisdoctoral
Cartwright, N., & Munro, E. (2010). The limitations of randomized
studies and research. D.M. is supported by a University Research
controlledtrialsinpredictingeffectiveness.JournalofEvaluation
Chair.
inClinicalPractice,16,260–266.
Chalmers,I.(2003).Tryingtodomoregoodthanharminpolicyand
References practice:Theroleofrigorous,transparent,up-to-dateevaluations.
Abraham, C., for the Workgroup for Intervention Development and TheANNALSoftheAmericanAcademyofPoliticalandSocialSci-
EvaluationResearch.(2009).WIDERrecommendationstoimprove ence,589,22–40.
reporting of the content of behaviour change interventions. Cochrane Collaboration. (2013). Retrieved April 26, 2013, from
Retrieved November 14, 2012, from http://interventiondesign. http://www.cochrane.org/
co.uk/wp-content/uploads/2009/02/wider-recommendations.pdf Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., &
American Educational Research Association. (2006). Standards for Petticrew, M. (2008). Developing and evaluating complex inter-
reporting on empirical social science research in AERA publica- ventions: The new Medical Research Council guidance. British
tions.EducationalResearcher,35,33–40. MedicalJournal,337,979–983.
American Psychological Association Publications and Communica- Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G.,
tions Board Working Group on Journal Article Reporting Stan- Knatterud, G. L., Orleans, C. T., Spring, B.,... Whitlock, E. P.
dards. (2008). Reporting standards for research in psychology: (2003).Evidence-basedbehaviouralmedicine:Whatisitandhow
Whydoweneedthem?whatmighttheybe?AmericanPsycholo- doweachieveit?AnnalsofBehavioralMedicine,26,161–171.
gist,63,839–851. Des Jarlais, D. C., Lyles, C., & Crepaz, N., & the TREND Group.
Armstrong,R.,Waters,E.,Moore,L.,Riggs,E.,Cuervo,L.G.,Lum- (2004).Improvingthereportingqualityofnonrandomizedevalua-
biganon,P.,&Hawe,P.(2008).Improvingthereportingofpublic tions of behavioral and public health interventions: The TREND
healthinterventionresearch:AdvancingTRENDandCONSORT. statement.AmericanJournalofPublicHealth,94,361–366.
JournalofPublicHealth,30,103–109. Evans,T.,&Brown,H.(2003).Roadtrafficcrashes:Operationalizing
Bonell,C.(2002).Theutilityofrandomizedcontrolledtrialsofsocial equity in the context of health sector reform. Injury Control and
interventions: An examination of two trials of HIV prevention. SafetyPromotion,10,11–12.
CriticalPublicHealth,12,321–334. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., &
Bonell,C.,Fletcher,A.,Morton,M.,Lorenc,T.,&Moore,L.(2012). Wallace,F.(2005).Implementationresearch:Asynthesisofthelit-
Realistrandomisedcontrolledtrials:Anewapproachtoevaluating erature.Tampa,FL:UniversityofSouthFlorida.
complex public health interventions. Social Science & Medicine, Fraser,M.W.,Galinsky,M.J.,Richman,J.M.,&Day,S.H.(2009).
75,2299–2306. Interventionresearch:Developingsocialprograms.Oxford,Eng-
Bonell,C.,Oakley,A.,Hargreaves,J.,Strange,V.,&Rees,R.(2006). land:OxfordUniversityPress.
Assessment of generalisability in trials of health interventions: Gill,C.E.(2011).Missinglinks:Howdescriptivevalidityimpactsthe
Suggested framework and systematic review. British Medical policy relevance of randomized controlled trials in criminology.
Journal,333,346–349. JournalofExperimentalCriminology,7,201–224.
Bonell,C.P.,Hargreaves,J.,Cousens,S.,Ross,D.,Hayes,R.,Petti- Glasziou,P.,Meats,E.,Heneghan,C.,&Shepperd,S.(2008).Whatis
crew,M.,&Kirkwood,B.R.(2011).Alternativestorandomisation missingfromdescriptionsoftreatmentintrialsandreviews?Brit-
intheevaluationofpublichealthinterventions:Designchallenges ishMedicalJournal,336,1472–1474.
andsolutions.JournalofEpidemiologicalCommunityHealth,65, Goldbeck,L.,&Vitiello,B.(2011).Reportingclinicaltrialsofpsy-
582–587. chosocial interventions in child and adolescent psychiatry and
Boutron,I.,Moher,D.,Altman,D.G.,Schulz,K.,&Ravaud,P.,for mental health. Child and Adolescent Psychiatry and Mental
the CONSORT group. (2008a). Extending the CONSORT state- Health,5,4.
menttorandomizedtrialsofnonpharmacologictreatment:Expla- Grant,S.,Montgomery,P.,&Mayo-Wilson,E.(2012).Development
nation and Elaboration. Annals of Internal Medicine, 148, of a CONSORT extension for interventions in public health and
295–309. related disciplines. The Lancet, 380, S14. Retrieved April 26,
Grant et al. 601
2013, from http://www.thelancet.com/abstracts/public-health-sci- Moher,D.,Hopewell,S.,Schultz,K.F.,Montori,V.,Gøtzsche,P.C.,
ence-in-the-uk Devereaux, P. J.,...Altman, D. G. (2010). CONSORT 2010
Hardeman,W.,Michie,S.,Fanshawe,T.,Prevost,A.T.,McLoughlin, ExplanationandElaboration:Updatedguidelinesforreportingpar-
K., & Kinmonth, A. L. (2008). Fidelity of delivery of a physical allelgrouprandomisedtrials.BritishMedicalJournal,340,c869.
activityintervention:Predictorsandconsequences.Psychology& Moher,D.,Pham,B.,Jones,A.,Cook,D.J.,Jadad,A.R.,Moher,M.,
Health,23,11–24. Klassen,T.P.(1999).Doesqualityofreportsofrandomizedtrials
Hawe,P.,Shiell,A.,&Riley,T.(2004).Complexinterventions:How affectestimatesofinterventionefficacyreportedinmeta-analyses?
‘‘outofcontrol’’canarandomisedcontrolledtrialbe?BritishMed- Lancet,352,609–613.
icalJournal,328,1561–1563. Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010). Gui-
Henggeler, S. W., Schoenwald, S. K., Rowland, M.D., & Cunning- dancefordevelopersofhealthresearchreportingguidelines.PLoS
ham,P.B.(2002).Seriousemotionaldisturbancesinchildrenand Medicine,7,e1000217.
adolescents:Multisystemictherapy.NewYork,NY:Guilford. Moore,L.(2002).Researchdesignfortherigorousevaluationofcom-
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., plex educational interventions: Lessons from health services
Altman, D. G., & Schulz, K. F., & the CONSORT Group. research.BuildingResearchCapacity,1,4–5.
(2008).CONSORTforreportingrandomisedtrialsinjournaland Moore, L., & Moore, G. F. (2011). Public health evaluation: Which
conferenceabstracts.Lancet,371,281–283. designs work, for whom and under what circumstances? Journal
Ivers,N.,Jamtvedt,G.,Flottorp,S.,Young,J.M.,Odgaard-Jensen,J., ofEpidemiology&CommunityHealth,65,596–597.
French, S. D., ... Oxman, A. D. (2012). Audit and feedback: Murphy,M.K.,Black,N.A.,Lamping,D.L.,McKee,C.M.,Sanders,
Effects on professional practice and healthcare outcomes. C. F. B., Askham, J., & Marteau, T. (1998). Consensus develop-
CochraneDatabaseofSystematicReviews,6,CD000259. ment methods, and their use in clinical guideline development.
Ladd,B.O.,McCrady,B.S.,Manuel,J.K.,&Campbell,W.(2010). HealthTechnologyAssessment,2,1–88.
Improving the quality of reporting alcohol outcome studies: Naleppa, M. J., & Cagle, J. G. (2010). Treatment fidelity in social
Effects of the CONSORT statement. Addictive Behaviors, 35, work intervention research: A review of published studies.
660–666. ResearchonSocialWorkPractice,20,674–681.
Littell,J.H.,Campbell,M.,Green,S.,&Toews,B.(2009).Multisys- Newman,M.,&Elbourne,D.(2004).Improvingtheusabilityofedu-
temic therapy for social, emotional, and behavioral problems in cationalresearch:GuidelinesfortheREPOrtingofprimaryempiri-
youth aged 10-17. Cochrane Library of Systematic Reviews, 4, cal research Studies in Education (The REPOSE Guidelines).
CD004797. Evaluation&ResearchinEducation,18,201–212.
Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., & Pawson,R.,Greenhalgh,T.,Harvey,G.,&Walshe,K.(2004).Realist
Fenton,M.(2000).Unpublishedratingscales:Amajorsourceof synthesis:Anintroduction.Manchester,England:ESRCResearch
bias inrandomised controlledtrialsoftreatments forschizophre- MethodsProgramme,UniversityofManchester.
nia.BritishJournalofPsychiatry,176,249–252. Perry,A.E.,&Johnson,M.(2008).ApplyingtheConsolidatedStan-
Mayo-Wilson, E. (2007). Reporting implementation in randomized dardsofReportingTrials(CONSORT)tostudiesofmentalhealth
trials:Proposedadditionstotheconsolidatedstandardsofreporting provision for juvenile offenders: A research note. Journal of
trialsstatement.AmericanJournalofPublicHealth,97,630–633. ExperimentalCriminology,4,165–185.
McGrath,P.J.,Stinson,J.,&Davidson,K.(2003).Commentary:The Perry,A.E.,Weisburd,D.,&Hewitt, C.(2010).Are criminologists
journalofpediatricpsychologyshouldadopttheCONSORTstate- describing randomized controlled trials in ways that allow us to
mentasawayofimprovingtheevidencebaseinpediatricpsychol- assess them? Findings from a sample of crime and justice trials.
ogy.JournalofPediatricPsychology,28,169–171. JournalofExperimentalCriminology,6,245–262.
MedicalResearchCouncil.(2008).Aframeworkfordevelopmentand Pildal,J.,Hro´bjartsson,A.,Jørgensen,K.J.,Hilden,J.,Altman,D.G.,
evaluation of RCTs for complex interventions to improve health. & Gøtzsche, P. C. (2007). Impact of allocation concealment on
London,England:Author. conclusionsdrawnfrommeta-analysesofrandomizedtrials.Inter-
MedicalResearchCouncil.(2011).Usingnaturalexperimentstoeval- nationalJournalofEpidemiology,36,847–857.
uatepopulationhealthinterventions:Guidanceforproducersand Prescott,R.J.,Counsell,C.E.,Gillespie,W.J.,Grant,A.M.,Russell,
usersofevidence.London,England:Author. I.T.,Kiauka,S.,...Russel,D.(1999).Factorsthatlimitthequal-
Michie,S.,Abraham,C.,Eccles,M.P.,Francis,J.J.,Hardeman,W., ity, number and progress of randomised controlled trials. Health
&Johnston,M.(2011).Strengtheningevaluationandimplementa- TechnologyAssessment,3,1–143.
tionbyspecifyingcomponentsofbehaviourchangeinterventions: Schulz,K.F.,Altman,D.G.,&Moher,D.,fortheCONSORTGroup
Astudyprotocol.ImplementationScience,6,10. (2010).CONSORT2010statement:Updatedguidelinesforreport-
Michie,S.,Fixsen,D.,Grimshaw,J.M.,&Eccles,M.P.(2009).Spe- ingparallelgrouprandomisedtrials.BritishMedicalJournal,340,
cifying and reporting complex behaviour change interventions: 698–702.
Theneedforascientificmethod.ImplementationScience,4,40. Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995).
Moher,D.,Altman,D.G.,Schulz,K.F.,&Elbourne,D.R.(2004). Allocation concealment in randomised trials: Defending against
Opportunitiesandchallengesforimprovingthequalityofreport- deciphering.Lancet,359,614–617.
ingclinicalresearch:CONSORTandbeyond.CanadianMedical Semaan, S., Kay, L., Strouse, D., Sogolow, E., Mullen, P. D., Neu-
AssociationJournal,171,349–350. mann,M.S.,...DesJarlais,D.C.(2002).AprofileofU.S.-based
602 Research on Social Work Practice 23(6)
trialsofbehavioralandsocialinterventionsforHIVriskreduction. trials (CONSORT) and the completeness of reporting of rando-
JournalofAcquiredImmuneDeficiencySyndromes,30,S30–S50. mised controlled trials (RCTs) published in medical journals.
Shepperd,S.,Lewin,S.,Straus,S.,Clarke,M.,Eccles,M.P.,Fitzpatrick, CochraneDatabaseofSystematicReviews,11,MR000030.
R.,...Sheikh,A.(2009).Canwesystematicallyreviewstudiesthat vonElm,E.,Altman,D.G.,Egger,M.,Pocock,S.J.,Gotzsche,P.C.,
evaluatecomplexinterventions?PLoSMedicine,6,31000086. &Vandenbroucke,J.P.(2007).Thestrengtheningthereportingof
Stinson,J.N.,McGrath,P.J.,&Yamada,J.T.(2003).Clinicaltrials observational studies in epidemiology (STROBE) statement:
in the journal of pediatric psychology: Applying the CONSORT Guidelinesforreportingobservationalstudies.AnnalsofInternal
statement.JournalofPediatricPsychology,28,159–167. Medicine,147,573–577.
Tong,A.,Sainsbury,P.,&Craig,J.(2007).Consolidatedcriteriafor Wang,S.,Moss,J.R.,&Hiller,J.E.(2006).Applicabilityandtrans-
reporting qualitative research (COREQ): A 32-item checklist for ferabilityofinterventionsinevidence-basedpublichealth.Health
interviewsandfocusgroups.InternationalJournalforQualityin PromotionInternational,21,76–83.
HealthCare,19,349–357. Welch, V., Petticrew, M., Tugwell, P., Moher, D., O’Neill, J., &
Torgerson,C.J.,Torgerson,D.J.,Birks,Y.F.,&Porthouse,J.(2005). Waters, E.,...the PRISMA-Equity Bellagio group. (2012).
A comparison of RCTs in health and education. British Educa- PRISMA-Equity 2012 extension: Reporting guidelines for sys-
tionalResearchJournal,31,761–785. tematicreviewswithafocusonhealthequity.PLoSMedicine,9,
Tugwell, P.,Petticrew, M.,Kristjansson, E.,Welch,V.,Ueffing, E., e1001333.
Waters,E.,... Kelly,M.P.(2010).Assessingequityinsystematic Zwarenstein,M.,Treweek,S.,Gagnier,J.J.,Altman,D.G.,Tunis,S.,
reviews: Realising the recommendations of the commission on & Haynes, B.,...for the CONSORT and Pragmatic Trials in
socialdeterminantsofhealth.BritishMedicalJournal,341,c4739. Healthcare (Practihc) group. (2008). Improving the reporting of
Turner,L.,Shamseer,L.,Altman,D.G.,Weeks,L.,Peters,J.,Kober, pragmatictrials:AnextensionoftheCONSORTstatement.British
T.,...Moher, D. (2012). Consolidated standards of reporting MedicalJournal,337,a2390.