Utilization of a population health survey in policy and practice: a case study.

0.29 MB·English

by  Laws, Rachel

Lawsetal.HealthResearchPolicyandSystems2013,11:4
http://www.health-policy-systems.com/content/11/1/4
RESEARCH Open Access
Utilization of a population health survey in policy
and practice: a case study
Rachel Laws1†, Lesley King1†, Louise L Hardy1*†, Andrew Milat1†, Chris Rissel1†, Robyn Newson1†,
Lucie Rychetnik2† and Adrian E Bauman1†
Abstract
Background: There is growing interest by funding bodies and researchers inassessing theimpact ofresearch on
real world policy and practice. Population health monitoring surveys provide animportant source of data on the
prevalence and patterns of health problems, butfew empirical studies have explored if and how such data is used
to influence policy or practice decisions. Here weprovidea case study analysis of how the findings from an
Australian population monitoring survey series of children’s weight and weight-related behaviors (Schools Physical
Activity and Nutrition Survey (SPANS)) have been used,and the key facilitators and barriers to their utilization.
Methods: Datacollection included semi-structured interviews with thechief investigators (n =3) and end-users (n
=9) of SPANS data to explore if, how and under what circumstances the survey findings had been used,
bibliometric analysis and verification using documentary evidence.Data analysis involved thematic coding of
interview data and triangulation with other data sources to produce case summaries of policy and practice impacts
for each of the three survey years (1997, 2004,2010). Case summaries were then reviewed and discussed bythe
authors to distilkey themeson if, how and why theSPANS findings had been used to guide policy and practice.
Results: We found that the survey findings were used for agenda setting (raising awareness of issues), identifying
areas and target groups for interventions, informing new policies, and supporting and justifying existing policies
and programs across a range ofsectors. Reported factors influencing useof the findings were: i) the perceived
credibility of survey findings;ii) disseminationstrategiesused; and, iii) a range of contextualfactors.
Conclusions: Using a novel approach, our case study provides important new insights intohow and under what
circumstances population health monitoring data can be used to influence real world policy and practice. The
findings highlight theimportanceof population monitoring programs being conducted byindependent credible
agencies, researchers engaging end-users from the inception of survey programs and utilizing existing policy
networks and structures, and using a range ofstrategies to disseminate thefindings that go beyond traditional
peer review publications.
Keywords: Government, Policy, Population health, Research
Background researchers in measuring the impact of research. Over
Public funds are expended through health research to the past two decades many theoretical frameworks and
lead to improvements in policy [1,2], practice, resource approaches to measuring research impacts have been
allocation and, ultimately, the health of the community proposed [4], but there have been few empirical studies
[3]. This can only occur if the evidence derived from the exploringhow andwhyresearchisused[5,6].
research is used to inform practice and policy decisions. Population health monitoring surveys form one com-
There is growing interest by both funding bodies and ponent in the public health research cycle, providing a
key source of information about the prevalence and pat-
*Correspondence:[email protected] terns of public health problems [7]. Such information
†Equalcontributors
may assist in guiding appropriate interventions, track
1PreventionResearchCollaboration,SchoolofPublicHealth,Universityof
Sydney,Sydney2006,NSW,Australia changes over time, and support evaluation processes.
Fulllistofauthorinformationisavailableattheendofthearticle
©2013Lawsetal.;licenseeBioMedCentralLtd.ThisisanOpenAccessarticledistributedunderthetermsoftheCreative
CommonsAttributionLicense(http://creativecommons.org/licenses/by/2.0),whichpermitsunrestricteduse,distribution,and
reproductioninanymedium,providedtheoriginalworkisproperlycited.
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page2of11
http://www.health-policy-systems.com/content/11/1/4
That is, population monitoring surveys potentially pro- Methods
vide valuable information to support policy and practice, This study utilized a case study approach to explore if,
although further research is generally required to test and in what ways, a population health monitoring pro-
anddisseminateeffectiveinterventions[8]. gram was used to influence policy and practice, and to
Despite the high investment in monitoring, little is identify the key factors influencing its use. Case study
known about if and how this type of population methods are appropriate for answering ‘how’ and ‘why’
health data is used to inform policy and practice or questions when the phenomenon of interest (SPANS
the key factors influencing its use. We identified only surveys) is embedded within a real-life context (policy
one such study by de Goede and colleagues, exploring and practice environment) [14]. The study was approved
the use of epidemiological research in the develop- by the University of Sydney Human Research Ethics
ment of local public health policy in the Netherlands Committee and all participants gave written informed
[9,10]. This study found that survey data was more consent to take part in the study. An overview of the
often used in a conceptual way to improve under- study methods and key steps in the research process are
standing of the health problem or issue, rather than giveninFigure1.
in a specific and direct way (instrumental use) or to
justify a position or particular course of action (sym- Step1-Researchscoping
bolic use) [10]. Research use was influenced by inter- The impact of SPANS was examined using the theoret-
action between researchers and local health officials, ical framework proposed by Banzi et al. based on a sys-
the personal belief systems of the actors involved, and tematic review of available models [4]. The framework
a range of contextual factors [9]. has five broad impact categories: i) Advancing know-
Further research is required to explore if, and in ledge and research related impacts (peer review articles,
what ways, population health monitoring data is used impact on research methods, better targeting for future
in other contexts and settings. Understanding the fac- research); ii) Capacity building (development of research
tors influencing the use of this type of research is capacity of staff, students, others); iii) Informing policies
important in informing how population health moni- and product development (policy, guidelines, product,
toring systems can be planned and implemented to intervention development); iv) Health and health sector
enhance the uptake of findings into the policy and benefits (improvements in service delivery, effectiveness
program planning process. of services, equity of services, cost reductions, etc.); and,
Therefore, the aim of this paper is to provide a case v) Societal and economic impacts (e.g., improvements in
study analysis of the utilization of the findings from health status, social benefits, shift in knowledge, atti-
an Australian series of population monitoring surveys tudes,behaviors,social capital,macroeconomic).
of school-aged children’s weight and weight-related
behaviors, the Schools’ Physical Activity and Nutrition
Survey (SPANS). Three surveys have been conducted
Step 1: Research scoping
in 1997, 2004 and 2010 in the state of New South 1a: Research impact literature scan –selection of
Wales (NSW), and provide a key source of data on conceptual framework
NSW school children’s weight status, physical activity, 1b: Development of data collection methods
1c:Determine sample frame for interviews
sedentary behaviors and nutrition [11-13]. Funded by
government educational and health agencies since
1997, this monitoring program provides information
Step 2: Data collection
on changes over time in key variables, and can poten-
tially contribute to guiding and monitoring the pro-
2a: Determine sample frame for interviews
gress of public health policy and programs that 2b:In-depth interviews with Chief investigators (n=3)
address children’s weight-related behaviors. However, 2c: In-depth interviewswith end users (n=9)
the extent to which the surveys have actually been 2d: Document verification & bibliometricanalysis
used for these and related purposes, has not yet been
systematically investigated or documented.
The specific aims of this paper are to: i) describe if Step 3: Data Analysis
and how SPANS findings have been used at various 3a:Case study preparation
time points to influence policy and practice; ii) ex- 3b:Impact assessment meeting
plore the key facilitators and barriers to the use of 3c: Identify mediators of impact
the findings; and, iii) critically reflect on the methods
Figure1Overviewstudymethodsandkeystepsinthe
applied to assess research use and make recommen-
researchprocess.
dations for future studies.
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page3of11
http://www.health-policy-systems.com/content/11/1/4
The Banzi framework was chosen as it is based on a (cid:129)Interface withresearchers–howwereend-users
systematic review of existing models and the set of ca- involvedinthe research project,howdid they hear
tegories provide an integrated ‘map’ of a broad range of aboutthe findings (end-usersonly)
potential areas of impact, which largely reflect the range (cid:129)Overallimpact–howhavethefindings beenused
of other commonly used models, for example, the pay- (cid:129)Specificimpacts–capacity building,partnerships,
back framework [15]. We will briefly report on the im- policy and productdevelopment,healthand other
pact of SPANS in relation to categories one and two sectorimpacts,societaland economicimpacts
(advancing knowledge and capacity building), and pro- (cid:129)Circumstancessurroundingtheuseofthefindings,or
vide more detailed results relating to practice, policy and limited impact ofthe findings
broader impacts (categories three to five), as well as (cid:129)Evidenceofimpacts–documentary sources
reportingonthekeymediators oftheseimpacts. (cid:129)Nominationofend-users(investigatorsonly)
Step2-Datacollection Bibliometricanddocumentanalysis
Chiefinvestigatorandend-userinterviews A bibliometric analysis was undertaken in Scopus data-
Semi-structured interviews were conducted with the base to examine the total and mean number of citations
Chief Investigators (CIs) for each of the three surveys (excluding self citations) for all peer review publications
(1997, 2004, 2010) and relevant end-users (EUs) in- arising from SPANS for each survey year. Survey reports
cluding policymakers and practitioners who were in available in the public domain were also examined to
positions to make decisions regarding programs and documentthe key findings andrecommendationsarising
policies related to SPANS findings on school-aged from SPANS. CIs and EUs were also asked to provide a
children’s nutrition, physical activity, sedentary beha- copy of any documentary sources which provided evi-
vior and obesity prevention in NSW. Both the CIs dence of how SPANS findings had been used, such as
and EUs were invited to participate in the interviews policy documents, briefs, reportsand curriculummateri-
by email, with non-responders sent a reminder email als. Additional searches of the grey literature were
after one week and then followed-up by telephone. undertaken to identify documentary evidence of impacts
Interviews were conducted by an experienced identifiedintheinterviews.
research officer (RN) who had a good working
knowledge of SPANS and related policy context, but Step3-Dataanalysis
was independent of the SPANS investigators. All All interviews were transcribed verbatim and coded by
interviews were tape recorded with participants’ one author (RL) using Nvivo qualitative software pro-
permission. gram (NVivo (version 9) Burlington, MA, USA). The
The CI interviews explored their perspectives on the Banzi categories were adapted and expanded for the
overall impacts of the survey, asked about specific coding framework to encompass other categories of
impacts in accordance with the categories outlined in impact arising during the interviews and to code for
the Banzi framework, and any factors contributing to important contextual information. Further, additional
such impacts or lack thereof (List 1). The CIs were also codes were developed to capture content related to key
asked to nominate up to three end-users (EUs), defined factorsinfluencing research use.
as ‘individuals who could provide a perspective on how Detailed case reports for each survey (1997, 2004 and
the monitoring data had been used in policy, practice, 2010) were compiled. These consisted of a summary of:
organizational development, further research or in appli- i) key SPANS survey findings and recommendations; ii)
cations such as guidelines or teaching materials’. the perspectives of CIs and EU on how the survey fin-
Attempts were made to identify EUs from a range of dings had been used and key factors influencing their
sectors including health, education and sport and recre- use; iii) bibliometric analysis; iv) documentary evidence
ation. These EUs were then approached and invited to of impacts; and, v) notes and observations made during
participate in an interview exploring how the SPANS CIandEUinterviews.Thecasesummarieswereprepared
programfindingshadbeenusedfrom theirperspective. bytwoauthors(RLandRN),whowerenotinvolvedinei-
List 1. Semi-structured interview topic guide: investi- ther the implementation or administration of the SPANS
gatorsandend-users surveys,inordertoensurethatthe‘insider’experiencesof
three of the authors who were involved in one or more
(cid:129)Recallofresearch aims,keyfindings andimplications surveys(LLH,LK,AB)werenotundulyinfluential.
(cid:129)Dissemination process(how, factors influencingthe The three case summaries were then independently
dissemination process) reviewed by each of the authors. There was high
(cid:129)Interface withend-users–howresearchteam worked consistency in findings across surveys, allowing for the
with potential end-users(investigatorsonly) distillation of key themes across all three surveys,
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page4of11
http://www.health-policy-systems.com/content/11/1/4
covering how and why SPANS findings had been used in reports,presentationstostakeholdergroups,mediareleases
policy and practice. Attention was also paid to examin- and conference presentations. In total there were 32 peer
ing any differences between surveys, the perspectives of review publications (1997, n = 11; 2004, n = 18; and
CIsandEU,andkeycontextualfactors. 2010 n = 3, at the time of this analysis - July 2012)
whichtogetherhadbeencited720times(Scopus,exclud-
Results ing self citations) with a mean of 22 citations per article
Participantcharacteristics (SD: 34, range: 0 to 140, H index: 12: Scopus). The key
All three of the survey CIs approached agreed to partici- findings and recommendations from each survey, along
pate in the interview. A total of 14 EUs were nominated with the dissemination process that was employed and
and approached, and nine agreed to participate. Of the surveyfundingsourcearepresentedinTable2.Whiledis-
non-participant EUs, two did not respond to the invita- semination processes varied for each survey, in each case
tion, two were not available during the data collection they included presentations to health and education pro-
period and one had moved on from the policy area and fessionals(mostlyhealthpromotionstaffand teachers),as
therefore did not feel able to comment on the survey well as senior NSW policymakers in health, education,
impacts. The EUs worked in a variety of sectors inclu- sportsandcommunitysectors.
ding education, health, and sport and recreation, and in Almost all EUs reported that they had direct involve-
most cases had direct responsibility for issues related to ment in the administration or funding of the survey
child obesity prevention (Table 1). A total of 16 inter- from its inception as well as the dissemination process,
views were conducted, with CI interviews lasting longer including the drafting of key recommendations. Some
(mean; 64 minutes; range 53 to 70 minutes) than EU EUs also took an active role in disseminating the fin-
interviews (mean; 50 minutes; range 17 to 87 minutes). dingswithin theirsector,asdiscussedbythisEU:
Not all CIs and EUs were in a position to comment on
theimpactofeachsurvey.Table2showsCIandEUpar- “...aftereachoftheSPANSsurveysthatcameout...we
ticipantsbysector,roleandsurveyyear. hadtheseworkshops...sowewereprovidedthedataand
theinformationtoourindustry,buttriedtodoitthrough
Researchimpacts ameaningfulwaysotheycanapplyitwhentheygointo
Disseminationandadvancingknowledge schoolsthemselves.”(EU2,2004and2010surveys).
In general, the survey findings were considered to advance
knowledge through the findings being widely reported in CIsandEUsdiscussedthatthedisseminationofSPANS
journal articles, and extensively disseminated through key findings was important in raisingoverall awareness of the
Table1Participantcharacteristicsbysectorandsurvey
Participant Participantsectorandrole Survey
1997 2004 2010
CI1 Academicresearcher ✓ ✓
CI2 Academicresearcher ✓
CI3 Academicresearcher ✓
EU1 Health1,2,3,4 ✓ ✓
EU2 Communitysportandrecreation2,3,4 ✓ ✓
EU3 Education3,4 ✓ ✓
EU4 Education4 ✓
EU5 Education4 ✓
EU6 Health1,2,3,4 ✓
EU7 Health4 ✓
EU8 Health4 ✓
EU9 Health1,2,3,4 ✓
Totalnumberofinterviews 12 2 7 7
CIinterviews:3 (1CI (2CIs (1CI
EUinterviews:9 1EU) 5EUs) 6EUs)
1WorkedinpolicybranchresponsibleforfundingSPANS.
2Memberofatleast1SPANSAdvisoryCommittee.
3MemberofGovernmentChildObesityPreventionImplementationCommittee.
4SPANSinformationrelevanttoworkarea.
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page5of11
http://www.health-policy-systems.com/content/11/1/4
Table2Keyfindings,implicationsanddisseminationprocessfortheSPANSmonitoringprogram
KeyFindings KeyFindings Dissemination FundingSource
Process
1997SchoolFitnessandPhysicalActivitySurvey
Poorfundamentalmovementskills(FMS)amongst NeedtoaddressFMSandphysicalactivityin Mainandsummary Departmentof
schoolstudents schools reportavailableon Educationand
website Training
Lowlevelsofphysicalactivity,particularlyamongst Surveyshouldberepeatedperiodicallywith Summaryreport
adolescentgirls comparablemeasures senttoallNSW
schools
Fewprimaryschoolshaveteacherswithtrainingin Presentationstoa
sportandphysicaleducation rangeofend-user
groups
Peerreview
publications(n=11,
totalcitations1=
589)
2004SchoolPhysicalActivityandNutritionSurvey
Almost25%ofstudents5–16yearswere FocusonFMSworking,butneedstobecontinued Main,shortand NSWDepartment
overweightorobese,risingfrom1997 summaryreport ofHealth
availableonwebsite
Wideningofthegapinprevalenceofoverweight Needtolimitsmallscreenuseto2hoursperday 40Regional
andobesityamongstlowsocio-economicstatus stakeholder
andculturallyandlinguisticallydiversegroupssince workshopsacross
1997 NSW
FMSandphysicalactivitylevelshadimproved Reinforcedtheneedtoimplementgovernment Mediareleasesand
significantlysince1997surveybuthighscreentime policythatschoolsshouldoffer2hoursofplanned interviews
use physicalactivityperweek
Chronicdiseaseriskfactorscommonamongst Effortstoincreasephysicalactivitypromotionin Peerreview
overweigh/obeseadolescents schoolsandcommunityshouldbecontinuedand publications(n=18,
increased totalcitations1=
131)
Highintakeofenergydensefoodsanddrinksand Limitconsumptionandpromotionofenergydense
lowlevelsofvegetableconsumption nutrientpoorfood
Implementstrategiestotarget‘high-risk’groups
2010SchoolPhysicalActivityandNutritionSurvey
Prevalenceofoverweightandobesitystabilized Continuetoimplementschoolcanteenprogram, Main,shortand NSWDepartment
since2004 focusonFMSandprovideminimumof2hoursof summaryreport ofHealth(now
plannedphysicalactivityperweekinschools availableonwebsite MinistryofHealth)
Socioeconomicandculturaldisparitiesin Implementprogramstosupportparticipationin Mediareleasesand
prevalenceofoverweight/obesityremain. communitysport interviews
Highconsumptionofenergydensenutrientpoor Advocatefornationalregulationstolimitthe Peerreview
foodsanddrinksandlowlevelsofvegetable marketingofunhealthyfoodstochildrenand publications(n=4,
consumption policiestopromoteactivetransport totalcitations1=1)
Declineinproportionofstudentsmeetingphysical Widespreaddisseminationofconsistentmessages Presentationsto
activityguidelines toparents/familiesregardinghealthylifestyle stakeholdergroups
behaviors (withmoreto
follow)
FMSshowedimprovementinsomeskillsand Importanceoftargetingfamilybehaviorsandearly
declineinothers childhoodsectortoreduceobesityinpreschoolers
Highscreentimeuse
Overweightandobesityincreasingamongchildren
enteringfirstyearofschool(~5yearsold)
1Excludingselfcitations.
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page6of11
http://www.health-policy-systems.com/content/11/1/4
issueofchildobesityamongstthegeneralcommunityand Nevertheless, one EU suggested that SPANS may have
keystakeholderssuchtheeducation,health,andsportand contributed to improvement in FMS over time because
recreation sectors, as well as highlighting areas for inter- of the change in curriculum focus and resources pro-
ventionandfutureresearch. vided to schools following the 1997survey. Others dis-
cussed that SPANS may have contributed indirectly to
Capacitybuildingimpacts the leveling-off of childhood obesity between 2004 and
The surveys had considerable impact on building re- 2010, as new policies and programs arose, in part, be-
search capacity with two PhDs, two masters/honors re- cause ofthesurveyfindings.
search projects and two postdoctoral positions arising
from SPANS. For EU’s the survey provided professional “I think the fact that obesity has plateaued should
development opportunities for the 55 school teachers be taken that all of the investments that have
who were involved in data collection, including up- been happening...well at least since SPANS
skilling in the measurement of fundamental move- 2004...are obviously having some degree of
ment skills, cardiorespiratory fitness and anthropom- impact” (CI3).
etry. Further, the survey methodology was used to
inform the methods for other research studies and Documentary checks were undertaken and documen-
survey programs at both the state and national level tary evidence identified relating to the majority of docu-
(Additional file 1). ments and initiatives referred to by CIs and EUs. Key
documentsarelistedintheAdditionalfile1.
Policyandpracticeimpacts
Table 3 presents the key content themes regarding FactorsinfluencingtheuseofSPANSfindingstoinform
ways in which the survey findings were used to inform policyandpractice
policy and practice, as well as illustrative quotes. At a An analysis of key perceived facilitators and impedi-
policy level, this ranged from broad agenda setting and ments to the use of the SPANS findings in influencing
policy debates on the importance of childhood obesity, policy and practice revealed three main inter-related fac-
to underpinning specific new policies such as banning tors: i) the perceived credibility and trustworthiness of
soft drink sales in schools. Each survey had specific the survey findings; ii) the dissemination processes used;
policy impacts (e.g., 1997 findings informed the devel- and, iii) a range of contextual factors (illustrative quotes
opment of a policy to focus on fundamental movement about these factors are presented in Table 4). In 1997,
skills (FMS) in schools, and 2004 findings underpinned the survey was funded by government as part of a broad
the ban of soft drink sales in schools). The EUs and CIs child health and fitness agenda, and from 2004, as part
identified a range of policy documents which cited or of a government policy response to monitor child
applied SPANS findings (Additional file 1). obesity. As such the survey content and findings were
At the practice level, SPANS findings were used to in- designed to be closely aligned to the interest of policy
form program planning across a range of sectors. In the makers and key EU groups such as health, education,
education sector, the findings led to a standardized ap- and community sport and recreation sectors. The survey
proach to teaching FMS in schools and underpinned the was perceived by EUs interviewed to be a trustworthy
rationale for new initiatives in the community sports source of data as it was conducted by an independent
sector (such as healthy sports canteens). The findings andcredible researchgroup.
provided support for some existing health promotion Partnerships between EU groups and the CIs oper-
programs, and informed the development and refine- ated from the inception of the SPANS program, and
ment of educational resources both within schools more regular and specific policy forums were estab-
and for the broader community. The impact on practices lished as part of the NSW ‘whole of government’
across sectors largely flowed on from policy impacts. approach to addressing child obesity in 2003 [16].
For example, the policy focus on FMS in schools led These policy forums provided the continuity and
to the development of new curriculum materials, mechanisms for survey findings to be disseminated
resources and professional development for teachers and considered, and for EU groups to become
(Additional file 1). engaged in formulating the key policy recommenda-
tions and the wider dissemination processes. There
Broaderhealth,economicandsocietalimpacts were, however, some barriers identified to the use of
Most interview respondents found it difficult to com- the findings, including some related to the nature of
ment on the broader health, economic or societal the survey itself, in that it mainly identified the pro-
impacts of SPANS, because of the long term nature and blems and target groups, but did not provide specific
the multiple factors contributing to such impacts. guidance on solutions, and the reports were not
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page7of11
http://www.health-policy-systems.com/content/11/1/4
Table3HowtheSPANSprogramfindingswereusedtoinformpolicyandpractice
KeyImpacts IllustrativeQuotes
PolicyImpacts
(cid:129)Agendaandprioritysettinge.g.,attractingfundingtotheissueof “Iguessitsortofdrives,tosomedegree,theprioritiesinthe[government
childobesityprevention,identifyingprioritygroupsandsettingsfor obesity]plan....aretheactionsintheplantherightones,aretheyworking?...
intervention Soitgivesusabitof...anideaofwhetherornotthere’sgapsthathaven’t
beenaddressed,intermsofthebroadercrossgovernmentstrategy....arethere
(cid:129)Informedpolicydebatese.g.,datausedinbriefingswithhealth areaswherewecouldpullbackon,investmore,focusinanotherarea.”(EU9,
ministertoinformparliamentarydebates
2010survey)
(cid:129)Informedpolicyplanninge.g.,identifyingareasforinvestment,de- “....there’shighlevelsofscreentime,soIguesswecankindofuseitasa
investmentandstakeholderinvolvementingovernmentobesityplan basisorrationaleforourpolicyinput,andalsoforhowweapproachdifferent
issueswithdifferentgovernmentagencies.”(EU9,2010survey)
(cid:129)Directlyunderpinnednewpolicye.g.,banningsoftdrinksalesin
schools “Iknowthatthephysicalactivitycomponents[1997survey]werealsousedto
advocateforfundamentalmovementskilldevelopmentwiththeDepartment
(cid:129)Indirectlyusedtoadvocatefornewpolicye.g.,foodmarketingto ofEducation.”(EU1,1997&2004surveys)
children
“SPANSwouldcertainlyhavebeenoneofthethingsthatcontributedtothe
(cid:129)Usedtosupportexistingpolicye.g.,afterschoolphysicalactivity developmentofthatpolicy[nutritioninschools].”(EU5,2010survey)
programsforchildren
“thestatsthataresoterriblyimportant,particularlywhenyou’retryingtosell
something”(EU4,2010survey)
(cid:129)Policyevaluatione.g.,performancemonitoringtoolforgovernment “AndtheEducationDepartmentwasparticularlyinterestedtoseewhathad
obesityplan happenedintermsoffundamentalmovementskills,becausethey’dmadea
significantinvestmentinfundamentalmovementskilleducationinprimary
andhighschools,sotheywerereallyinterestedinwhathadhappenedthere.
AndanotherpartofitwasanevaluationoftheHealthyCanteenstrategythat
hadbeenputinplace,maybeayearortwoearlier.Andtherewasalsoan
aspectofthestudythatwaslookingattheschoolenvironmenttoseeagainif
therehadbeenanychangesfollowingthepreviousstudy.”(CI3,2004survey)
“SPANSjustshinesalightonwhat’snotworking.”(EU2,2004&2010surveys)
PracticeImpacts
SchoolSector “...theinitialresearchthatwasdone[1997survey]...wegotverystrong
agreementaroundthosefundamentalmovementskills,andwehadsome
(cid:129)Informedcurriculumdevelopmente.g.,standardizedapproachto
clearsupportmaterialsdevelopedforschoolsandtheGetSkilledGetActive
teachingFMS DVD,whichisaresourcethatwestillpromoteinschools.”(EU3,2004&2010
(cid:129)Leadtonewcurriculumresourcese.g.,standardizedteaching surveys)
materialsforFMS
(cid:129)InformedProfessionaldevelopmentforteacherse.g.,measuring
FMS,raisingawarenessofchildobesity
(cid:129)Informedandsupportedexistinghealthpromotionprograms
e.g.,healthycanteens
SportandRecreationSector “...the[sport]developmentofficernetwork,theylovedit[surveydata]and
theygotit.Particularlythefundamentalmovementskills,becausethey’reout
(cid:129)Informedsportscoachingprogramse.g.,howFMSaretaughtin thereteachingthem,throughsport,everyday...they’venowrescheduledtheir
somesports formatofhowtheygointoteachgirlsRugby...theydothekickinglast
(cid:129)Supportedrationalefornewprogramsorpilotse.g.,healthy becausetheyknowtheirstrongerskillisintherunningandthecatching.So,
sportscanteens it’sreallypractical...”(EU2,2004&2010surveys)
CrossGovernment/CommunitySector “...whenit[2004survey]showedthat20%ofkidswhentheystartedgrade1
werealreadyfat,italsogaveemphasistodostuffinchildcare...thatwas
(cid:129)Informeddevelopmentandrefinementofeducationalresources reallycompelling...wewouldhaveneverknownthatifthatsurveywasn’t
e.g.,websiteonhealthylivingforparentsandgeneralcommunity[35] done.”(EU6,2004survey)
“SPANS,asIsaid,isamomentintimethatgivesyoutheheads-uponwhat
areasyoushouldbefocusingoninyounextplan;that’swhatitshouldbe
doing.”(EU2,2004&2010surveys)
HealthSector “IthinkSPANScontributedtohavingtreatmentprogramsiswhattheyare–I
callthemtreatmentprogramsforpeoplewithalready–with–withalready
(cid:129)Informedprogramplanninge.g.,choiceoftargetgroupsand withtheproblem.”(EU6,2004)
settingsforinterventionandavailabilityoftreatmentprograms
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page8of11
http://www.health-policy-systems.com/content/11/1/4
Table4FactorsinfluencingtheuseoftheSPANSfindingstoinformpolicyandpractice
Facilitators Barriers
ResearchQualityandContent
(cid:129)Surveyprogramperceivedtobeofhighmethodologicalquality: (cid:129)Surveyfindingsdidnotprovidealltheanswers:neededtobe
longitudinalandconductedbyanindependentandcredibleresearch consideredalongsideevidenceabouteffectiveinterventions
team
“...ofcoursejustmonitoringisnotasolution”(EU6,2004)
(cid:129)Surveyprogramperceivedtobealignedtotheprioritiesofpolicy Lackofspecificityofthedata:Unabletoprovidedataatregional/
makersandpractitionerswithadaptationsmadeovertimetomeet sectorlevel
needs
“IthinkSPANScannotstandalone,ithastohaveanevidencesummaryof
“theMinistry/Departmenthasworkedreallyhard...tomakesurethat...the interventionsbehindittoinformgoodpolicy”(EU2,SPANS2004&2010)
questionsthatthesurvey’saskingare,Isuppose,therightonesforthe
priorityareasaswell...SoIthinkovertimethesurveyshavekindof
adaptedtohavedifferentkindsofquestionsinthem...it’sbeenpurposefully
closelyalignedwithpolicyprioritiesandgovernmentprioritiesandIthink
thathelps”(EU9,2010survey)
DisseminationProcess
(cid:129)Useofactivedisseminationstrategiese.g.,discussionoffindingsat Lackofreportstailoredtospecificend-usergroups/sectors
workshopsbetweenresearchersandend-users. highlightingkeyimplicationsofthefindings
“...therewasamajordisseminationstrategyamongstasetoforgs”(EU6, “...thereprobablyshouldbeareportforpractitioners...Whatdoesthis
2004) meantoyou?”...
(cid:129)Arangeof‘knowledgetransfer’productsproducede.g.,short Whatdoesitmeanforateacherwho’s55,hasneverplayedsportbefore....I
reportshighlightingkeyfindingsandrecommendations. thinkifwereallywanttogetgoodatthis,takeSPANSandwriteitfor
differentaudiences...SoSPANSforacademia,SPANSforpolicy,SPANSfor
(cid:129)End-usersactedas‘knowledgebrokers’facilitatingdisseminationof thegeneralpublic”(EU2,2004&2010surveys)
SPANSfindingswithintheirsector
“....we’vedonecanteennewslettersandthingslikethatandwe’vereported
SPANSresultsandwereferenceitthere”(EU5,2010)
(cid:129)Activeengagementofthemediaresultinginhighlevelsofmedia
coverage,publicdebateanddiscussion
“Ialsositonthe[policy]groupforobesity,sothereportwasdiscussedona
numberofoccasionsatthatmeetingaswell.Youseethemediareports
they’vegenerated...wereceive...draftcopiesofthereportaswell,interms
ofthediscussionsthatwehadwithHealth”(EU5,2010survey)
ContextualIssues
(cid:129)Supportivepolicycontextforaddressingchildobesitywiththe Politicalinstabilityandpoortiminge.g.,frequentchangesinministerial
releaseoftheSPANSfindingsfittingwellwithsomepolicycycles positionsandpoorfitwithsomepolicycycles
(governmentobesityplanandplanningofnationalcurriculum)
Limitedsectorcapacityandresourcese.g.,lackoffundstoimplement
(cid:129)ContinuityandPartnershipsbetweenresearchersandend-usersfrom thefindingsinsomesectorsatcertaintimepoints
theinceptionofSPANSprogramfacilitateddissemination,ownershipand
“.....thenthere’ssomeinfrastructureissuesweneedtolookat;thereneeds
useofthefindings
tobesomeresourcing”(EU2,2004&2010surveys)
(cid:129)Mechanismsandstructuresinplacetoimplementrecommendations “...probablypriorto2002...therewasarelativelylimitedamountof
e.g.,Policyrelevantforumsinvolvingkeyend-users fundinginthisaspectofHealthPromotion-...Sointermsofinfluencing
programsitwouldhavebeendifficult‘causewe’ve...nomoneytoinclude
(cid:129)Goodfitwithorganizationalcultureandwaysofworkinge.g.,value programs”(EU1,1997&2004surveys)
placedonhavingan‘evidencebase’forpracticeandpolicydecisions
amongstend-usergroups
“IthinktheDepartmentofEducationknewofthefindingsorthe
recommendationsthatweregoingtobemadebecausetheyalsohad
ownershipofthose....theyhadtoagreeonthoseforthedraftreport.SoI
thinkbythatstagethey’dalreadysaid:Wewantthisrecommendationhere
andthisishowwe’regoingtorespondtoit.”(CI,1997&2004surveys)
written for specific EU groups. A number of context- Discussion
ual factors were also identified as impediments to ap- This is one of the few studies to document the temporal
plying the findings, including ongoing organizational impact of a government population health monitoring
changes in government agencies, and limited capacity program. We found that SPANS was successful in
and resources to implement recommendations among advancing knowledge and understanding of the issue of
some sectors at various points in time. child obesity, weight-related behaviors, building capacity
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page9of11
http://www.health-policy-systems.com/content/11/1/4
oftheacademicresearchersandsomeEUs,andinfluencing the formation of partnerships between survey resear-
policyandpracticedecisionsacrossarangeofsectors.The chers and EUs, and provided mechanisms and structures
actual policy and practice impacts were in line with key (such as the NSW Government Action Plan for Child
recommendations arising from the survey reports. The Obesity Prevention 2003–2007, [16] and a Senior Offi-
positive impacts of the survey findings were attributed to cers Group with responsibility for overseeing implemen-
the perceived credibility of the survey by EU groups, the tation of this plan) which facilitated application of
processes used to disseminate the findings and a range of survey findings. In addition, ongoing media coverage of
contextualfactors.Contextually,thesurveyhada support- child obesity throughout this period reinforced the poli-
ive government policy, strong partnerships between CIs ticalandsocial relevance ofthese surveys [25].
andEUs,andestablishedmechanismsfordisseminationof The findings highlight the importance of researchers
thefindingsacrossarangeofgovernmentsectors. tapping into existing policy networks, structures and
This case study demonstrates the positive impact a processes,utilizing policy windows and building partner-
population health monitoring program can have on a ships and trust with policy makers and other EUs over
range of policy and practice decisions. In particular, the time. This is in accordance with the findings of Innvær
findings of this study have demonstrated that population and colleagues who found that personal contact and
health monitoring data can be used as an advocacy tool timely relevance were the most commonly reported
(to attract attention and funding to an issue), priority facilitators of research use in a systematic review of 24
setting (identifying areas and target groups for interven- studies of the use of evidence by health policy makers
tion), support and justify existing programs/approaches, [22]. Additionally, our findings point to the importance
or identify the need for alternatives. While others have of the dissemination process in influencing the use of
suggested that epidemiological research can be used in a findings from population health monitoring surveys, in
conceptual way to improve understanding of a health particular, the use of active dissemination strategies,
problem [9,10], this case study found that the survey such as workshops and presentations, to encourage con-
findings were also used instrumentally to directly under- tact and interaction between academic researchers and
pin new policies (such as banning soft drink sales in EUs. The use of a range of other dissemination mecha-
schools), and symbolically to justify existing policies and nisms such as short summary reports in lay language
programs (such as introducing healthy school canteens), andmediareleases,aswellastraditionalacademicpubli-
in addition to improving understanding of the issues cations, were all shown to be important in facilitating
associated withchildobesity(conceptualuse). use of the findings by EUs. There is increasing emphasis
The instrumental and symbolic use of the survey fin- from funding agencies on making research evidence
dings in influencing policy decisions may reflect the fact readily available, however recent studies of public health
that the EUs interviewed in this study were directly research suggest that most dissemination activity beyond
involved in the development and implementation of po- publishing academic papers appears to be undertaken in
licies and programs relevant to the survey findings. In an ad hoc, unfunded fashion and that access to dissem-
contrast, the study by de Goede and colleagues involved ination advice and support for researchers from funding
a broader range of EUs, most of whom were not directly agencies and academic institutions is lacking [26,27].
involved in execution of health surveys. The use of the This suggests that more emphasis should be placed on
survey findings may also reflect the political imperative funding and supporting a range of dissemination acti-
to take action to reduce child obesity, particularly in the vities on behalf of both funding bodies and academic
early stage of child obesity prevention responses when institutionsthemselves.
there was an absence of other evidence on effective Traditionally, tools for measuring research impacts
intervention approaches [17,18]. It is important to note, and assessing ‘research excellence’ are based on the
however, that epidemiological evidence in itself does not number and significance of academic publications (such
provide solutions to health problems, and needs to be as bibliometrics and journal impact factors). However,
consideredalongside thebroaderbody ofevidenceabout there is growing interest in assessing actual impact on
effective intervention and policy approaches, as high- real world policy and practice. While a number of theor-
lighted bymany oftheEUsinthis study. etical frameworks for assessing impact have been pro-
It is clear from this study that population monitoring posed [5,28-32], the findings of our study suggest that
surveys are one of many factors influencing public the impact categories proposed in such frameworks, in-
health policy and practice. In line with previous research cluding that of Banzi et al. [4], can capture the practical
[5,6,19-24], we found that a range of contextual factors ways in which survey data is utilized. However, the fin-
were critical in facilitating the use of the survey findings. dings also indicate that it can be difficult to attribute
In particular, the 2002 NSW Child Obesity Summit [18] impacts to a single piece of research, particularly the
provided a supportive policy context which facilitated longer term societal, health and economic impacts,
Lawsetal.HealthResearchPolicyandSystems2013,11:4 Page10of11
http://www.health-policy-systems.com/content/11/1/4
which almost always arise for a complex array of to influence policy and practice. This ranged from rais-
contributing and contextual factors. This highlights the ing awareness of the issue (i.e., child obesity), identifying
need for alternative ways of conceptualizing research priority areas and target groups for interventions,under-
‘impacts’. The use of ‘contribution mapping’as proposed pinning new policies and supporting and justifying exist-
by Kok and colleagues [33] may provide an alternative ing policy and/or programs. Our findings highlight the
way forward. This approach aims to assess ‘contribu- importance of survey monitoring programs being con-
tions’ of research as opposed to ‘impacts’and focuses on ducted by credible independent agencies, the use of a
understanding how research and knowledge utilization range of dissemination strategies that directly engages
processes evolve amongst the key actors involved. Fur- EUs, and the need for strong partnerships with policy
ther clarity is also required by researchers and funding makers and EUs from the inception in order to increase
agencies about the intended contribution that popula- ownership over the findings and commitment to action.
tion health monitoring surveys are expected to make to Population monitoring data, however, needs to be
policy and practice, in order to assess the degree to considered alongside other evidence about effective
which thiswasachieved. intervention approaches to maximize its influence on
This study has a number of strengths and limitations. practiceandpolicydecisions.
In terms of strengths, we assessed impacts using mul-
tiple methods, including bibliometric analysis, interviews Additional file
with survey researchers and EUs, and documentary
checks. This allowed for triangulation of data in the
Additionalfile1:DocumentaryEvidence:Listofreportsand
form of case summaries. The documentary checks lend documentscitedbyCIsandEUs(excludedpeerreview
confidence that the perspectives of the EUs are credible. publications).
While theSPANSreportshavebeen disseminatedwidely
across the NSW education sector, the full range of users, Abbreviations
CIs:Chiefinvestigators;EUs:End-users;SPANS:Schools’PhysicalActivityand
and their responses and application of findings, are
NutritionSurvey.
unknown, although likely to be highly variable. It must
be acknowledged that the EU interviewees were pur-
Competinginterests
posefully sampled on the basis of their role as the pri- Fouroftheauthorsofthecurrentstudy(LLH,LK,AB,AM)wereeitherCIsor
mary target audience and having direct responsibility for contractmanagersoftheSPANSsurveys.Inordertomanagethispotential
conflictofinterest,theseindividualswereexcludedfromconducting
child obesity prevention policy initiatives. There may
interviewswithCIsorEUsandwerealsoexcludedfromtheproductionof
also have been some degree of social desirability re- casesummarieswhichprovidedassessmentofresearchimpacts.The
sponse bias, whereby EUs felt obliged to report positive potentialconflictsofinterestwereacknowledgedbytheindividualsatthe
ImpactAssessmentMeetingandwerefactoredintocommitteedeliberations
or over-inflated impacts. We attempted to reduce social
whendesigningthisstudy.
desirability response bias by having researchers not pre-
viously involved in SPANS conducting the interviews Authors’contributions
and undertaking the analysis. The recall of impacts was RL,LK,LLH,CRandAMconceivedthestudyanddesignedthemethods.RN
wasresponsibleforconductingtheinterviewsandcollectingdocumentary
somewhat problematic for the 1997 survey, as this was
sources.RLundertookanalysisofthecasestudydata,producedthecase
conducted over 15 years ago and only one EU (out of summariesanddraftedthemanuscript.Allauthorscontributedtodata
three) was available to be interviewed. Hence we are un- interpretationandhavereadandapprovedthefinalmanuscript.
able to ascertain whether the impacts of the 1997 survey
Acknowledgements
may be underestimated or overestimated in this case
ThisworkwassupportedbyaCapacityBuildingInfrastructureGrantfrom
study. Overall, there has been a high degree of continu- NSWMinistryofHealth.Theauthorswouldliketothankparticipantsfortheir
ity in the CIs involved in SPANS and EUs in NSW child timeandcontributiontothiswork.
obesitypreventionoverthestudyperiod,whichmayalso
Authordetails
have increased difficulties in attributing influence to any 1PreventionResearchCollaboration,SchoolofPublicHealth,Universityof
single one of the three surveys, and lent a positive bias. Sydney,Sydney2006,NSW,Australia.2SchoolofPublicHealth,Universityof
Sydney,Sydney2006,NSW,Australia.
We recognize that there is no single or straightforward
way of determining the full range of impacts across mul- Received:16August2012Accepted:16January2013
tiple potential EUs; however this case study provides a Published:30January2013
practical illustration of how a Government population
References
surveycan beused.
1. AndersonWP,PapadakisEM:Researchtoimprovehealthpracticeand
policy.MedJAust2009,191(11):646–647.
Conclusions 2. CookseyD:AreviewofUKhealthresearchfunding.London:HMSO;2006.
3. CampbellDM,RedmanS,JormL,CookeM,ZwiAB,RychetnikL:Increasing
The SPANS case study provides important new insights
theuseofevidenceinhealthpolicy:practiceandviewsofpolicymakers
into how population health monitoring data can be used andresearchers.AustNewZealandHealthPolicy2009,6:21.