Big science : public investment in large scientific facilities

2007·6 MB·English

by  Great Britain. National Audit Office

i
SSNWS
National Audit Office
Big Science: Public investment In
large scientific facilities
RT BY THE COMPTROLLER AND AUDITOR GENERAL |H C 153 Session 2006-2007 |2 4 Janue
OE ET ET ;
ee oO
General Collections
The National Audit Office serutinises
public spending on behalf of
Parliament. The Comptroller and
Auditor General, Sir John Bourn, is
an Officer of the House of Commons.
He is the head of the National Audit
Office, which employs some 850 staff.
He, and the National Audit Office, are
totally independent of Government.
He certifies the accounts of all
Government departments and a wide
range of other public sector bodies;
and he has statutory authority to report
to Parliament on the economy,
efficiency and effectiveness with
which departments and other bodies
have used their resources. Our work
saves the taxpayer millions of pounds
every year. At least £8 for every
£1 spent running the Office.
National Audit Office
Big Science: Public investment in
large scientific facilities
fan seine
# €
Bo MSF Lak ae
OReMT yty e Se BTr ar! gonlyy
wee me
f
NA Louw ce.
LOAGMEHYIAg OS EINE
Ordered by the
LONDON: The Stationery Office House of Commons
tok. oU) to be printed on 22 January 2007
SUIT
22502929944
REPORT BY THE COMPTROLLER AND AUDITOR GENERAL | HC 153 Session 2006-2007 |2 4 January 2007
This report has been prepared under Section 6
of the National Audit Act 1983 for presentation
to the House of Commons in accordance with
Section 9 of the Act.
John Bourn
Comptroller and Auditor General
National Audit Office
15 January 2007
The National Audit Office
study team consisted of:
David Howes, Tim Bryant, Daisy Hodgson,
Thomas Mirfield, Santosh Gora, Sacha Bakhtiar
and Anushiya Kugathas, under the
direction of Peter Gray.
This report can be found on the National Audit
Office web site at www.nao.org.uk
For further information about the
National Audit Office please contact:
National Audit Office
Press Office
157-197 Buckingham Palace Road
Victoria
London
SW1W 9SP
Tel: [phone]
Email: [email protected]
© National Audit Office 2007
CONTENTS
EXECUTIVE SUMMARY PART TWO
Delivering projects
PART ONE
Prioritising investments in large Using the Gateway review process
scientific facilities
Project management
Responsibilities for investment in large 10 Defining target outcomes and outputs
scientific facilities
Delivering on budget - capital costs
Processes for prioritising and funding 1]
large scientific facilities Delivering on budget - costs in use
Evaluating scientific benefits Delivering on time
Evaluating economic benefits Procurement strategies
Evaluating costs Planning for operations
Handling international collaborations
Managing the demand for funds APPENDICES
1 Study methods 26
2 Performance measurement case study 28
— the ISIS neutron source
3 Project summaries 30
ENDNOTES 48
Photographs courtesy of Diamond Light Source Ltd, Biotechnology and Biological Sciences Research Council,
Council for the Central Laboratory of the Research Councils, Engineering and Physical Sciences Research Council,
Medical Research Council and Natural Environment Research Council.
1. The Government invests in a range of large 2 The Department has earmarked £830 million
‘scientific facilities to support and develop the nation’s from the central fund to 15 prioritised projects as set
science base. Since 2000 the Department of Trade out in Figure 1 on page 6. Once prioritised, these
and Industry's (the Department) Office of Science projects progress by presenting science and business
and Innovation has established new arrangements cases to secure project approval. To date 10 projects
to co-ordinate planning for large facilities. The main have received Departmental approval. Part 1 of this
components are a road map describing large facilities report evaluates the strength of current processes for the
which UK scientists are likely to need in the next 10 to identification, appraisal and prioritisation of potential
15 years, and a central fund (the Large Facilities Capital investments in large scientific facilities. Part 2 evaluates
Fund) of around £100 million per year to support performance in delivering those facilities prioritised for
investments in such facilities by Research Councils. The support. Ten projects examined in detail in this report are
Research Councils are the Arts and Humanities Research outlined in Figure 2 on pages 8 and 9. In total they have
Council (AHRC), Biotechnology and Biological Sciences a capital budget of £1 billion. The study methods are
Research Council (BBSRC), Council for the Central summarised at Appendix 1.
Laboratory of the Research Councils (CCLRC), Economic
and Social Research Council (ESRC), Engineering and
Physical Sciences Research Council (EPSRC), Medical
Research Council (MRC), Natural Environment Research
Council (NERC) and Particle Physics and Astronomy
Research Council (PPARC).
BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES
SUMMARY
3 __U ltimately, the value for money to be obtained 6 Our detailed conclusions are as follows:
from these facilities will depend on the scientific
Current arrangements identify potential projects over
discoveries they help make and the effective exploitation
a sensible planning horizon, allow prioritisation
of that science. Our study concludes that, though such
across the science base, and are delivering a
outcomes will always remain uncertain, the current
significant programme of new or replacement large
arrangements should deliver a significant contribution to
scientific facilities. The road map approach was
the development of the nation’s scientific infrastructure.
pioneered by the United Kingdom and has been
The arrangements include the development of a common
widely commended and adopted by other countries.
plan known as the road map. This is co-ordinated by
Research Councils UK — a partnership of the research m Prioritisation and assessment strongly reflect the
councils — which addresses priorities which cut across primary policy objective of advancing scientific
scientific disciplines. The road map has allowed scientific knowledge, but economic potential and possible
priorities to be considered in a more systematic way exploitation by industry are less fully analysed.
across disciplines. Working within HM Treasury’s Green
| Current arrangements concentrate attention on
Book guidance on project appraisal and evaluation, '
availability of capital funding to build facilities but
special attention now needs to be paid to strengthening
there are significant weaknesses in assessing their
the information available to support choices between large
ongoing costs, and the impact (normally in future
facility projects.
spending review periods) of meeting those costs on
the balance of Research Council funded activities.
4 __ The projects we examined had generally been
established in accordance with good practice principles m Atprioritisation estimates of costs and assessments
and standards as set out in methodology advocated by of benefits are preliminary, yet priorities are not
the Office of Government Commerce for managing reviewed if costs or benefits are significantly revised
projects, called PRINCE2.? More consistent application of as business cases are prepared. Opportunities to
that Office’s Gateway reviews to the key stages of every maximise the overall science benefits of the portfolio
project, would ensure all teams benefit from wider sources of projects may thus be missed.
of advice on areas such as project management, project
m Procurement strategies have been adapted to
costing and funding and procurement options.
the particular circumstances of each project.
Future projects can benefit from better sharing of
5 It is still early to assess fully the portfolio of projects
experience and lessons learned.
against delivery to time and budget. To date, performance
against the approved capital budgets suggests some good m= More attention needs to be given to specifying from
budget management, for example on the James Cook the start how the success of individual projects will
research ship, but also projects where forecast capital be assessed and measured, drawing on examples
costs already exceed budget even though still at an from some current and existing projects. This should
early stage. More significantly, some project teams have enable a fuller assessment of value for money to
significantly underestimated the likely running costs of be made once facilities are operational and inform
facilities once they are delivered. In addition, more work appraisal of future potential projects.
is needed by Research Councils to examine the potential
impact of these facilities on the future demand for research
funding, their capacity to support any new demand, or
the effect of doing so on other areas of activity. Full use of
these facilities will depend on research ideas competing
successfully for research funding, through peer review,
against other calls on limited Research Council budgets.
As the new facilities come on stream, the Research
Councils will need to monitor the impact on the demand
for research funding and ensure lessons are learned for the
appraisal of similar facilities in the future.
BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES 5
EXECUTIVE SUMMARY

900Z UWINyND 4D sD sNyDIS JOJO] joyido> (uo 3) Buluolss!wwo> c €8E Buluolss!wwos Alaaljap {sod O'Or UOHONISUOS JepUury 9'STL UOONJJSUOD JapU/) Cle PEPJOMD $}D.1jUO> Z9Z% WIM suoyoHOBEU Uy OS9 peajesd seppiq UOHONSUOD JepU : Pun LU OrBeujoiAu oBgadwjapasiiygnj sd 0id Oj osLdop0i jaoojupui4dlj ou0ddd n eeeB uijoudAeeBwaiejigon .dyos0 id peulwipuaaljp seoidpoyos iw o lg 90' 0SB9u rppuenxyJ owine O'16Beu ippuanxyz owins Q'0=¢ Buipunpye yinwine 0'06 Buipun, pexiowipe OCS Buipuny pexinwioe 0'1 S°670'7
9 1 6
JBUIO joyido> (uonjiw 3) OS 00 OO Ze 00 00 ; O'el 00 02 00 yo4uo Au ny S8 lOaWs 00 00 00 €°7
p
i
u
youpasay Joyidp> jIoun (uo 3) Ol? ee ct VEG 08 Z9 O€l : Gg 0°88 O'€Z Ll jpeu6Bpp ng 00 éZl Zi Z'89 CO} 90€9
o
>

sayiiop4 e610] += youpasey pdapuny joyido> jlounoy fe6png ,Buipun4 ,Buipun4 pUOUDIO]|V (uo 3) 9 80€ 2}: 1/8)8) OCC YIN GON Bhieo COL 1D 0'0Z YIN O'S OaSds : GL DuVdd yw0- 29 ySOargie /DDYYWD aDe yw YIDDc ic rea!) 877 OuSa arc 1DD er1z YIN fone 9°0€8
]

L jO 1D2), z}aloig uoypsyold y‘¢ (Il PUD | SesDYd) UO4osyoUAg puowDIg i sles OOD sawn diyg youpesay jpAoy fofoy dow SJUBWNASU! JO A}INs {SJ PUD UOHOIS JeHID] Puz Sjs| adAjojoig 2Du!] AlaAooday ABseu GUOHDIS YoiDesey SdsDjUY AB||OH] (Buyndwod pug y81H) YoLd3H aspyd) Juawiuedxy Buljoo> uoljpsiuo) uon dow p00! £00Z (| 284d) a euoc® | von AJsosO DAyBjDonIjDOoegljDgo] W eJ;ONJ}yOo yW{sIu/yUD Y9 H youxpeeBjsudalpwyani onjsjo oun!s yso4yu! [OaU1jO0NH44|O YDSNUOY[]I Y PoEsWD asay udAededy0-o1yiv4xon 41g89 8{90{1]1 34 4X) udyeBid}o4ei snu3nB wusgyo2ny2dilewMgoje > g C00?O O1 pjojyaeusdAnge onyi ng ee sjs| puz joBuD] U— OJ PUddaBsJI NJSO SJUSWINISU! a1ydoiBoyuonineasds9ie yyg J>40L :201N0g JOUONORJYI PNW adIYOC sISAJjOO UD YoIDasay [IOUNODDI DP
6 BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES
EXECUTIVE SUMMARY
Recommendations 6 The Science and Technology Facilities Council,
which will be established in April 2007, should:
1 The Office of Science and Innovation, Research
m= =promote awareness of knowledge and lessons
Councils UK and individual Research Councils should
from planning, procurement and operation of large
strengthen project appraisal by:
scientific facilities by bringing project teams or
m ensuring the production of more consistent estimates
members together to share experiences and training;
of costs and assessments of benefits at the initial
point of prioritisation; and mw develop and promote a consistent means of applying
the science performance management framework
m ensuring that if a project’s expected costs or benefits
developed by the Office of Science and Innovation in
at business case are significantly different from those
2005 to large facilities planning and operation; and
initially anticipated, its priority is reconsidered at the
mw use its own skills base and partnerships with
next available opportunity.
external providers to improve other Research
2 The Office of Science and Innovation, Research Councils’ access to high-grade project management
Councils UK and individual Research Councils should skills for large projects.
give greater attention to the future financial sustainability
7 The Department ofT rade and Industry should work
of projects. Project proposals should be based on realistic
estimates of their ongoing costs, the sources of funding with Research Councils to ensure the Government-wide
available to cover those costs and any implications for Gateway review process is applied to large facility projects
consistently and with a level of independence appropriate
other activities funded by Research Councils.
to their assessment of risk.
3 Research Councils UK should ensure that the road
8 Large facility project teams should build on
map differentiates projects where there is a choice of
procurement lessons from previous projects to secure
location from those where no such choice is practically
improvements in value for money. Across the portfolio of
available. Research Councils UK and the Office of Science
projects there is scope:
and Innovation should provide Research Councils with
guidance to aid preparation of comparisons of different m to undertake a deeper analysis of risks so that
locations where a choice is available. project teams can separate those which should be
transferred to a contractor and those which should
4 _ To improve the transparency of investment decisions,
be retained;
and provide a better opportunity for scrutiny or challenge
m to make greater use of incentives to encourage
by scientific and industrial stakeholders, Research
the timely delivery of key components or
Councils UK should publish the outcomes of and rationale
project milestones;
behind the prioritisation of proposals as part of the
large facilities road map. The rationale should include m to extend the use of pain/gain share conditions
commentary on the implications for the overall research in contracts, thereby increasing the incentives for
programme of supporting the construction and operation contractors to contain costs;
of prioritised projects.
m to improve the packaging of work by considering
the separation of those elements where there is
5 The Office of Science and Innovation, Research
a limited pool of potential suppliers from less
Councils UK and individual Research Councils should:
demanding elements; and
m ~~ ensure an evaluation of the nature and scale of
m= for more active promotion of the work on offer
the economic impacts derived from building and
to potential suppliers who might otherwise be
operating large scientific facilities, once they have
deterred from bidding by the scientific nature of
been brought into service, is undertaken; and
the overall project.
m _ provide guidance to project teams on assessing and
presenting anticipated economic impacts of large
facility proposals.
BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES Fs
EXECUTIVE SUMMARY
Ea |T he National Audit Office looked at ten projects in detail’
Diamond Synchrotron Location: Harwell Science and Innovation Campus, Oxfordshire.
(Diamond Light Source Ltd)
Budget and Funding: £145.6 million for the first phase, with
Diamond is a 24 cell, £127.9 million from the Large Facilities Capital Fund.
3 giga electron volt, 3rd
Delivery: The experimental programme is set to begin in
generation synchrotron light
October 2008.
source producing intense
x-rays and shorter wavelength emissions for research in the
Energy Recovery Linac
biological, physical, environmental and engineering sciences.
Prototype (Council for the
The synchrotron is being built by, and will be operated by, a joint
Central Laboratory of the
venture company Diamond Light Source Ltd, partly owned by the
Research Councils)
Council for the Central Laboratory of the Research Councils and
partly by the Wellcome Trust. The Prototype is phase one
of the 4th Generation Light
Location: Harwell Science and Innovation Campus, Oxfordshire.
Source (4GLS) project. The project will use free electron lasers and
Budget and Funding: £383.2 million for Phases | and Il, with synchrotron radiation covering the terahertz to soft X-ray energy
£308.6 million from the Large Facilities Capital Fund. regimes for ultra fast dynamic studies of matter. The first phase
has been designed to address some of the principal technical
Delivery: Phase |, including the first seven beamlines, is due to
challenges that would be faced in a full 4GLS facility.
begin operations in January 2007 and Phase Il, including the next
15 beamlines, is due to be‘completed in 2011. Location: Daresbury Science and Innovation Campus, Cheshire.
Budget and Funding: £21.3 million, with £10.1 million from the
Royal Research Ship James
Large Facilities Capital Fund.
Cook (Natural Environment
Research Council) Delivery: Full operational energy recovery by April 2007.
The RRS James Cook is a
Halley VI Antarctic Research
replacement for the RRS
Station (Natural Environment
Charles Darwin and is
Research Council)
sponsored by the Natural Environment Research Council. Its users
will be marine scientists based, for example, at UK universities
The project involves the building
and the Research Council’s National Oceanographic Centre
of the Halley VI Antarctic
in Southampton.
research station and the
Location: Worldwide but mainly Atlantic waters — built in Poland removal of the existing station, Halley V. Halley provides a unique
and Norway. location for monitoring climate, ozone and space weather and
forms a key part of the UK’s regional presence. The primary users
Budget and Funding: £40 million, of which £25 million will come
of Halley VI will come from within the British Antarctic Survey, an
from the Large Facilities Capital Fund.
institute of NERC.
Delivery: The ship was delivered to the National Oceanographic
Location: Antarctic Ice Shelf.
Centre in August 2006.
Budget and Funding: £34.7 million for both construction of Halley
VI (26.2 million) and decommissioning of Halley V (8.5 million).
ISIS Neutron Source, Second
Target Station (Council for the The Large Facilities Capital Fund is providing £20 million
Central Laboratory of the for construction.
Research Councils)
Delivery: Delivery of Halley VI and decommissioning of Halley V
The ISIS Neutron and Muon by end of 2009-10 Antarctic summer.
Scattering Facility is the
most powerful neutron producer of its kind in the world. The first
phase of the project involves supplementing the existing facilities
with a second target station and the installation of a first suite of
instruments. It will enable the ISIS science programme to attract
new users from the key research areas of soft matter, advanced
materials and bio-science.
Source: National Audit Office
NOTES
1 Project summaries set out the position as at autumn 2006. More detail on each of the ten pak is oreidetsi n ‘Apponalc3a. oe
2 £67 million for the redevelopment of the Institute for Animal Health is being provided by the Ree Ss Environment, er aa RuralA cs.
8 BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES