Measuring up : improving health system performance in OECD countries.

347 Pages·2002·2.918 MB·English

by  OECD

«
Measuring Up
IMPROVING HEALTH SYSTEM PERFORMANCE
Measuring Up
IN OECD COUNTRIES
Proceedings of the Ottawa Conference, November 2001.
IMPROVING HEALTH SYSTEM
How can we measure the performance of different health systems, and how can we use such
information to support on-going health system improvement? Those are the central questions PERFORMANCE
addressed in this volume. Health policy makers have a growing interest in finding ways of
encouraging health systems to improve their performance, where performance is measured against IN OECD COUNTRIES
quality, efficiency or equity goals. Improving performance has the potential to reduce the tensions
between rising demands and limited resources. There is also a growing demand for accountability
among funders and providers of health services.
M
e
This book highlights the core elements of a possible performance measurement framework to assess a
health systems at the international and national levels. It also addresses further challenges which s
u
remain: how do we overcome the lack of health outcome measures? How do we better align r
i
n
performance information and incentives with policy objectives? And how do we reconcile the g

traditional professional self-regulation approach with greater public accountability for health care U
quality? p ^
IM
P
R
O
V
IN
G
H
E
A
L
T
H
S
Y
S
T
E
M ^
P
E
R
F
O
R
M
A
N
C
E
IN
OECD's books, periodicals and statistical databases are now available via www.SourceOECD.org, our online library. O
E
This book is available to subscribers to the following SourceOECD themes: C
D
Social Issues/Migration/Health C
O
U
Ask your librarian for more details of how to access OECD books online, or write to us at N
[email protected] T
R ^
IE
S
www.oecd.org
ISBN 92-64-19676-5
81 2002 01 1 P
-:HSTCQE=V^[\[[:
© OECD, 2002.
© Software: 1987-1996, Acrobat is a trademark of ADOBE.
All rights reserved. OECD grants you the right to use one copy of this Program for your personal use only. Unauthorised reproduction,
lending, hiring, transmission or distribution of any data or software is prohibited. You must treat the Program and associated materials
and any elements thereof like any other copyrighted material.
All requests should be made to:
Head of Publications Service,
OECD Publications Service,
2, rue André-Pascal,
75775 Paris Cedex 16, France.
Measuring Up
IMPROVING HEALTH SYSTEM
PERFORMANCE
IN OECD COUNTRIES
ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT
THE ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD)
Pursuant to Article 1 of the Convention signed in Paris on 14th December 1960, and which came
into force on 30th September 1961, the Organisation for Economic Co-operation and Development
(OECD) shall promote policies designed:
– to achieve the highest sustainable economic growth and employment and a rising standard of
living in Member countries, while maintaining financial stability, and thus to contribute to the
development of the world economy;
– to contribute to sound economic expansion in Member as well as non-member countries in the
process of economic development; and
– to contribute to the expansion of world trade on a multilateral, non-discriminatory basis in
accordance with international obligations.
The original Member countries of the OECD are Austria, Belgium, Canada, Denmark, France,
Germany, Greece, Iceland, Ireland, Italy, Luxembourg, the Netherlands, Norway, Portugal, Spain,
Sweden, Switzerland, Turkey, the United Kingdom and the United States. The following countries
became Members subsequently through accession at the dates indicated hereafter: Japan
(28thApril1964), Finland (28th January 1969), Australia (7th June 1971), New Zealand (29th May 1973),
Mexico (18th May 1994), theCzech Republic (21st December 1995), Hungary (7th May 1996), Poland
(22ndNovember 1996), Korea (12th December 1996) and Slovak Republic (14th December 2000). The
Commission of the European Communities takes part in the work of the OECD (Article 13 of the OECD
Convention).
Publié en français sous le titre :
Être à la hauteur
MESURER ET AMÉLIORER LA PERFORMANCE DES SYSTÈMES DE SANTÉ DANS LES PAYS DE L’OCDE
© OECD 2002
Permission to reproduce a portion of this work for non-commercial purposes or classroom use should be obtained
through the Centre français d’exploitation du droit de copie (CFC), 20,rue des Grands-Augustins, 75006 Paris, France,
tel. [phone], fax [phone], for every country except the United States. In the United States permission
should be obtained through the Copyright Clearance Center, Customer Service, [phone], 222 Rosewood Drive,
Danvers, MA 01923 USA, or CCC Online: www.copyright.com. All other applications for permission to reproduce or translate
all or part of this book should be made to OECD Publications, 2,rue André-Pascal, 75775 Paris Cedex 16, France.
FOREWORD
This report examines progress and challenges in the effective measurement and application of
performance indicators to improve health systems. Measuring and improving health system
performance is a key priority for OECD countries. In May2001, OECD Ministers noted that “health
systems are an important element in social cohesion and represent the largest service sector in many
OECD countries. Their efficiency, effectiveness and equity consequences, their impact on public
finances, and their ability to meet the challenges of medical advances, ageing populations and rising
expectations require creative policy approaches”.
Annual healthcare costs in OECD countries have topped US$2.3trillion –almost 10 per cent of GDP.
Health systems play a crucial role in promoting people’s well-being. They are also major employers of
skilled workers and an important source of demand for high-tech industries such as pharmaceuticals,
diagnostic and imaging products and biotechnology. However, OECD health systems are coming under
increasing pressure, and additional funding alone will not be enough to respond to steadily rising
demands. There is an on-going need to improve the cost-effectiveness of health care spending.
In response to requests from Member governments, the OECD launched in2001 a new Health
Project. The main theme of this project is performance measurement and improvement in OECD health
systems. Through comparing experiences across OECD countries, the project will seek to address the
challenge of deciding the appropriate amount of resources to be devoted to health care as well as ways
in which these resources should be financed and allocated.
As part of this new Health Project, the OECD and Health Canada held a conference entitled
“Measuring Up: Improving health system performance in OECD countries”, on 5-7November2001 in
Ottawa. The conference brought together more than 400participants from OECD countries and other
international organisations, to discuss good practices in measuring key components of health system
performance and to share experiences on how best to report and use performance measures to build
more effective health systems. The conference assembled not only experts and policy-makers, but also
representatives from civil society and key actors directly involved in the functioning of health care
systems, be they patients, health care professionals or managers of health care institutions. The views
of these different actors in health care systems were factored into the conference deliberations through
the active participation of a “Commentary Panel” which provided useful inputs at different points
during the meeting. The conference ended with a Ministerial Roundtable discussion, which provided an
opportunity for Ministers from several OECD countries to share views about the political challenges
their countries face in developing and applying performance measures. This report contains the main
papers presented at the conference together with the highlights of the Ministerial Roundtable.
The Ottawa Conference was an important milestone in OECD’s work on health system performance.
It provided a marvellous opportunity to discuss practical ideas on ways to measure and improve the
efficiency and equity of health care systems. It also demonstrated strong political commitments to
international co-operation in developing performance measurement frameworks that support on-going
health systems improvement.
Donald J. Johnston Allan Rock
Secretary-General of the OECD Former Minister of Health, Canada 3
© OECD 2002
Measuring Up: Improving Health System Performance in OECD Countries
Acknowledgements
The OECD would like to thank all those who contributed to the Ottawa conference and to
this volume. It would like to express particular gratitude to the Government of Canada and the
former Canadian Minister of Health, Allan Rock, for hosting this conference. Thanks go to Denis
Gauthier, David Kelly, David Hoye, and François Sauvé in Health Canada and to Peter Scherer,
Jeremy Hurst, Stéphane Jacobzone and Gaétan Lafortune in the OECD Secretariat for their
contributions in planning and organising this conference. Special thanks go also to Peter Smith
from the Centre for Health Economics at the University of York in the United Kingdom, for
assuming the responsibility of Editor of these conference proceedings. The contributions of
Geoff Anderson from the Department of Health Policy, Management and Evaluation at the
University of Toronto in Canada and Tom Reilly from the Agency for Healthcare Research and
Quality in the United States as members of the Conference Advisory Group, are also gratefully
acknowledged.
4
© OECD 2002
TABLE OF CONTENTS
Editor’s Preface
by Peter Smith.......................................................................................................................................................................... 7
Part I
Overview of Issues and Challenges
Chapter 1. Measuring Health System Performance: Problems and Opportunities in the Era of Assessment
and Accountability
by C. David Naylor, Karey Iron and Kiren Handa................................................................................................ 13
Chapter 2. Performance Measurement and Improvement in OECD Health Systems: Overview of Issues
and Challenges
by Jeremy Hurst................................................................................................................................................... 35
Part II
Performance Measurement and Performance Management
from the Perspective of Various Actors: A Review of Experiences in Selected Countries
Chapter 3. Improving Value for Money in the United Kingdom National Health Service:
Performance Measurement and Improvement in a Centralised System
by Clive H. Smee.................................................................................................................................................. 57
Chapter 4. Improving Accountability in a Decentralised System: A Swedish Perspective
by Nina Rehnqvist................................................................................................................................................ 87
Chapter 5. Providing Performance Information for Consumers: Experience from the United States
by Thomas Reilly, Gregg Meyer, Carla Zema, Christine Crofton, David Larson, Charles Darby
and Katherine Crosson ......................................................................................................................................... 97
Chapter 6. Can a Tulip Become a Rose? The Dutch Route of Guided Self-Regulation Towards
a Community-Based Integrated Health Care System
by Niek Klazinga, Diana Delnoij and Isik Kulu-Glasgow....................................................................................... 117
Chapter 7. Towards Integrated and Coherent Health Information Systems for Performance Monitoring:
The Canadian Experience
by Michael Wolfson and Richard Alvarez............................................................................................................... 133
Part III
Performance Measurement Activities at the International Level:
How can International Comparisons Assist National Policy-Making?
Chapter 8. Opening the Black Box: What Can Be Learned from a Disease-based Approach?
Stephane Jacobzone, Pierre Moise and Lynelle Moon.............................................................................................. 159 5
© OECD 2002
Measuring Up: Improving Health System Performance in OECD Countries
Chapter 9. The Evolution of WHO’s Approach to Health System Performance Assessment
by David B. Evans................................................................................................................................................. 197
Chapter 10. Measuring and Improving Patients’ Experiences: How Can we Make Health Care Systems Work
for Patients?
by Angela Coulter and Paul Cleary....................................................................................................................... 211
Chapter 11. Equity in the Use of Physician Visits in OECD Countries: Has Equal Treatment
for Equal Need Been Achieved?
by Eddy van Doorslaer, Xander Koolman and Frank Puffer.................................................................................... 225
Part IV
Best Practices in Measuring Different Dimensions of Health System Performance
Chapter 12. Measuring the Quality of Hospital Care: The State of the Art
by Gérard de Pouvourville and Étienne Minvielle................................................................................................... 251
Chapter 13. Measuring the Quality of Long-term Care in Institutional and Community Settings
by Naoki Ikegami, John P. Hirdes and Iain Carpenter............................................................................................ 277
Chapter 14. Developing Composite Indicators for Assessing Health System Efficiency
by Peter Smith...................................................................................................................................................... 295
Part V
Applying Performance Indicators to Health System Improvement
Chapter 15. Applying Performance Indicators to Health System Improvement
by Sheila Leatherman............................................................................................................................................ 319
Part VI
Summary and Conclusions
Chapter 16. Measuring Up: Lessons and Potential
by Jean-François Girard and Étienne Minvielle...................................................................................................... 337
Part VII
Ministerial Roundtable: Leadership, Successes and Challenges
Chapter 17. Measuring Health System Performance and the Impact on Political Decision-Making:
The Views of OECD Ministers
by Julio Frenk (Minister of Health, Mexico), John Hutton (Minister of State for Health, United Kingdom),
Bernard Kouchner (Minister of Health, France), Allan Rock (Minister of Health, Canada), Edward Sondik
(on behalf of Tommy Thompson, Secretary of Health and Human Services, United States)...................................... 351
6
© OECD 2002
EDITOR’S PREFACE
by
Peter Smith*
The concern with measuring the performance of health systems and health care is not recent. In the
1860s Florence Nightingale pioneered the systematic collection, analysis and dissemination of
comparative hospital outcomes data in order to understand and improve performance. Fifty years later,
Ernest Codman promoted the need for scrupulous collection and public release of surgical outcome
data (Spiegelhalter, 1999). However, there were many practical, professional and political impediments
to making such principles operational. It is only in the last ten years that the vision of using large-scale
data sources to help improve health system performance has become a reality.
A number of developments have led to this recent transformation. Most importantly, the
accelerating advances in medical technology have led to enormous potential for new interventions and
methods of delivering and organising health care, with a concomitant pressure on expenditure levels.
There is therefore an urgent need to check that innovations – of whatever sort– promote system
objectives and avoid adverse side-effects. At the same time, popular expectations have become ever
more demanding, not least in the accountability and transparency they demand of their health systems.
In this respect, performance information serves as a fundamental tool of communication. Finally,
developments in information technology have revolutionised the capacity to capture almost
instantaneously many measures of process and outcome in a way that was until recently unimaginable.
The interest in system performance was given added impetus by the World Health Report 2000
produced by the World Health Organisation, entitled “Health systems: improving performance” (World
Health Organisation, 2000). The debate initiated by WHR2000has been dominated by responses to the
league tables of national health system performance presented in its statistical annexes. However the
main body of the report raises fundamental issues associated with defining, measuring and promoting
health system performance.
In this context, performance data have two broad functions: to identify in general “what works” in
promoting the objectives of the health system; and to identify the functional competence of specific
practitioners or organisations. These correspond, respectively, to the research and managerial roles of
performance data. In whichever role, such data can then be used to promote continuous improvement,
by enabling policy makers to adopt the most cost-effective technologies, and helping practitioners to
identify opportunities for personal or organisational improvement.
This volume seeks to reflect the current “state of the art” in performance measurement across
OECD health systems. It is organised into seven sections, reflecting different perspectives on the
performance measurement process. PartI sets the scene. Naylor, Iron and Handa (Chapter1) summarise
progress to date, emphasizing the simple but powerful truth that we cannot manage what we do not
measure. Hurst (Chapter2) surveys the issues and challenges faced by OECD countries as they seek to
measure and improve their health system’s performance. He pays particular attention to the role of
international comparisons, in which the pioneering work of the OECD has played a central role (OECD, 2001).
* Centre for Health Economics, University of York. 7
© OECD 2002
Measuring Up: Improving Health System Performance in OECD Countries
PartII of the book summarises developments in five contrasting OECD countries. Smee (Chapter3)
describes the move towards a philosophy of “performance management” in the English National Health
Service, the archetypal centralised health system, an approach that relies heavily on the massive
compulsory provision of reliable, comprehensive and timely performance data. In contrast, Rehnqvist
(Chapter4) presents the Swedish case, in which responsibility for the health system is devolved to local
government, and the provision of performance data relies on clinically led collaboratives known as
quality registers, to which subscription is voluntary. The US health system has a uniquely strong focus
on consumers and the market, and Reilly, Meyer and colleagues (Chapter5) summarise experience with
public release of performance data to inform consumer choice. They report that –although consumers
appear to take little notice of such data – the public release nevertheless stimulates important
improvement efforts by providers. A more formal approach to performance management amongst
providers is the Dutch experience with “guided” self-regulation, presented by Klazinga, Delnoij and
Kulu-Glasgow (Chapter6). They notethat the key challenge is to translate the undoubted progress
made in clinically based performance measurement to the more ambitious agenda of managing whole
system performance. Finally, Wolfson and Alvarez (Chapter7) summarise the enormous progress made
towards integrating information systems in Canada, and the promise it holds for performance
monitoring.
Part III examines the methodological issues raised when seeking to make international
performance comparisons. Jacobzone, Moise and Moon (Chapter8) describe one of the OECD
approaches towards international comparison, which concentrates on micro data for specific conditions.
The intention is to use the natural experiment of international variations to secure an improved
understanding of the links between technology adoption and clinical outcome. Evans (Chapter9)
describes some of the methodological issues pursued by the World Health Organisation since
publication of World Health Report 2000, in particular the problem of securing comparability when
interpreting survey results from different countries. Coulter and Cleary (Chapter10) examine the
complex issue of measuring the patient’s experience. They emphasize the importance of this dimension
of health care, but notethe methodological difficulties involved in developing reliable and comparable
indicators of system responsiveness. Van Doorslaer, Koolman and Puffer (Chapter11) summarise their
work on the important issue of measuring horizontal equity, in the sense of equal treatment for equal
need. They confirm that –even in health systems that explicitly seek to promote equity and solidarity–
there exist important unexplained variations in treatment associated with income levels.
In PartIV the book turns to a series of key measurement issues. De Pouvourville and Minvielle
(Chapter12) describe the rich experience of seeking to develop indicators of hospital performance,
based mainly on US research. The principal methodological challenges are to develop adequate
patient outcome measures and to adjust satisfactorily for variations in the complexity of the hospital’s
case-mix. Long term care for people with physical or mental disability is a key policy issue for many
OECD countries, and the provision of good quality performance data is a prerequisite for the
development of effective policy. The advantage in this sector is that –compared with hospitals– the
client groups are relatively homogeneous, and Ikegami, Hirdes and Carpenter (Chapter13) report a
great deal of progress on a range of quality indicators. To conclude part four, Smith (Chapter14)
examines the issues that arise when seeking to combine individual indicators into a single composite
measure of system performance. There may often be a need to construct such composites, but to date
the methodology underlying such efforts has been weak.
Although most of the papers in the first four parts of the book concentrate on methodological topics,
many make reference to the key issue of how performance data can be used to secure health system
improvement. It is not enough merely to produce the performance data. They must be embedded in a
system that uses them to best effect (and avoids unintended dysfunctional consequences). In PartV,
Leatherman (Chapter15) outlines the range of levers that exist to secure performance improvements,
grouped into five categories: external oversight, professional development and education, empowering
consumers, incentives, and regulation. She concludes that the gaps in our evidence base make it
8 imperative to employ a blend of approaches that give rise to complementary effects.
© OECD 2002