Comparison of case note review methods for evaluating quality and safety in health care

185 Pages·2017·1.49 MB·English

This is a repository copy of Comparison of case note review methods for evaluating quality
and safety in health care.
White Rose Research Online URL for this paper:
http://eprints.whiterose.ac.uk/10704/
Article:
Hutchinson, A., Coster, J.E., Cooper, K.L. et al. (8 more authors) (2010) Comparison of
case note review methods for evaluating quality and safety in health care. Health
Technology Assessment, 14 (10). pp. 1-170. ISSN 1366-5278
Reuse
Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright
exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy
solely for the purpose of non-commercial research or private study within the limits of fair dealing. The
publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White
Rose Research Online record for this item. Where records identify the publisher as the copyright holder,
users can verify any specific terms of use on the publisher’s website.
Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing
RHeefearltehn Tceschnology Assessment 2010; Vol. 14: No. 10
Health Technology Assessment 2010; Vol. 14: No. 10
AApbpsetrnadcixt 1
COPD review criteria
Contents
Appendix 2
ListH oef aarbtb fraeilvuiraet iorenvsiew criteria
AEpxpeecuntdiivxe 3 s u mmary
Validity of review criterion questionnaire (COPD)
Appendix 4
A holistic review data collection page
Comparison of case note review
Appendix 5
methods for evaluating quality ChaRpetveire w1 e r training scenarios to assist in recognising variation in care quality using holistic
rIenvtireowduction
and safety in health care
ACphpaepntdeirx 2 6
RAescsoersdsi nregv qieuwa lfitoyr osaf fceatrye a fnrodm qu haolistpyi tsatul dcayse notes: comparison of reliability and utility of
holistic (implicit) and criterion-based (explicit) methods
Appendix 7
A Hutchinson, JE Coster, KL Cooper, COPD – correlations between holistic mean overall scale scoresa and outcome variables
A McIntosh, SJ Walters, PA Bath,
Appendix 8
M Pearson, TA Young, K Rantell, Heart failure – correlations between holistic mean overall scale scoresa and outcome
Chavpatreiarb 3le s
MJ Campbell and J Ratcliffe What is the relationship between information on quality of care from case notes and
Appehnodsipxi t9a l -level outcomes of care?
COPD – correlations between holistic mean phase scale scores and outcome variables
Appendix 10
Heart failure – correlations between holistic mean overall scale scores and outcome
variables
Chapter 4
AppeOnvdeixra 1ll1 c onclusions of the research
COPD – correlations between holistic mean criterion scores and outcome variables
Appendix 12
ChaHpetear t5 f a ilure – correlations between mean criterion scores and outcome variables
Future research agenda
Appendix 13
AckCnoomwlpeadrgiseomne onft hsolistic and criterion-based review methods using structured clinical records
in stroke care
Appendix 14
The place of trigger tool methodology in case note review for quality and safety
Health Technology Assessment reports published to date
Health Technology Assessment
programme
February 2010
DOI: 10.3310/hta14100
Health Technology Assessment
NIHR HTA programme
www.hta.ac.uk
HTA
How to obtain copies of this and other HTA programme reports
An electronic version of this title, in Adobe Acrobat format, is available for downloading free of charge for
personal use from the HTA website (www.hta.ac.uk). A fully searchable DVD is also available (see below).
Printed copies of HTA journal series issues cost £20 each (post and packing free in the UK) to both
public and private sector purchasers from our despatch agents.
Non-UK purchasers will have to pay a small fee for post and packing. For European countries the cost is
£2 per issue and for the rest of the world £3 per issue.
How to order:
– fax (with credit card details)
– post (with credit card details or cheque)
– phone during ofice hours (credit card only).
Additionally the HTA website allows you to either print out your order or download a blank order form.
Contact details are as follows:
Synergie UK (HTA Department) Email:
Comparison of case note review
methods for evaluating quality
and safety in health care
1 1 1
A Hutchinson, * JE Coster, KL Cooper,
1 2 3
A McIntosh, SJ Walters, PA Bath,
4 5 2
M Pearson, TA Young, K Rantell,
2 5
MJ Campbell and J Ratcliffe
1Section of Public Health, ScHARR, University of Shefield, UK
2Section of Health Services Research, ScHARR, University of Shefield, UK
3Department of Information Studies, University of Shefield, UK
4Clinical Effectiveness and Evaluation Unit, Royal College of Physicians,
London, UK
5Section of Health Economics and Decision Sciences, ScHARR, University of
Shefield, UK
*Corresponding author
Declared competing interests of authors: none
M Pearson is now at the Department of Medicine, University of Liverpool.
Published February 2010
10.3310/hta14100
This report should be referenced as follows:
Hutchinson A, Coster JE, Cooper KL, McIntosh A, Walters SJ, Bath PA, et al. Comparison
of case note review methods for evaluating quality and safety in health care. Health Technol
Assess 2010;14(10).
Health Technology Assessment is indexed and abstracted in Index Medicus/MEDLINE, Excerpta
Medica/EMBASE, Science Citation Index Expanded (SciSearch) and Current Contents/Clinical
Medicine.
NIHR Health Technology Assessment programme
he Health Technology Assessment (HTA) programme, part of the National Institute for Health
TResearch (NIHR), was set up in 1993. It produces high-quality research information on the
effectiveness, costs and broader impact of health technologies for those who use, manage and provide care
in the NHS. ‘Health technologies’ are broadly deined as all interventions used to promote health, prevent
and treat disease, and improve rehabilitation and long-term care.
The research indings from the HTA programme directly inluence decision-making bodies such as the
National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee
(NSC). HTA indings also help to improve the quality of clinical practice in the NHS indirectly in that they
form a key component of the ‘National Knowledge Service’.
The HTA programme is needs led in that it ills gaps in the evidence needed by the NHS. There are three
routes to the start of projects.
First is the commissioned route. Suggestions for research are actively sought from people working in the
NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS
trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service
users). The HTA programme then commissions the research by competitive tender.
Second, the HTA programme provides grants for clinical trials for researchers who identify research
questions. These are assessed for importance to patients and the NHS, and scientiic rigour.
Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme
commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together
evidence on the value of speciic technologies.
Some HTA research projects, including TARs, may take only months, others need several years. They
can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence,
undertaking a trial, or other research collecting new data to answer a research problem.
The inal reports from HTA projects are peer reviewed by a number of independent expert referees before
publication in the widely read journal series Health Technology Assessment.
Criteria for inclusion in the HTA journal series
Reports are published in the HTA journal series if (1) they have resulted from work for the HTA
programme, and (2) they are of a suficiently high scientiic quality as assessed by the referees and
editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal
and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication
of the review by others.
The research reported in this issue of the journal was commissioned by the National Coordinating Centre
for Research Methodology (NCCRM), and was formally transferred to the HTA programme in April 2007
under the newly established NIHR Methodology Panel. The HTA programme project number is 06/91/02.
The contractual start date was in June 2004. The draft report began editorial review in March 2009 and
was accepted for publication in May 2009. The commissioning brief was devised by the NCCRM who
speciied the research question and study design. The authors have been wholly responsible for all data
collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher
have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their
constructive comments on the draft document. However, they do not accept liability for damages or losses
arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the HTA
programme or the Department of Health.
Editor-in-Chief: Professor Tom Walley CBE
Series Editors: Dr Martin Ashton-Key, Dr Aileen Clarke, Professor Chris Hyde,
Dr Tom Marshall, Dr John Powell, Dr Rob Riemsma and Professor Ken Stein
ISSN 1366-5278
© 2010 Queen’s Printer and Controller of HMSO
This journal may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that
suitable acknowledgement is made and the reproduction is not associated with any form of advertising.
Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of
Southampton Science Park, Southampton SO16 7NS, UK.
Published by Prepress Projects Ltd, Perth, Scotland (www.prepress-projects.co.uk), on behalf of NETSCC, HTA.
Printed on acid-free paper in the UK by the Charlesworth Group. MR
DOI: 10.3310/hta14100 Health Technology Assessment 2010; Vol. 14: No. 10
Abstract
Comparison of case note review methods for
evaluating quality and safety in health care
1 1 1 1 2
A Hutchinson, * JE Coster, KL Cooper, A McIntosh, SJ Walters,
3 4 5 2 2 5
PA Bath, M Pearson, TA Young, K Rantell, MJ Campbell and J Ratcliffe
1Section of Public Health, ScHARR, University of Shefield, UK
2Section of Health Services Research, ScHARR, University of Shefield, UK
3Department of Information Studies, University of Shefield, UK
4Clinical Effectiveness and Evaluation Unit, Royal College of Physicians, London, UK
5Section of Health Economics and Decision Sciences, ScHARR, University of Shefield, UK
*Corresponding author
Objectives: To determine which of two methods of groups and between review methods. To explore the
case note review – holistic (implicit) and criterion- process–outcome relationship, a range of publicly
based (explicit) – provides the most useful and reliable available health-care indicator data were used as proxy
information for quality and safety of care, and the level outcomes in a multilevel analysis.
of agreement within and between groups of health-care Results: Overall, 1473 holistic and 1389 criterion-
professionals when they use the two methods to review based reviews were undertaken in the irst part of
the same record. To explore the process–outcome the study. When same staff-type reviewer pairs/groups
relationship between holistic and criterion-based reviewed the same record, holistic scale score inter-
quality-of-care measures and hospital-level outcome rater reliability was moderate within each of the three
indicators. staff groups [intraclass correlation coeficient (ICC)
Data sources: Case notes of patients at randomly 0.46–0.52], and inter-rater reliability for criterion-based
selected hospitals in England. scores was moderate to good (ICC 0.61–0.88). When
Review methods: In the irst part of the study, different staff-type pairs/groups reviewed the same
retrospective multiple reviews of 684 case notes were record, agreement between the reviewer pairs/groups
undertaken at nine acute hospitals using both holistic was weak to moderate for overall care (ICC 0.24–0.43).
and criterion-based review methods. Quality-of-care Comparison of holistic review score and criterion-
measures included evidence-based review criteria and a based score of case notes reviewed by doctors and
quality-of-care rating scale. Textual commentary on the by non-clinical audit staff showed a reasonable level
quality of care was provided as a component of holistic of agreement (p-values for difference 0.406 and 0.223,
review. Review teams comprised combinations of: respectively), although results from all three staff types
doctors (n = 16), specialist nurses (n = 10) and clinically showed no overall level of agreement (p-value for
trained audit staff (n = 3) and non-clinical audit staff difference 0.057). Detailed qualitative analysis of the
(n = 9). In the second part of the study, process (quality textual data indicated that the three staff types tended
and safety) of care data were collected from the case to provide different forms of commentary on quality of
notes of 1565 people with either chronic obstructive care, although there was some overlap between some
pulmonary disease (COPD) or heart failure in 20 groups. In the process–outcome study there generally
hospitals. Doctors collected criterion-based data from were high criterion-based scores for all hospitals,
case notes and used implicit review methods to derive whereas there was more interhospital variation
textual comments on the quality of care provided between the holistic review overall scale scores. Textual
and score the care overall. Data were analysed for commentary on the quality of care veriied the holistic
intrarater consistency, inter-rater reliability between scale scores. Differences among hospitals with regard to
pairs of staff using intraclass correlation coeficients the relationship between mortality and quality of care
(ICCs) and completeness of criterion data capture, were not statistically signiicant.
and comparisons were made within and between staff iii
© 2010 Queen’s Printer and Controller of HMSO. All rights reserved.
Abstract
Conclusions: Using the holistic approach, the three value. Therefore, when measuring quality of care
groups of staff appeared to interpret the recorded from case notes, consideration needs to be given to
care differently when they each reviewed the same the method of review, the type of staff undertaking
record. When the same clinical record was reviewed the review, and the methods of analysis available to
by doctors and non-clinical audit staff, there was no the review team. Review can be enhanced using a
signiicant difference between the assessments of combination of both criterion-based and structured
quality of care generated by the two groups. All three holistic methods with textual commentary, and
staff groups performed reasonably well when using variation in quality of care can best be identiied from
criterion-based review, although the quality and type a combination of holistic scale scores and textual data
of information provided by doctors was of greater review.
iv
DOI: 10.3310/hta14100 Health Technology Assessment 2010; Vol. 14: No. 10
Contents
Contents ................................................... v Appendix 5 Reviewer training scenarios to
assist in recognising variation in care quality
List of abbreviations ................................. vii using holistic review ................................... 101
Executive summary ................................. ix Appendix 6 Record review for safety and
quality study ............................................... 103
1 Introduction .............................................. 1
Appendix 7 COPD – correlations between
2 Assessing quality of care from hospital case holistic mean overall scale scores and
notes: comparison of reliability and utility outcome variables ...................................... 119
of holistic (implicit) and criterion-based
(explicit) methods .................................... 3 Appendix 8 Heart failure – correlations
Background ............................................... 3 between holistic mean overall scale scores
Methods ..................................................... 5 and outcome variables ............................... 121
Results ........................................................ 16
Discussion .................................................. 34 Appendix 9 COPD – correlations between
holistic mean phase scale scores and outcome
3 What is the relationship between variables ..................................................... 123
information on quality of care from
case notes and hospital-level outcomes Appendix 10 Heart failure – correlations
of care? ...................................................... 41 between holistic mean overall scale scores
Background ............................................... 41 and outcome variables ............................... 125
Methods ..................................................... 42
Results ........................................................ 47 Appendix 11 COPD – correlations between
Discussion .................................................. 61 holistic mean criterion scores and outcome
variables ..................................................... 127
4 Overall conclusions of the research ........ 65
Implications for reviewing quality of care . 65 Appendix 12 Heart failure – correlations
Relationships between quality and outcome between mean criterion scores and outcome
of care .................................................... 66 variables ..................................................... 129
5 Future research agenda ........................... 67 Appendix 13 Comparison of holistic and
criterion-based review methods using
Acknowledgements .................................. 69 structured clinical records in stroke care ... 131
References ................................................ 71 Appendix 14 The place of trigger tool
methodology in case note review for quality
Appendix 1 COPD review criteria ............ 75 and safety ................................................... 141
Appendix 2 Heart failure review criteria . 83 Health Technology Assessment reports
published to date ...................................... 145
Appendix 3 Validity of review criterion
questionnaire (COPD) ............................... 91 Health Technology Assessment
programme ............................................... 167
Appendix 4 A holistic review data collection
page ........................................................... 99
v
© 2010 Queen’s Printer and Controller of HMSO. All rights reserved.
DOI: 10.3310/hta14100 Health Technology Assessment 2010; Vol. 14: No. 10
List of abbreviations
ADE adverse drug event MI myocardial infarction
CI conidence interval NICE National Institute for Health and
Clinical Excellence
COPD chronic obstructive pulmonary
disease NPSA National Patient Safety Agency
HCC Healthcare Commission RCP Royal College of Physicians
HES Hospital Episode Statistics RCP CEEu Royal College of Physicians Clinical
Effectiveness and Evaluation Unit
HRG Healthcare Resource Group
SD standard deviation
HSMR Hospital Standardised Mortality
Ratio SMR standardised mortality ratio
ICC intraclass correlation coeficient SpR specialist registrar
IHI Institute for Healthcare TIA tranisent ischaemic attack
Improvement
All abbreviations that have been used in this report are listed here unless the abbreviation is well
known (e.g. NHS), or it has been used only once, or it is a non-standard abbreviation used only in
igures/tables/appendices, in which case the abbreviation is deined in the igure legend or in the
notes at the end of the table.
vii
© 2010 Queen’s Printer and Controller of HMSO. All rights reserved.