Evaluation of the secondary triage pilots in the Ambulance Communications Centres

54 Pages·2013·1.31 MB·English

by  Rae Torrie

Evaluation of the Secondary Triage
Pilots in the
Ambulance Communication Centres

Final Report

Prepared for: Ministry of Health

Prepared by: Rae Torrie, Robyn Bailey, Julian King, Gary
Strong with Jeni Irving and Sandy Dawson

Contract held by: Evaluation Works Ltd
Table of contents
Executive summary .................................................................................................. 1
Introduction ............................................................................................................. 6
Background .............................................................................................................. 6
The secondary triage pilots ....................................................................................... 6
Strategic overview ................................................................................................................ 7
Delivery processes ................................................................................................................ 7
Evaluation brief ........................................................................................................ 8
Methodology............................................................................................................ 9
Key evaluation questions (KEQs) ......................................................................................9
Evaluation design ............................................................................................................9
Methods ....................................................................................................................... 10
Stakeholder interviews ....................................................................................................... 10
Telephone survey to service users ..................................................................................... 11
Clinical safety review of calls .............................................................................................. 12
Desktop data analysis ......................................................................................................... 12
Analysis ............................................................................................................................... 12
Findings .................................................................................................................. 13
KEQ 1: How well are the secondary triage processes working in each pilot site? .............. 13
Overall finding (level of effectiveness against rubric 1) ..................................................... 13
Triage tools ......................................................................................................................... 14
Range of dispositions .......................................................................................................... 14
Recruitment, selection and training of personnel .............................................................. 15
Clinical governance and risk management ......................................................................... 15
Clinical safety ...................................................................................................................... 16
Seamless interface .............................................................................................................. 18
Timeliness ........................................................................................................................... 19
Credibility and acceptability to service users ..................................................................... 21
Credibility and acceptability to health professionals ......................................................... 23
Mechanisms for learning and improving ............................................................................ 23
KEQ 2: How well are the secondary triage pilots achieving their intended outcomes? ...... 24
Overall finding (level of effectiveness against rubric 2) ..................................................... 24
Clinical safety ...................................................................................................................... 24
Level of care to which people are triaged .......................................................................... 26
Impact on ambulance dispatch, ......................................................................................... 29
Impact on emergency departments ................................................................................... 30
KEQ 3: What are the efficiency implications of the pilot processes and outcomes? .......... 31
Operational costs of call triaging ........................................................................................ 32
Impacts on wider health system costs ............................................................................... 33
Implications for health system efficiency ........................................................................... 33
KEQ 4: What can be learned from the evaluation to inform future decisions? .................. 34
Success factors.................................................................................................................... 34
Considerations for the future of secondary triage ............................................................. 36
Appendices ............................................................................................................ 39
Appendix One: Process maps of secondary triage in practice .......................................... 39
Appendix Two: KEQs, rubrics and exploratory questions ................................................. 41
Appendix Three: Phone survey of users of secondary triage ............................................ 46
i
Acknowledgements: The evaluation team gratefully acknowledges the willing and
positive assistance of all three secondary triage pilot providers – St John, Wellington
Free Ambulance and Medibank Health Solutions - in facilitating arrangements for visits,
providing information and generally supporting the evaluation process.

Disclaimer: The information in this report is presented in good faith using the
information available to us at the time of preparation. It is provided on the basis that
the authors of the report are not liable to any person or organisation for any damage
or loss which may occur in relation to taking or not taking action in respect of any
information or advice within this proposal.

Citation: Any content reproduced from this report should be cited as follows:
Torrie, R., Bailey, R., King, J., Strong, G., Irving, J., and Dawson, A. (2013). Evaluation of
the Secondary Triage Pilots in the Ambulance Communications Centres. Prepared for
the Ministry of Health by Evaluation Works Limited.

ii
Executive summary

The Ministry of Health (MoH) contracted Evaluation Works Ltd to conduct an
evaluation of two secondary triage pilots. The evaluation covered a five-month
period, from the start of the pilots in Oct 2012 till end Feb 2013.

The objectives of the pilots are to:
1. Provide people calling 111 for an ambulance with the lowest safe level of
care appropriate to their clinical need
2. Contribute to the cost-effectiveness of that care, for example by improved
utilization of current ambulance resources.

The secondary triage pilots are part of the Urgent and unplanned care integration
project. There is interest in the extent to which the pilots, by changing service
delivery arrangements, contribute to achieving more integrated care.

The secondary triage pilots were undertaken by St John in the St John Canterbury
dispatch area, and Wellington Free Ambulance (WFA) and Medibank Health
Solutions NZ Ltd in WFA’s dispatch area which covers the greater Wellington region
and Wairarapa.

The pilots focus on 111 calls that are deemed low acuity and appropriate for
secondary triage. Transfer of 111 calls to secondary triage means that instead of an
emergency ambulance being immediately dispatched, further clinical triage is
provided for callers, and appropriate advice given on the health service they should
access and in what timeframe. This includes the option of an ambulance if needed.

The evaluation addressed four key questions:
1. How well are the secondary triage processes working in each pilot site?
2. How well are the secondary triage pilots achieving their intended outcomes?
(i.e., lowest safe level of care appropriate to clinical need)
3. What are the efficiency implications of the pilot processes and outcomes?
4. What can be learned from the evaluation to inform future decisions?

Evaluative criteria (attached at Appendix Two) were developed in consultation with
the MoH, National Ambulance Sector Office (NASO), and the three providers
involved in the pilots. A mixed method evaluation was undertaken, triangulating
evidence collected through stakeholder interviews (7 funder and 19 provider
individual or group interviews), a phone survey of 50 111 callers (25 from each
location) who were transferred to secondary triage, a clinical safety review of 9-10
calls in each location, and a desktop analysis of operational data.

The evaluation found that the secondary triage pilots are being delivered well in
each site. They are highly effective in terms of achieving their intended objectives
(refer above). There is also a growing recognition, through the secondary triage pilot
Evaluation of the secondary triage pilots in the Ambulance Communications Centres 1
process, of the ambulance sector as an integral part of primary health care, as well
as part of the emergency response sector.

More detailed findings are summarized below. We note that the evaluation is at a
point in time and all provider organisations continue to make changes in the way the
pilots are run and managed. We understand that some of the suggestions and
opportunities identified for improvement now form part of the service delivered.

Secondary triage processes
Secondary triage is being delivered well by the providers in each location. All three
providers have a high level of engagement and commitment at the strategic level
and a big operational investment in making secondary triage work effectively, with a
key person in each site overseeing and troubleshooting any issues.

The evaluation highlighted very different approaches to doing secondary triage
between the two locations. Medibank has a very systematic approach based on the
progressive exclusion of serious concerns supported by algorithmic software. In
Christchurch a diagnostic1 or medical model of assessment is used that is based on
pattern recognition. In terms of the balance between fidelity to a tool (clinicians
adhering religiously to the script and guidelines) and clinical discretion, Medibank is
located toward the fidelity end of continuum and St John the discretion end. (See
diagram 1 on p.18.)

Mostly both secondary triage approaches are working well, with some issues but real
strengths in each place. In Wellington, Medibank provides well-established,
systematic and robust secondary triage processes, using an internationally validated
tool and robust governance and management processes. In Christchurch, St John is
particularly good in ensuring the patient is able to access the agreed disposition,
with a range of local community options based on good relationships and networks,
and going the ‘extra mile’ in their service to users.

The evaluation’s clinical safety review found there was not a consistent application
of the secondary triage process or the triage tool by St John paramedics. In the
Wellington calls selected for review, there was evident reluctance on the part of
some nurses to refer back to WFA for transport. Whilst this may seem to support
the aim of reducing numbers of ambulance transports, the clinical review team felt
that it had the potential to affect access to health care for people with fewer
financial and transport resources. Going forward with integrated healthcare design,
the issue of ‘transport only’ needs should be considered.

The majority of callers transferred to secondary triage who are dealt with by ‘hear
and treat’2, are receiving a more rapid clinical response (in this case speaking with a

1 Both approaches involve seeking a provisional diagnosis, but one seeks to minimize or eliminate risk using
systematic questioning before arriving at a preliminary diagnosis and plan (disposition) for the patient; whereas
the other seeks to discover the cause of the chief complaint, (preliminary diagnosis) before going on to assess
risks that may be associated with the emerging plan (disposition).
2 This service is perhaps more accurately described as ‘hear and advise’ but hear and treat is the term used
Evaluation of the secondary triage pilots in the Ambulance Communications Centres 2
nurse or paramedic) than they would have waiting in the ‘code grey’ queue for an
ambulance dispatch. Because secondary triage calls subsequently assessed as
needing an ambulance response are returned to 111 with an upgraded priority code,
these calls may receive a more immediate ambulance response than they would
have without secondary triage. It was not possible to confirm this with data.

Staff in all locations who were interviewed are supportive of secondary triage.
Service user’ feedback is mixed, attributable in part to a lack of publicity,
understanding, and conflicting messages between the ProQA script and the
secondary triage process.

Secondary triage outcomes
The two secondary triage pilots were highly effective in achieving their intended
objectives, to:
 Provide people calling 111 for an ambulance with the lowest safe level of
care appropriate to their clinical need
 Contribute to the cost-effectiveness of that care, for example by a reduction
in the number of lower acuity patients unnecessarily taken to an emergency
department (ED) and improved utilization of current ambulance resources.

Clinical safety standards have been met, and nearly 2000 calls resolved by secondary
triage over 5 months. There has been a practically significant reduction in the
dispatch of ambulances, freeing up resources equivalent to approximately one year’s
growth in demand.

WFA transferred 2,713 calls to Medibank for secondary triage, and St John
transferred 1,6413 calls to their in-house secondary triage during the five-month
period covered by the evaluation. Wellington resolved by ‘hear and treat’ 54% of the
total number of 111 calls transferred to secondary triage. Christchurch resolved
30% by ‘hear and treat’. (Confounding factors in interpreting this data include that
WFA elected not to secondary triage some 111 calls aligned with a ‘grey’ response
while St John did so, and alarm calls are referred to secondary triage in Christchurch
but not in Wellington.)

Health efficiencies
Savings from the secondary triage pilots include a decrease in ambulance dispatches
equivalent to approximately one year’s growth in demand. The estimated savings
over the five months of the pilot on a marginal cost basis (the extra cost of staffing
available ambulances) $109,000, and on an average cost basis (which takes into
account the eventual need to add an extra crewed ambulance in response to
growth) is $1.375 million4. There are also improved health system efficiencies with
the potential for further improvement.

colloquially by both services, so is used throughout the report.
3 The substantial difference in numbers is due to the fact that the St John secondary triage pilot was
operationalonly from 7am to 7pm, compared with Wellington operating 24/7.
4 If extrapolated to annual savings the translates to $262,000 based on marginal cost, or $3.3 million based on
Evaluation of the secondary triage pilots in the Ambulance Communications Centres 3
Learnings to inform future decisions
Success of the pilots has been demonstrated by the realisation of two key objectives:
the lowest safe level of care appropriate to clinical need for users, and health system
efficiencies. The pilots have provided proof of concept in relation to secondary triage.

The following features of the pilots were identified as important contributors to their
effectiveness and would need to be considered if secondary triage is scaled up:
 a 24/7 service and a quick response to secondary triage calls
 a seamless transfer of calls
 robust clinical governance and risk management processes
 a key manager in each provider organization who understands both business
and secondary triage well
 a systematic approach to secondary triage, supported by good IT-based
decision support
 training, support and ongoing staff development in telephone triage
generally, and emergency medicine specifically
 a responsive secondary triage person who can balance safe outcomes for the
user/patient (including enabling patients/users to access the recommended
course of action) with operational imperatives, such as timeliness
 aligned data systems that enable analysis by and across the providers.

All stakeholders consulted are in agreement that a single approach across the
country would be ideal if secondary triage is scaled up. For a single approach to work
well providers would need to reach agreement on:
 a single tool
 classification of determinants that can be effectively dealt with by ‘hear and
treat’
 the transfer process used between 111 and secondary triage (warm transfer as
used in Wellington or call-back as used in Christchurch5 )
 whether there is a preference about where secondary triage clinicians should be
located6.

There was also agreement that roll-out of secondary triage needs an infrastructure
in place to support it. This would include:
 a range of local health services that users can be referred to
 a range of transport options to enable users to access recommended actions
from secondary triage

The evaluation highlighted different approaches to secondary triage, on a continuum
from a more process-oriented approach to a more diagnostic approach. At the

average cost.
5 A warm transfer occurs when the call taker remains on the line until the caller is connected to the next person
in the process. In a call-back process the caller is disconnected from the 111 call-taker and receives a call back
from the next person in the process.
6 These possibilities were raised in the course of the evaluation. While they have not yet been seriously
considered, they are one of a number of issues that will need to be addressed if a single approach to secondary
triage is to be agreed.
Evaluation of the secondary triage pilots in the Ambulance Communications Centres 4
topline findings workshop there was consideration of the possibility that both
approaches may be useful in particular circumstances. Feedback from the small
group of service users surveyed indicated that satisfaction with a more process-
oriented approach was higher for cases that users considered to be of a less serious
or urgent nature. This would suggest that a more diagnostic approach is better for
more serious cases, but these ideas would need to be tested further. Investigation
would also be needed to identify the ProQA determinants which may be better
suited to resolution by hear and treat and are relatively easy to close off, and those
which may require a more diagnostic approach.

For any roll-out of secondary triage there would need to be some standardization of
data capture and analysis across providers, and better processes for monitoring the
health outcomes of people using secondary triage. For example, in the short to
medium term there is a need to capture data on the number of re-presentations,
where a second 111 call has been received after secondary triage.

The single biggest issue that needs to be addressed if secondary triage is to be rolled
out is user/patient understanding of what secondary triage involves, why they are
being transferred, and the potential advantage of secondary triage for them.

Evaluation of the secondary triage pilots in the Ambulance Communications Centres 5
Introduction

The Ministry of Health (MoH) contracted Evaluation Works Ltd to conduct an
evaluation of the two secondary triage pilots being delivered, one in Canterbury and
one in Wellington. This document presents the evaluation findings and identifies key
issues for consideration going forward.
Background

The secondary triage pilots in the Ambulance Communications Centres (referred to
as communication or call centres) are part of the Urgent and unplanned care
integration project being led by MoH. This project is a national initiative focusing on
unplanned care, where any person needs to access care quickly. The project aims to
develop an integrated approach ‘where people are consistently triaged and
signposted to the care that is most appropriate to their needs’.

The secondary triage pilots have occurred over a period of continuous change in the
ambulance sector. Clinical support desks in the communication centres were
introduced in July in both Christchurch and Wellington, the new priority codes were
introduced in August and the pilots just a month after this.
The secondary triage pilots

The pilots focus on 111 calls that are deemed low acuity and appropriate for
secondary triage. Transfer of 111 calls to secondary triage means that, instead of an
emergency ambulance being immediately dispatched, further clinical triage is
provided for callers, and appropriate advice given on the health service they should
access and in what timeframe. The range of options that could be provided includes:
 Referral back to 111 for dispatch of an emergency ambulance or dispatch of
another health resource (e.g. acute demand )
 Advice to urgently attend an Emergency Department
 Advice to see GP or an after hours clinic in 4 or 8 hours
 Advice to see GP within 24 hours
 Deferred referral pathway, for example to see GP within 72 hours
 Patient self-care.

The objectives of the pilots are to:
 Provide people calling 111 for an ambulance with the lowest safe level of
care appropriate to their clinical need
 Contribute to the cost-effectiveness of that care, for example by a reduction
in the number of lower acuity patients unnecessarily taken to an emergency
department (ED) and improved utilization of current ambulance resources.

Evaluation of the secondary triage pilots in the Ambulance Communications Centres 6
The pilots, which commenced on 1 October 2012, are:
 The SouthComms pilot, which covers the St. John Canterbury dispatch area
(which is broadly consistent with the areas covered by Canterbury DHB)
 The CentralComms pilot, which covers the Wellington Free Ambulance
greater Wellington region and Wairarapa dispatch area (which equate to
areas covered by Capital and Coast, Hutt, and Wairarapa DHBs).

There are some variations in responsibility for, and delivery of, secondary triage in
the pilot locations.
Strategic overview
St. John is responsible for the SouthComms pilot. Secondary triage is a new
component added to core business, and as such, is located in a single organisation
with one culture. St John has developed a secondary triage tool (adapted from the
internationally recognised Briggs nursing triage tool) specifically for use in the pilot,
The triage tool was initially paper-based and then available on a tablet. From the
outset it was recognized that a more robust system tool would be needed if rolled
out further. The pilot was always viewed as a developmental, time-bound exercise to
inform future decisions about secondary triage.

Wellington Free Ambulance (WFA) in partnership with Medibank Health Solutions NZ
Ltd (currently New Zealand’s national Healthline provider) are responsible for the
CentralComms pilot. Medibank handles the secondary triage component of the
pilot, enabling each partner to concentrate on their core business albeit with some
new processes within and between each organisation. Medibank uses an established,
validated secondary triage tool. The algorithms are computer-based with a mature
training programme and continuing quality assurance. The two organisations are
located in different locations and have different cultures to be managed. The pilot
was approached with a view to the potential to be ongoing and scalable.
Delivery processes
Secondary triage is used with 111 incidents that are coded ‘grey’7 through the
Medical Priority Dispatch System (MPDS) ProQA used in all communication centres.
“These incidents have been selected based on:
 The statistical probability of the patient being low acuity
 The MPDS determination that the patient has no immediate threat to life
 The likelihood that further evaluation might identify an alternative patient
pathway (in some cases this may necessitate an urgent response)”8.

In the Canterbury dispatch area, all such incidents are transferred to secondary
triage; in the Wellington dispatch area, the majority of these are transferred to
secondary triage, while some are referred to ambulance dispatch as usual. The 111

7 The ambulance response system allocates 5 colour categories to determine the priority of emergency calls.
Purple is immediately life threatening; Red is immediately life threatening or time critical; Orange is urgent and
potentially serious but not immediately life threatening; Green is non-urgent (not serious or life threatening);
Grey is also non-urgent (not serious or life threatening) with further telephone triage appropriate.
8 Wellington Free Ambulance and St John (2012). Secondary Triage Initiatives: Ambulance Communications
Centres (COMMS) Education, Internal memo, 25 September 2012, pp 2,3.
Evaluation of the secondary triage pilots in the Ambulance Communications Centres 7