Return on Investment in Training and Performance Improvement Programs

387 Pages·2003·2.411 MB·English

by  Jack J. Phillips, Ph.D. (Eds.)

Copyright © 2003, Jack J. Phillips. All rights reserved.
Butterworth–Heinemann is an imprint of Elsevier Science.
No part of this publication may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, electronic, mechanical,
photocopying, recording, or otherwise, without the prior written permission
of the publisher.
Recognizing the importance of preserving what has been written, Elsevier
Science prints its books on acid-free paper whenever possible.
Library of Congress Cataloging-in-Publication Data
Phillips, Jack J., 1945–
Return on investment in training and performance improvement
programs / Jack J. Phillips.—2nd ed.
p. cm.
Includes bibliographical references and index.
ISBN 0-7506-7601-9 (alk. paper)
1. Employees—Training of—Evaluation—Handbooks, manuals, etc.
2. Rate of return—Handbooks, manuals, etc. I. Title.
HF5549.5.T7P434 2003
658.3¢124–dc21
2003045312
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library.
The publisher offers special discounts on bulk orders of this book.
For information, please contact:
Manager of Special Sales
Elsevier Science
200 Wheeler Road
Burlington, MA 01803
Tel: [phone]
Fax: [phone]
For information on all Butterworth–Heinemann publications available,
contact our World Wide Web home page at: http://www.bh.com
10 9 8 7 6 5 4 3 2 1
Printed in the United States of America
Preface
he se of s xploding
T U ROI I E
Return on investment (ROI) has become one of the most challenging and
intriguing issues facing the human resources development (HRD) and
performance improvement field. The interest in ROI has been phenom-
enal. The topic appears on almost every HRD conference and conven-
tion agenda. Articles on ROI appear regularly in HRD practitioner and
research journals. Several books have been developed on the topic, and
consulting firms have been developed to tackle this critical and impor-
tant issue.
Several issues are driving the increased interest in ROI. Pressure from
clients and senior managers to show the return on their training invest-
ment is probably the most influential driver. Competitive economic pres-
sures are causing intense scrutiny of all expenditures, including all
training and development costs. Total quality management, reengineer-
ing, and Six Sigma have created a renewed interest in measurement and
evaluation, including measuring the effectiveness of training. The general
trend toward accountability with all staff support groups is causing some
HRD departments to measure their contribution. These and other factors
have created an unprecedented wave of applications of the ROI process.
eeded n ffective ethodology
N : A E ROI M
The challenging aspect of ROI is the nature and accuracy of its develop-
ment. The process often seems very confusing, surrounded by models,
ix
x Measuring Return on Investment
formulas, and statistics that often frighten the most capable practition-
ers. Coupled with this concern are misunderstandings about the process
and the gross misuse of ROI techniques in some organizations. These
issues sometimes leave practitioners with distaste for ROI. Unfortunately,
ROI cannot be ignored. To admit to clients and senior managers that
the impact of training or performance improvement cannot be measured
is to admit that training does not add value or that HRD should not
be subjected to accountability requirements. In practice, ROI must be
explored, considered, and ultimately implemented in most organizations.
What is needed is a rational, logical approach that can be simplified
and implemented within the current budget constraints and resources of
the organization. This book presents a proven ROI methodology, based
on almost 20 years of development and improvement. It is a process that
is rich in tradition and refined to meet the demands facing training and
performance improvement programs.
The ROI methodology described in this book meets the requirements
of three very important groups. First, the practitioners who have used
this model and have implemented the ROI process in their organizations
continue to report their satisfaction with the process and the success that
it has achieved. The ROI methodology presented here is user-friendly,
easy to understand, and has been proven to pay for itself time and time
again. A second important group, the clients and senior managers who
must approve training and performance improvement budgets, want
measurable results, preferably expressed as a return on investment. The
ROI methodology presented here has fared well with these groups.
Senior managers view the process as credible, logical, practical, and easy
to understand from their perspective. More importantly, it has their buy-
in, which is critical for their future support. The third important group
is the evaluation researchers who develop, explore, and analyze new
processes and techniques. When exposed to this methodology in a two-
day or one-week workshop, the researchers, without exception, give
this process very high marks. They often applaud the techniques for
isolating the effects of training and the techniques for converting data
to monetary values. Unanimously, they characterize the process as an
important—and needed—contribution to the field.
hy his ook at his ime
W T B T T ?
Since this book was first published, several ROI related books have been
developed. When examining current publications, there is still no other
book providing a comprehensive, practical presentation on ROI that
Preface xi
uses a process that meets the demands of the three groups previously
described. Most models and representations of the ROI process ignore,
or provide very little insight into, the two key elements essential to devel-
oping the ROI: isolating the effects of training and converting data to
monetary values. Recognizing that there are many other factors that will
have an influence on output results, this edition provides nine techniques
to isolate the effects of training with an example of each technique, far
more than any other presentation on the topic. Not enough attention
has been provided to the issue of assigning monetary values to the ben-
efits derived from training. This edition also presents ten techniques for
converting data to monetary values with an example of each technique.
The wide scale implementation of the ROI methodology has signifi-
cantly added to the ROI literature. Hundreds of papers, articles, cases,
and research reports have been developed and published. This new
edition captures the latest thinking on the use of ROI. Our consulting
group has also assisted hundreds of clients with the ROI methodology.
We have learned much from these assignments and have reflected the
experiences in this new edition.
The first edition of this book was developed at the request of many
clients and colleagues who have asked for a simplified, concise descrip-
tion of the ROI process, presented in a step-by-step approach. A new
edition has been requested and supported by clients and colleagues who
want the latest information on ROI. While the original book provided
a sound reference, this new edition provides expanded coverage of many
topics and issues important to the development and use of ROI. The
original book has become a standard reference in the HRD field and a
widely used workshop companion. This new edition should surpass the
previous success as it presents a model that is rational, feasible, and
understandable to the typical practitioner.
arget udience
T A
The primary audience for this book are managers and professionals
involved in training, development, and performance improvement.
Whether an individual is involved in needs assessment, instructional
design, delivery, or evaluation, this book will be an indispensable refer-
ence. Individuals in training and HRD leadership positions (i.e., man-
agers, supervisors, team leaders, directors, and vice presidents) will find
it to be a helpful guide to ROI. With its step-by-step approach and case
presentations, it will also be useful as a self-study guide, particularly
when used with the ROI field book.
xii Measuring Return on Investment
A second audience is the management group. Because of the tremen-
dous interest from the management team and clients involved in ROI
studies, this book should be a very useful reference for them. In a simple,
easy-to-understand approach, it shows how the ROI is developed in
language that managers understand.
A third audience are those individuals involved in the design and
implementation of change programs other than training, development,
and performance improvement. Because the ROI process is appropriate
for all change efforts, this book should be a useful reference for HR
managers, change agents, quality managers, reengineering coordinators,
and information technology specialists. The ROI process described in
this book has been used to measure the impact of a variety of human
resources programs, reengineering initiatives, and technology imple-
mentation situations, as well as in changes of procedures, practices, and
policies. Any time change is implemented, this process will capture the
return on investment.
The fourth target audience are consultants, researchers, and seminar
presenters who find this process to be an effective way to measure the
impact of their programs. It provides a workable model for consultants
to evaluate change initiatives or consulting interventions. The book pro-
vides researchers with a sound tool for evaluating a variety of programs.
Seminar presenters will find this book to be a helpful tool to measure
the success of their workshops. The ROI process has been applied effec-
tively in measuring the impact of public seminars.
Finally, professors and educators will find this book extremely useful
for evaluating education and training. Also, with this group, the book
should be a useful textbook or supplemental book for a course on
evaluation. Its content should stimulate thought and debate on how the
ROI is developed for the educational field.
eatures of the econd dition
F S E
To make the second edition a more valuable tool, several changes have
been made:
1. The introduction is updated to reflect the current material on
trends and issues.
2. A new detailed ROI case study on the evaluation of leader-
ship development is included, which contains ten parts. Leader-
ship is one of the most common applications of the ROI
methodology.
Preface xiii
3. The most recent version of the ROI model is presented.
4. Data collection is expanded to include collection for measuring
reaction (Level 1) and learning (Level 2) as well as measuring
application (Level 3) and impact (Level 4).
5. A chapter on ROI forecasting is added to reflect the use of the
ROI methodology to justify new programs.
6. The guiding principles are prominently displayed and described
in their first use.
7. The chapter on isolating the effects of programs has been
enhanced and includes the specific steps for participant’s estima-
tion, an interpretation of estimation, and the credibility and
reliability of the process.
8. The chapter on converting data to monetary values is expanded
to include more detail on how and when to convert intangible to
monetary values.
9. The chapter on costs has been expanded to show an actual
example of a fully loaded cost calculation.
10. The chapter on ROI is significantly expanded to include more
information on the interpretation, use, and abuse of ROI.
11. More examples are included to reflect a variety of different types
of programs and settings.
12. The references are updated, including those from all types of
processes, with 90% of the references after the year 2000.
13. All forms and templates have been updated.
14. A new chapter on communicating results is included to bring
attention to this important issue.
15. The chapter on intangible measures has been expanded to include
material on intellectual and human capital.
16. Additional resources are included at the end of the book. Also,
the book contains links to websites, software, casebooks, and a
newly developed ROI field book.
In all, these changes have made this second edition an absolute
requirement for the library of anyone interested in training and devel-
opment and performance improvement.
tructure of the ook
S B
This book has two unique features that make it a very useful guide. First,
it presents the ROI model in a step-by-step process. A chapter is devoted
to each major part of the model as the pieces of the ROI puzzle are
xiv Measuring Return on Investment
methodically put together. At the conclusion, the reader has a clear
understanding of the overall ROI process.
The second unique feature is an application of the model in a detailed
case that is based on an actual situation. The case is divided into ten
parts. One part is included at the end of each chapter, beginning with
Chapter 2. Readers can work through the case, step-by-step, exploring
the issues uncovered in the chapter and learn how to apply them to their
own organizations. The results of each part are presented in the next
chapter where a new issue is addressed. This case presentation is a
proven learning tool to understanding the ROI process.
hapter escriptions
C D
Chapter 1: Measuring the Return on Investment: Key Issues
and Trends
This chapter describes how the ROI process has evolved in recent years
and describes how organizations are tackling this important issue. Key
issues and trends are briefly described. Various ROI criteria and require-
ments are presented to build a foundation for the remainder of the book.
Chapter 2: ROI Model
Initially conceived in the late 1970s, the model has been developed,
changed, and refined in the past 25 years to arrive at what users char-
acterize as the most logical, rational, and credible approach to the ROI.
This chapter presents a brief summary of the model for those who are
being introduced to the methodology for the first time.
Chapter 3: Collecting Data
This chapter presents a variety of approaches to one of the most
fundamental issues. Ranging from conducting surveys to monitoring
performance data, the most common ways to collect data at all levels
are described in this chapter. Useful tips and techniques to help select
the appropriate method for a specific situation are presented.
Chapter 4: Isolating the Effects of Training
This chapter presents what is perhaps the most important aspect of
the ROI process. Ranging from the use of a control group arrangement
Preface xv
to obtaining estimates directly from participants, the most useful
techniques are presented for determining the amount of improvement
directly linked to the training program. The premise of this chapter is
that there are many influences on business performance measures with
training being only one of them.
Chapter 5: Converting Data to Monetary Benefits
This chapter presents an essential step for developing an economic
benefit from training. Ranging from determining the profit contribution
of an increased output to using expert opinion to assign a value to data,
the most useful techniques to convert both hard and soft data to mon-
etary values are presented, along with many examples.
Chapter 6: Tabulating Program Costs
This chapter details specifically what types of costs should be included
in the ROI formula. Different categories and classifications of costs are
explored in this chapter with the goal for developing a fully loaded cost
profile for each ROI impact study.
Chapter 7: Calculating the Return
This chapter describes the actual ROI calculation and presents several
issues surrounding its development, calculation, use, and abuse. The
most accepted ROI formulas are presented, along with examples to
illustrate the calculation. Common ROI myths are dispelled.
Chapter 8: Identifying Intangible Measures
This is a brief chapter that focuses on nonmonetary benefits from the
program. Recognizing that not all of the measures can or should be
converted to monetary values, this chapter shows how the intangible
measures should be identified, monitored, and reported. Over twenty-
five common intangible measures are examined.
Chapter 9: ROI Forecasting
This chapter shows how the return on investment can be used to fore-
cast the payoff of a program before it is implemented. Several examples
are presented to highlight each concept. This chapter underscores the
xvi Measuring Return on Investment
range of possibilities available for calculating the ROI at different time
frames, using different types of data.
Chapter 10: How to Communicate Results
This chapter provides best-practice approaches to communicate the
results of impact studies. The chapter details how to plan for commu-
nications, select audiences and media, develop impact studies, and
address typical issues that surface during communication.
Chapter 11: Implementation Issues
This concluding chapter addresses a variety of implementation issues. To
implement the ROI process effectively requires following logical steps
and overcoming several hurdles. This chapter identifies the important
issues that must be tackled for the ROI process to become a productive,
useful, and long-lasting process.
Acknowledgments
In no way has developing this book been a single-handed effort. Many
individuals have helped shape the content and issues contained in each
chapter. Much appreciation goes to our clients who provide us the
opportunity to continue to experiment with the ROI process. We have
had the opportunity to work with hundreds of excellent organizations
and individuals.
Special thanks go to the ASTD ROI Network Board for their unwa-
vering support for our ROI methodology: Merrill Anderson, Gwen
Berthiez, Tim Bothell, Holly Burkett, Jim Chatt, Toni Hodges, Brian
Howard, Don Jacklich, Dan McLinden, Brenda Sayres, Connie Schmidt,
Lynn Schmidt, Mike Sullivan, Uichi Tsutsumi, Kyoko Watanabe, and
Deb Wharff.
I would like to thank Francine Hawkins for her patience, persistence,
and long hours of work to make this manuscript a reality. Joyce Alff
stepped up to a challenging assignment and has provided excellent
leadership for our publishing programs. She is an excellent editor who
contributes a unique professional approach.
I must also acknowledge the important work and contribution of
my spouse, business partner, and friend, Patti Phillips, who has applied
this process regularly and made significant progress. Patti provided
numerous suggestions for the book and reviewed the manuscript. With-
out her continued support, this book would not be possible.
xvii