Return on Investment in Meetings & Events: tools and techniques to measure the success of all types of meetings and events

302 Pages·2008·1.59 MB·English

by  Jack J. Phillips

RETURN ON INVESTMENT
IN
MEETINGS AND EVENTS
This page intentionally left blank
RETURN ON INVESTMENT
IN
MEETINGS AND EVENTS
TOOLS AND TECHNIQUES TO MEASURE
THE SUCCESS OF ALL TYPES OF
MEETINGS AND EVENTS
JACK J. PHILLIPS, PhD
M. THERESA BREINING, CMP, CMM
PATRICIA PULLIAM PHILLIPS, PhD
AMSTERDAM•BOSTON•HEIDELBERG•LONDON•NEWYORK•OXFORD
PARIS•SANDIEGO•SANFRANCISCO•SINGAPORE•SYDNEY•TOKYO
Butterworth-HeinemannisanimprintofElsevier
Butterworth-HeinemannisanimprintofElsevier
LinacreHouse,JordanHill,OxfordOX28DP,UK
30CorporateDrive,Suite400,Burlington,MA01803,USA
Firstedition2008
Copyright©2008,JackJ.Phillips,M.TheresaBreining,andPatriciaPulliamPhillips.
PublishedbyElsevierInc.Allrightsreserved
TherightofJackJ.Phillips,M.TheresaBreining,andPatriciaPulliamPhillipstobe
identifiedastheauthorsofthisworkhasbeenassertedinaccordance
withtheCopyright,DesignsandPatentsAct1988
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem
ortransmittedinanyformorbyanymeanselectronic,mechanical,photocopying,
recordingorotherwisewithoutthepriorwrittenpermissionofthepublisher
PermissionsmaybesoughtdirectlyfromElsevier’sScience&TechnologyRights
DepartmentinOxford,UK:phone:[phone],fax:[phone],
email:[e-mail]tyourrequestonline
byvisitingtheElsevierwebsiteathttp://elsevier.com/locate/permissions,andselecting
ObtainingpermissiontouseElseviermaterial
Notice
Noresponsibilityisassumedbythepublisherforanyinjuryand/ordamagetopersons
orpropertyasamatterofproductsliability,negligenceorotherwise,orfromany
useoroperationofanymethods,products,instructionsorideascontained
inthematerialherein
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-7506-8338-8
ForinformationonallButterworth-Heinemannpublications
visitourwebsiteatbooks.elsevier.com
PrintedandboundintheUSA
08 09 10 11 10 9 8 7 6 5 4 3 2 1
Working together to grow
libraries in developing countries
www.elsevier.com | www.bookaid.org | www.sabre.org
Contents
Preface, xvii
About the Authors, xxi
Acknowledgments, xxiii
Foreword – James J. McNamara, Chairman and CEO,
iDNA, Inc., xxv
1 The Need for the ROI Methodology in the Meetings
and Events Industry, 1
Industry Efforts for Meeting Professionals International, 2
Gaylord Hotels: Delivering Value is the Key, 3
ROI Institute, Inc.: Dispelling the Myths is a Critical Issue, 4
Concepts Worldwide: The Language of Business, 5
The Chapter Perspective: Cracking the ROI Code, 6
The Debate About ROI, 8
The Fifth Element, 9
The Great Debate, 10
ROI Will Not Go Away, 12
ROI Issues, 12
The Ultimate Level of Evaluation: ROI, 12
Best Practices, 14
ROI Myths, 17
The Benefits of ROI, 21
Final Thoughts, 22
References, 22
v
vi Contents
2 The ROI Methodology: A Brief Overview, 23
A Paradigm Shift, 23
Key Steps and Issues, 25
Stakeholders, 25
Levels and Steps, 27
Chain of Impact, 27
The ROI Process Model, 29
Objectives, 29
Evaluation Planning, 32
Data Collection, 32
Analysis, 33
Isolation of the Effects of Meetings, 34
Conversion of Data to Monetary Values, 34
The Cost of Meetings, 35
The Return on Investment Calculation, 35
Intangible Benefits, 37
Data Reporting, 37
Operating Standards, 37
Implementation Issues, 38
Final Thoughts, 39
3 The Alignment: Defining Needs and Objectives, 40
The Meeting Planner’s Role in Alignment, 40
The First Alignment Opportunity: Needs Analysis, 42
Linking Need with Evaluation, 42
Business Alignment, 43
Payoff Needs, 44
Probe for Details, 45
Obvious versus not so Obvious, 46
The Reasons for Meetings and Events, 47
Determining Costs of the Problem, 48
The Value of Opportunity, 49
To Forecast or not to Forecast, 50
Business Needs, 50
Define the Measure—Hard Data, 50
Define the Business Need—Soft Data, 52
Tangible versus Intangible—A Better Approach, 54
Data Sources, 55
Identifying all the Measures, 56
Performance Needs, 56
Contents vii
Learning Needs, 57
Subject Matter Experts, 58
Job and Task Analysis, 58
Observations, 58
Demonstrations, 59
Tests, 59
Management Assessment, 59
Preference Needs, 59
Input Needs, 60
Developing Objectives for Meetings and Events, 61
Input Objectives, 62
Reaction Objectives, 63
Learning Objectives, 63
Application and Implementation Objectives, 64
Impact Objectives, 66
ROI Objectives, 67
The Importance of Specific Objectives, 67
Case Study, 69
Background, 69
Initial Needs Assessment, 70
The Meeting, 70
How to Make the Transition, 71
Final Thoughts, 72
Reference, 72
4 Measuring Inputs and Indicators, 73
The Importance of Measuring Inputs and Indicators, 73
Input Categories, 74
Tracking Meetings and Events—Topics and Themes, 74
Tracking People, 74
Tracking Duration, 75
Tracking Coverage, 75
Tracking Requests, 76
Tracking Technology Use, 77
Tracking Costs, 78
Tracking Efficiencies, 81
Tracking Outsourcing, 81
Key Issues, 81
Final Thoughts, 82
Reference, 82
viii Contents
5 Measuring Reaction and Perceived Value, 83
Importance of Measuring Reaction and Perceived
Value, 83
Customer Service, 83
Forecasting Capability, 84
For Some, This Is the Most Important Data, 84
Compare Data with Other Meetings, 85
Macro-Level Scorecards, 85
Data Collection Issues, 85
Sources of Data, 85
Content versus Non-Content, 86
The Deceptive Feedback Cycle, 87
Key Areas for Feedback, 88
Data Collection Timing, 91
Questionnaires and Surveys, 92
Designing Questionnaires/Surveys, 92
Intensities, 95
Samples Surveys, 95
Securing High Response Rates, 96
Interviews and Focus Groups, 98
Improving Reaction Evaluation, 98
Keep Responses Anonymous, 98
Have a Neutral Person Collect the Feedback, 98
Provide a Copy in Advance, 99
Explain the Purpose of the Feedback and How it Will
be Used, 99
Explore an Ongoing Evaluation, 99
Consider Quantifying Ratings, 99
Collect Information Related to Improvement, 100
Allow Ample Time for Completing the Form, 100
Delayed Evaluation, 100
Ask for Honest Feedback, 101
Using Data, 101
Monitor Customer Satisfaction, 101
Identify Strengths and Weaknesses, 101
Evaluate Speakers, 101
Determine Participant Needs, 102
Evaluate Planned Improvements, 102
Develop Norms and Standards, 102
Link with Follow-up Data, 102
Marketing Future Meetings, 102
Building the Macro-Level Scorecard, 103
Contents ix
Final Thoughts, 103
References, 103
6 Measuring Learning and Confidence, 104
The Importance of Measuring Learning, 104
Learning versus Entertaining, 104
The Learning Organization, 105
The Compliance Issue, 105
The Development and Use of Competencies, 105
The Role of Learning in Meetings and Events, 106
The Chain of Impact, 106
Consequences of an Unprepared Workforce, 106
Measurement Issues, 106
The Challenges of Measuring Learning, 107
Objectives, 107
Typical Measures, 107
Timing, 108
Data Collection Methods, 109
Questionnaires and Surveys, 109
Objective Tests, 109
Performance Tests, 110
Technology and Task Simulations, 112
Case Studies, 113
Role Playing and Skill Practices, 114
Exercises/Activities, 114
Informal Assessments, 114
Administrative Issues, 115
Reliability and Validity, 115
Consistency, 117
Monitoring, 117
Pilot Testing, 117
Readability, 117
Scoring, 117
Reporting, 118
Confronting Test Failures, 118
Using Learning Data, 118
Ensuring That Learning Has Been Acquired, 118
Providing Individual Feedback to Build Confidence, 118
Improving the Meeting, 119
Evaluating Speakers, 119
Building a Database, 119