The Oxford handbook of evidence-based management

498 Pages·2012·7.117 MB·English

by  Denise M Rousseau

OX F O R D L I B R A RY O F P S Y C H O L O G Y
 . 
Editor-in-Chief
 . . 
Editor, Organizational Psychology
Th e Oxford Handbook
of Evidence-Based
Management
Edited by
Denise M. Rousseau
1
1
Oxford University Press, Inc., publishes works that further
Oxford University’s objective of excellence in
research, scholarship, and education.
Oxford New York
Auckland Cape Town Dar es Salaam Hong Kong Karachi
Kuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto
With offi ces in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Th ailand Turkey Ukraine Vietnam
Copyright © 2012 by Oxford University Press, Inc.
Published by Oxford University Press, Inc.
198 Madison Avenue, New York, New York 10016
www.oup.com
Oxford is a registered trademark of Oxford University Press
All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means,
electronic, mechanical, photocopying, recording, or otherwise,
without the prior permission of Oxford University Press
Library of Congress Cataloging-in-Publication Data
Th e Oxford handbook of evidence-based management / edited by Denise M. Rousseau.
p. cm. — (Oxford library of psychology)
ISBN-13: 978-0-19-976398-6 (acid-free paper)
ISBN-10: 0-19-976398-4 (acid-free paper) 1. Decision making. 2. Business intelligence. 3. Management.
I. Rousseau, Denise M.
HD30.23.O955 2012
658.4'03—dc23
2011036150
9 8 7 6 5 4 3 2 1
Printed in the United States of America
on acid-free paper
FOREWORD
We have a very long way to go to make management practice evidence-based. A
few years ago, while serving on the compensation committee for a publicly traded
NASDAQ company, we were considering what to do about the CEOs’ stock
options and our stock-option program in general. Just that day, articles appeared
in the mainstream business press on Donald Hambrick’s research showing that
stock options led to risky behavior (Sanders & Hambrick, 2007). Th at research
added to the growing body of evidence demonstrating that many executive pay
practices not only did not enhance company performance (Dalton, Certo, &
Roengpitya, 2003), but led instead to misreporting of fi nancial results (Burns &
Kedia, 2006). Th e fi nding that options led to riskier actions is logical. Once an
option is out of the money, there is no further economic downside for executives.
Th erefore, there is every incentive for managers to take big risks in the hope that
the stock price will go up, thereby giving those once previously worthless options
economic value.
At the meeting, a vice president from Aon Consulting who was advising the
compensation committee replied, “No,” without any hesitation or embarrassment
when I asked him, fi rst, if he knew about this research, and, second, if he was
interested in my sending him the original articles or other information about the
extensive research on stock options and their eff ects. What is particularly telling
is that many people from other compensation consulting fi rms to whom I have
related this story said it could have been their fi rm, too—that the perspective
refl ected is typical. Meanwhile, my colleagues on the compensation committee
seemed to believe that the fact that our professional advisor was both unknowl-
edgeable about and, even worse, uninterested in sound empirical research that
might inform our compensation policy decisions should not in any way aff ect our
continued reliance on his “advice.” Th is consulting example is all too typical of the
perspective of numerous, particularly U.S., executives, many of whom value their
“experience” over data, even though much research suggests that learning from
experience is quite diffi cult and much of such presumed learning is just wrong
(e.g., Denrell, 2003; Mark & Mellor, 1991; Schkade & Kilbourne, 2004).
Yes, it is true that there is much recent interest in investing in “big data” start-
ups and technology and in building software to comb through large data sets for
analytical insights, such as how to improve margins through pricing optimiza-
tion. However, as Jim Goodnight, the CEO of SAS Institute, a business analytics
and business intelligence software company, recently told me, that analytic work
is mostly done by professionals fairly far down in the company hierarchy who
sometimes have trouble getting the CEO interested in implementing their results.
Few companies or their leaders seem to see that information and data-driven
vii
business insight can be a competitive advantage. Although it may seem ironic
that companies invest in analytical capabilities and software that they don’t fully
use, such a circumstance is far from unusual. David Larcker, a well-known cost-
accounting professor, has said that few companies turn all the data they have
collected in their enterprise resource-planning systems into business intelligence
but instead use most of that information simply for purposes of control. And
purveyors of marketing research also tell a similar tale: companies purchase their
services but then don’t act on the results when the recommendations confl ict with
the preexisting views of senior management.
Th e problems confronting building evidence-based practice in management are
scarcely unique to this domain. It has taken more than 200 years for evidence-
based medicine to become a normative standard for professional practice. Th e
objections to applying standards of practice—protocols—in medicine are eerily
similar to those encountered in advocating evidence-based management: that each
situation is diff erent; that practitioners’ wisdom, experience, and insight is more
valuable than aggregated data-based information; and that statistical evidence
pertains to what happens “on average,” but that the particular individual speak-
ing is, of course, much above average and not, therefore, bound by the results
of aggregate information—a phenomenon reliably observed in psychology and
called the above-average eff ect (e.g., Williams & Gilovich, 2008). Although there
are evidence-based movements in various policy domains ranging from crimi-
nology to education, resistance to implementing science, particularly when that
science confl icts with belief and ideology, looms large.
Failure to adhere to clinical guidelines in medicine, unfortunately still quite com-
mon, costs lives and money (e.g., Berry, Murdoch, & McMurray, 2001). Pressures for
cost containment and the documented instances of literally hundreds of thousands
of lives lost from preventable medical errors have seemingly turned the tide so that
evidence-based medical practice now seems to be inevitable. However, the absence
of similar pressures in management means that the quest to bring evidence-based
practice into organizations still faces a long and tortuous journey. In the meantime,
damage occurs to both people and companies. According to Conference Board sur-
veys, job satisfaction is at an all-time low; most studies of employee engagement and
distrust of management portray a dismal picture of the typical workplace; and the
failure to heed the evidence about the physiological eff ects of workplace stress, eco-
nomic insecurity, and long working hours contributes to employee mortality and
morbidity (Pfeff er, 2010). Companies suff er from having disengaged workforces
with excessive turnover even as they neglect the science and information that could
improve their operations and profi tability (e.g., Burchell & Robin, 2011).
Simply put, there is a profound “doing-knowing” problem in management
practice: many managers make decisions and take actions with little or no con-
sideration of the knowledge base that might inform those decisions. Th e Oxford
Handbook of Evidence-Based Management intends to change this situation. It con-
tains more than just a review of the literature on evidence-based management
(EBMgt). Th e Handbook recognizes the need to provide information on how
to teach EBMgt in classes and executive programs and also the requirement to
viii 
provide role models—illustrations of practitioners who have embraced an
evidence-based approach—so that practicing leaders can see how evidence-based
principles might be applied in real organizations. Because EBMgt is relevant
in both the for-profi t and nonprofi t worlds, the handbook has examples from
both. With its focus on changing the practice of management, this work exam-
ines the important topic of how to get research fi ndings implemented in practice
and the barriers to connecting science with practice. Th e Oxford Handbook of
Evidence-Based Management provides a comprehensive overview of the informa-
tion required to understand what EBMgt is, how to teach it, how to apply it, and
how to understand and overcome the barriers that stand in the way of basing
management practice on the best relevant research. It represents an important
step on the road to building an EBMgt movement.
It is scarcely news that management is not a profession, even though it might
and should be. Th ere are two elements that defi ne a profession, one of which is
the development and adherence to specialized bodies of knowledge. Sometimes
specialized knowledge is refl ected in licensing examinations. However, licensing
is not the only path to ensuring that people know and implement knowledge and
standards of professional practice. What may be even more important are social
norms and public expectations and the sanctions for violating those expectations.
As the opening example—and hundreds of others—make clear, there is, at the
moment, neither the expectation that professionally educated practitioners will
know relevant management research nor any sanctions that punish their igno-
rance. Consequently, management, and indeed, much of the popular manage-
ment literature, is beset with myths, dangerous half-truths (Pfeff er & Sutton,
2006), and rules of thumb often based on little more than publicity, repetition,
and belief. Organizations and their leaders do a profound disservice to society
and even to themselves by not being more committed—not just in rhetoric but in
their behavior—to implementing management science.
In this project of professionalization, the role of educational institutions looms
large, which means that it is important that the handbook includes so much mate-
rial on teaching EBMgt and on linking science with practice. At the moment,
however, there is little to suggest that even in business-school classrooms, much
attention is paid to EBMgt. Business schools are increasingly staff ed by part-time
and adjunct faculty who have no requirement to know, let alone contribute, to
the science of management. Courses often use case examples as a primary peda-
gogic focus, and relatively few business schools impart the critical thinking and
evaluation of ideas and the skills necessary to separate good management science
from quackery. A recent review of more than 800 syllabi of required courses from
some 333 programs found that only about one-quarter utilized scientifi c evidence
in any form (Charlier, Brown, & Rynes, 2011). Th e fact that even business school
instructors often inhabit the world of folk wisdom rather than the domain of sci-
ence has sparked notice and commentary (e.g., Pearce, 2004).
If EBMgt is to become a reality in professional practice, business schools at
both the undergraduate and MBA levels will need to play an important role.
Th at’s because it is in school where students not only begin to learn about the
 ix
relevant research and theory but also come to understand standards of science and
what constitutes evidence and the sound basis for making decisions. It is inter-
esting that business schools are currently evaluated in many ways—the extent to
which they raise their graduates’ salaries, how “satisfi ed” they make their students
and the companies that recruit them, their adherence to guidelines that specify
what they must teach and the proportion of their faculty with terminal degrees,
their reputations as perceived by their peers, and various other criteria. What is
not measured is the extent to which they impart the science of management and
critical-thinking skills to their graduates, or the extent to which the graduates
actually practice EBMgt. If we are serious about implementing a science of man-
agement, we need to measure and evaluate both schools and their graduates, at
least to some degree, using these metrics.
We live in an organizational world. Decisions made in and by organizations
profoundly aff ect the working lives and economic well-being of people all over the
globe. As the fi nancial crisis of 2007–2008 well illustrates, many of those decisions
are all too frequently characterized not just by venality but also by profound incom-
petence (Lewis, 2010). As scholars and as practitioners whose work aff ects so many
lives in so many ways, we have a sacred obligation and responsibility to develop and
use the best knowledge possible to make the world a better place. It is only through
embarking on the goal of building an evidence-based body of knowledge linked to
professional practice that this obligation can be fulfi lled. In that regard, Th e Oxford
Handbook of Evidence-Based Management makes both a profoundly important con-
tribution and lays out a statement of intent to make EBMgt a reality.
—Jeff rey Pfeff er
References
Berry, C., Murdoch, D. R., & McMurray, J. J. V. (2001). Economics of chronic heart failure, European Journal
of Heart Failure, 3, 283–291.
Burchell, M., & Robin, J. (2011). Th e great workplace: How to build it, how to keep it, and why it matters. San
Francisco, CA: Jossey-Bass.
Burns, N., & Kedia, S. (2006). Th e impact of performance-based compensation on misreporting, Journal of
Financial Economics, 79, 35–67.
Charlier, S. D., Brown, K. G., & Rynes, S. L. (2011). Teaching evidence-based management in MBA programs:
What evidence is there? Academy of Management Learning and Education, 10, 222–236.
Dalton, D. R., Certo, S. T., & Roengpitya, R. (2003). Meta-analyses of fi nancial performance: Fusion or confu-
sion? Academy of Management Journal, 46, 13–28.
Denrell, J. (2003). Vicarious learning, undersampling of failure, and the myths of management, Organization
Science, 14, 227–243.
Lewis, M. (2010). Th e Big Short: Inside the Doomsday Machine, New York: W. W. Norton.
Mark, M. M., & Mellor, S. (1991). Eff ect of self-relevance on hindsight bias: Th e foreseeability of a layoff ,
Journal of Applied Psychology, 76, 569–577.
Pearce, J. (2004). What do we know and how do we really know it? Academy of Management Review, 29,
175–179.
Pfeff er, J. (2010). Building sustainable organizations: Th e human factor, Academy of Management Perspectives,
24, 34–45.
Pfeff er, J., & Sutton, R. I. (2006). Hard facts, dangerous half-truths, and total nonsense: profi ting from evidence-
based management. Boston: Harvard Business School Press, 2006.
Sanders, W. M., & Hambrick, D. C. (2007). Swinging for the fences: Th e eff ects of CEO stock options on
company risk-taking and performance, Academy of Management Journal, 50, 1055–1078.
Schkade, D. A., & Kilbourne, L. M. (2004). Expectation-outcome consistency and hindsight bias, Organizational
Behavior and Human Decision Processes, 49, 105–123.
Williams, E. F., & Gilovich, T. (2008). Do people really believe they are above average? Journal of Experimental
Social Psychology, 44, 1121–1128.
x 
PREFACE
What fi eld is this?
• “Th ere is a large research-user gap.”
• “Many practices are doing more harm than good.”
• “Practitioners do not read academic journals.”
• “Academics not practitioners are driving the research agenda.”
• “Practice is being driven more by fads and fashions than research.”
Th e fact is, these quotes all come from the early days of evidence-based medi-
cine (Sackett et al., 2000). More recently the same comments have been about the
(lack of) use of scientifi c evidence in management practice; no wonder, because
management practice today is as poorly aligned with organization and manage-
ment research as medicine and science were then. Scientifi c knowledge identi-
fi es what we know and what we don’t know about the natural and human-made
world. It is the bedrock of all evidence-based approaches to practice. From med-
icine to education, each area of evidence-based practice has needed to confront
these criticisms and work to make such comments descriptions of things past.
Evidence-based management (EBMgt) is the science-informed practice of man-
agement. It’s about using scientifi c knowledge to inform the judgment of manag-
ers and the nature, content, and processes of decision making in organizations.
Its practice reduces the costs of unaided judgment and limited and biased human
information processes. EBMgt allows managers, consultants, and other practitio-
ners to overcome these impediments through use of decision aids, practices, and
frameworks to support better quality decisions.
Th e notion of evidence-based practice is relatively new, but the vision of man-
agement informed by science is not. Mary Parker Follett a renowned management
consultant in the early twentieth century was perhaps the fi rst to recognize the
power of social science to inform management. Much of her infl uence today has
been channeled through the writings of Peter Drucker, her self-proclaimed disci-
ple, who advised practitioners to pay attention to scientifi c research and to pursue
feedback on the outcomes of their decisions more deliberately in order to learn
and perform better. Herbert Simon, the quintessential scientist and founder of
such fi elds as computer science, artifi cial intelligence, and robotics, identifi ed the
limits of unaided human decision making as well as the potential for science and
practice to partner in designing preferred conditions for our human-made world.
As we shall explore in this handbook, EBMgt is an adaptive evidence-informed
family of practices relevant to anyone seeking to improve how people and organi-
zations are managed. Th is handbook is intended to promote EBMgt’s broad use
in for-profi t businesses, nonprofi t organizations, and government. Management
means getting people together to accomplish some objective. Th is handbook is
xxiii
intended for anyone in an organization who steps up to make things happen by
organizing people, tasks, and processes. Th at includes executives, offi cers, and
department heads as well as by proactive workers and volunteers.
Th roughout history, management has refl ected fundamental beliefs about what
people can and should control in organizations. Manufacturing fi rms had no des-
ignated workplace safety function in the fi rst 100 years after the industrial revo-
lution began. Accidents were viewed as an act of God and, hence, could not be
reduced or managed. Periodically, through scientifi c and practice innovations,
such beliefs have changed. In its day, the notion of a planful enterprise man-
ager was a Utopian view, scarcely evident in the loosely tied small job shops that
comprised pre-industrial age shipbuilding or textile manufacturing. Practices pro-
moting effi ciencies in organizations became management’s job over subsequent
centuries, and by the twentieth century its functions included innovation and
marketing (Drucker, 2003). Th rough all these developments, the habits of mind
and knowledge used in the practice of management also changed.
Th e emergence of EBMgt in the twenty-fi rst century parallels in some ways
the systematization of fi nance and accounting in the last century. Management
concepts have evolved over time and with them the mental models of man-
agers and other practitioners. Nineteenth-century organizations for the most
part lacked fundamental concepts like the time-value of money. In the early
twentieth century, the concept of return on equity was developed and its com-
ponents explicated in order to better assess and analyze company performance.
Finance and accounting today have well-specifi ed logics that lead to wider con-
sistency in their practice and shared understandings of their meaning. In con-
trast, the logics that executives, managers, human-resource staff , and others use
in managing people, structuring organizations, and making strategic decisions
are far more inconsistent and ad hoc (Boudreau, 2010). EBMgt off ers evidence-
informed logics for eff ective management practice that complement other business
disciplines.
Why Now? Drivers of Evidence-Based Management
We can never understand the total situation without taking into account the evolving
situation. And when a situation changes we have not a new variation under the old fact,
but a new fact.
—Mary Parker Follett
Th e time is ripe for the emergence of EBMgt. A confl uence of factors (in italics
in the following sentence and the following paragraphs) provides unparalleled
opportunity to reconsider the fundamentals of managing organizations. Since
World War II, a large body of social science and management research has investigated
the individual, social, and organizational factors that impact managerial performance
and organizational eff ectiveness. It has produced hundreds of well-supported evi-
dence-based principles relevant to organizational decisions and practices. Aware-
ness of the large volumes of research relevant to real-world decisions is a major
factor in the current zeitgeist of evidence-based practice across innumerable pro-
fessions (e.g., Armstrong, 2010; Locke, 2009).
xxiv 
Th e Internet off ers broad access to scientifi c knowledge. From electronic libraries to
listservs, the Internet off ers fi ndings from organizational and management science
and opportunities to participate in communities interested in applying them.
Medicine and nursing, among other fi elds, have undergone their own Internet-
based transformations toward evidence-based practice. Using online sources, phy-
sicians and nurses regularly fi nd information about problems and decisions they
face. Th e result has been science-informed patient-care protocols, guidelines, and
procedures, turning clinical research into better patient outcomes. In our fi eld,
management research is becoming widely accessible through Google.scholar and
other web-based resources. Broadly available management research and summa-
ries provide the basis for more eff ective mental models, processes, and practices.
Th ese make it easier for managers to apply the products of scientifi c knowledge
daily in organizations.
Increasing awareness of the consequences from managerial decisions prompts
widespread concern with improving its quality. Advancements in human knowledge
give people new resources for tackling what once appeared to be intractable prob-
lems. Schools and criminal-justice institutions have been early adopters of evidence-
based practice, perhaps no surprise given that their reliance on public dollars can
require greater accountability (Orszag, 2010). Similarly, health-care managers are
being challenged to use the same evidential approach that their clinical practitio-
ners are using for patient care (Walshe & Randall, 2001). Th e movement toward
accountability is likely to grow, fueled by widely available information regarding
consequential decisions made with increasing global interdependence.
An evidence-based practice zeitgeist results from these forces across fi elds as
diverse as medicine, education, criminal justice, and public policy. Each of these
evidence initiatives refl ects a recognition that the well-being of people, organiza-
tions, and society in an increasingly vulnerable and interdependent world may
depend on how eff ectively we act on evidence. In the context of management, the
potential benefi ts from evidence use are no less.
Th is Handbook’s Organization
Th is handbook is designed with three goals in mind. First, it provides an overview
of key EBMgt concepts and puts them in context of broader eff orts promot-
ing evidence-based practice and the closing of research-practice gaps. Second, it
addresses the roles, contributions, and concerns of EBMgt’s three core constitu-
ents: practitioners, educators, and scholars, providing perspectives and resources
for each. Th ird, it incorporates critical perspectives to raise awareness of alterna-
tive views and possible unintended consequences and to stimulate future EBMgt
innovations. EBMgt’s development, adoption, and future progress depend on
the related eff orts of managers, consultants, and other practitioners who perform
EBMgt; the educators who help develop the professional skills on which EBMgt
rests; and the scholars who provide the basic research, summaries, and partner-
ships with practitioners that make EBMgt possible.
From health care to education, the many fi elds that now base core practices on
research fi ndings all began with the early adoption of evidence-informed practices
 xxv
by a small percentage of professionals (Rogers, 1995; Proctor, 2004). For this
reason, this handbook targets the early adopters in all three groups, that is, those
self-improving practitioners, educators, and scholars interested in learning and
innovating. Th e handbook is intended to inform all three communities, off ering
perspective, food for thought, and guides to action.
Th e Introductory section provides an overview of EBMgt and its facets and
puts these in context. I fi rst present its central elements and functions in my
chapter on envisioning EBMgt. Th en, because evidence-based practice began in
the fi eld of medicine, Barends, ten Have, and Huisman compare and contrast
EBMgt and its medical counterpart, addressing the commonalities as well as false
beliefs and misapprehensions regarding their association. Th e introductory sec-
tion concludes with van Aken and Romme’s chapter on design science, a broadly
applicable process originally developed by Herbert Simon (1997) that enables the
use of scientifi c knowledge to solve practical problems through the collaborations
between practitioners and scientists.
Th e Research section provides both resources and tools relevant to the role of
scholars and academic research in promoting EBMgt. Its fi rst set of chapters pre-
sent core domains of management and organizational research, addressing what
we know now and what we can do in future to advance each domain’s uptake
and use in EBMgt. “Micro” organizational behavior (OB), which I describe, is
perhaps the most mature research fi eld with a 100+ year history and consider-
able body of cumulative research. OB has yielded a host of evidence-based prin-
ciples to guide practice. “Macro” organizational theory and strategy, as Madhavan
and Mahoney detail, is a more recent fi eld of study characterized by disciplinary
disputes regarding the value of novelty versus cumulative research. Th ey call for
attention by macroscholars to moving this fi eld toward greater integration and
synthesis. Lastly, Frese, Bausch, Schmidt, Rauch, and Kabst describe the fi eld of
entrepreneurship, which has fostered cumulative research on critical issues related
to creating and managing start-ups and developing entrepreneurial capabilities.
Other Research chapters describe techniques and approaches to help make
research more accessible and useful to practice. A core aspect of all evidence-
based practice is systematic reviews (SRs). Th ese are summaries of a comprehen-
sive body of research undertaken to address a specifi c managerial (and sometimes,
too, scholarly) question. Briner and Denyer describe how SRs are conducted and
their role in making useful knowledge more readily available. Th e next two chap-
ters provide insights into how researchers can make their work more informative
and useful to practice. Giluk and Rynes present lessons learned from evidence-
based medicine to help us appreciate why practitioners might not believe, accept,
or act on the best available evidence, and how to overcome these barriers. Leung
and Bartunek address how scholarly research can be more eff ectively presented so
that practitioners fi nd it more memorable and easy to use.
Th e Practice section targets the realities of everyday management practice and
the role EBMgt can play in improving decision making and well-being of employ-
ees and organizations. Practicing EBMgt involves overcoming a variety of hurdles.
A fi rst set of Practice chapters provides resources for understanding practice issues.
xxvi 