Numbers, words and anonymity in 360-degree feedback

333 Pages·2017·1.24 MB·English

Loughborough University
Institutional Repository
Numbers, words and
anonymity in 360-degree
feedback: a qualitative study
ThisitemwassubmittedtoLoughboroughUniversity’sInstitutionalRepository
by the/an author.
Additional Information:
• A Doctoral Thesis. Submitted in partial fulfilment of the requirements
for the award of Doctor of Philosophy of Loughborough University. In
agreement with one of the case study companies, chapter five has not
been released for public access.
Metadata Record: https://dspace.lboro.ac.uk/2134/12564
Publisher: (cid:13)c Amanda Harrington
Please cite the published version.
This item was submitted to Loughborough University as a PhD thesis by the
author and is made available in the Institutional Repository
(https://dspace.lboro.ac.uk/) under the following Creative Commons Licence
conditions.

For the full text of this licence, please go to:
http://creativecommons.org/licenses/by-nc-nd/2.5/
Numbers, words and anonymity in
360-degree feedback:
a qualitative study

by

Amanda L. Harrington

Doctoral Thesis

Submitted in partial fulfilment of the requirements
for the award of
The Degree of Doctor of Philosophy of
Loughborough University
Abstract

Academic research in 360-degree feedback continues to be dominated
by a positivist approach with analyses of the feedback ratings. In
contrast, this qualitative study explores how people make sense of 360,
across the ‘chain of meaning making’ involving not only raters and
feedback recipients but also HR managers, facilitators and external
consultants. Two corporate case studies in the pharmaceutical sector
show how 360 evolves as a social process and carries a variety of
meanings in different organisations and management contexts. Quasi-
scientific rituals are revealed, demonstrating the existence of pseudo-
anonymity and of complex use of numerical ratings and narrative
comments. Woven alongside these corporate case studies is an
autoethnography, which examines emotional and cognitive responses to
two rounds of 360 asking for feedback on coaching performance. The
autoethnographic thread allows insights into the dynamic relationship
between academic and practitioner perspectives, as the researcher
moves between both worlds. This PhD makes three contributions: the
conceptualisation of 360 as a social process; the questioning of taken-for-
granted customs within 360; and a methodological contribution to the
development of autoethnographic practice.

KEYWORDS: 360-degree feedback; multisource feedback; anonymous
ratings; narrative comments; qualitative; McDonaldisation; encoding;
meaning-making.

- 2 -
Acknowledgements

Many moons ago, I went to Laurie Cohen to ask whether we could carry
out some research together. She said she had no time, but that she
would be happy to supervise me to do a PhD. That started me thinking,
and this thesis is the result.

My thanks, therefore, to Laurie for starting me on this journey and for
seeing me through to the end. My thanks also to Peter Ackers for being
steady as a rock. I was lucky enough to have not just one but two
supervisors who gave me helpful guidance and feedback, were interested
in my work and had unshakeable belief in my capability.

As an occupational psychologist, it was important to me to have someone
from that academic community overseeing my work. My thanks therefore
to John Loan-Clarke, who took on this role, helping to ensure that the
resulting work is at least potentially interesting for others in this field.

There were many others without whom this thesis would never have been
conceptualized or written:
 Kerry Young, whose voice rang in my ears throughout, (“structure
is everything”, “writing is re-writing”) and who taught me how to
structure, and how to use all those joining words; and who
generously read my drafts and helped me think how to restructure
and re-write
 Xiaozheng Zhang, who I sat next to on the first doctoral training
day, and with whom I have shared many times of mutual support
and encouragement
 Sara Bosley, who helped me to see that things were simpler than I
thought
 Bernadette Cass, Rachel Phillips and Peta Sandison, who all
coached me as I thought the next thoughts
- 3 -
 Sarah Dale of Creating Focus, who facilitated my first 360 as part
of this research, inspired me with her own writing, gave me
feedback on mine and encouraged me
 David Cooper of Lumus, who facilitated my second 360 as part of
this research, brought me back to the commercial world, gave me
feedback on my autoethnography chapter and with whom I
continue to discuss the details of 360 design and implementation
 John Arnold for all those many sessions of listening to my
complaints and not believing a word of them.
 Lesley Chandler, Mel Cowell, Tyra Till, Alex Charlton, Greg Foister
and many others who listened, and listened, laughed with me at
my struggles and cheered me on at every step.
 Marea O’Sullivan for being such a true and consistently
encouraging friend, passionate in her own learning and always
wanting more.
 Pamela Castro, Melanie Uhlmann and Paul Linde for their
companionship on the home straight, studying alongside me,
inspiring me with their learning, and keeping me focused
- 4 -
TABLE OF CONTENTS

ABSTRACT .............................................................................................. 2
ACKNOWLEDGEMENTS ........................................................................ 3
1 – INTRODUCTION .............................................................................. 11
2 - METHODOLOGY .............................................................................. 17
2.1 INTRODUCTION ............................................................................. 17
2.2 RELATIVISM AND INTERPRETIVISM ........................................... 18
2.3 NUMBERS: COUNTING PERFORMANCE .................................... 21
2.4 TRIANGULATION .......................................................................... 22
2.5 CREDIBILITY .................................................................................. 25
2.5.1 Theoretical position ................................................................... 26
2.5.2 Congruence between methodology and methods ..................... 28
2.5.3 Rigour ........................................................................................ 30
2.5.4 The analytic lens ........................................................................ 30
2.6 DEVELOPMENT OF RESEARCH QUESTIONS ............................ 31
3 – LITERATURE REVIEW .................................................................... 34
3.1 INTRODUCTION ............................................................................. 34
3.2 HISTORY AND DEVELOPMENT OF 360 ...................................... 34
3.3 PURPOSE AND VALUE OF 360 .................................................... 37
3.3.1 Assessment ............................................................................... 37
3.3.2 Development ............................................................................. 39
3.3.3 Organisational development ...................................................... 40
3.4 ANONYMITY AND ACCOUNTABILITY ......................................... 41
3.4.1 Disentangling the terminology ................................................... 41
3.4.1.1 Anonymity .......................................................................... 42
3.4.1.2 Confidentiality .................................................................... 43
3.4.1.3 Accountability ..................................................................... 44
3.4.2 The recipient .............................................................................. 45
3.4.3 The raters .................................................................................. 49
3.4.4 The impact of anonymity on participants’ understanding of the
content and process of 360 ................................................................. 52
3.5 Numerical ratings .......................................................................... 52
3.5.1 Reliability and validity ................................................................ 53
3.5.2 Performance Improvement ........................................................ 54
3.5.3 Self-Awareness ......................................................................... 56
3.5.4 Averaging .................................................................................. 57
3.5.5 The influence of rater goals and intentions ................................ 60
3.6 NARRATIVE COMMENTS ............................................................. 63
3.6.1 Recipients’ use of narrative comments ...................................... 64
3.6.2 Raters’ use of narrative comments ............................................ 65
3.6.3 Number and favourability of narrative comments ...................... 66
3.6.4 Managers’ reactions to ratings and narrative comments ........... 67
3.6.5 A qualitative approach to 360 .................................................... 68
3.7 RESEARCH METHODS ................................................................. 70
3.8 McDONALDISATION OF FEEDBACK .......................................... 73
3.8.1 Weber and Ritzer ....................................................................... 73
3.8.2 Applying Ritzer’s four principles ................................................. 74
3.8.2.1 Efficiency ........................................................................... 75
3.8.2.2 Predictability ........................................................................ 76
3.8.2.3 Calculability .......................................................................... 76
3.8.2.4 Control through technology .................................................. 77
3.8.3 Irrationalities of a rational system .............................................. 78
3.8.4 Simplification and Complexification ........................................... 80
3.8.5 Critique of McDonaldisation ....................................................... 81
3.9 STUART HALL’S ENCODING AND DECODING THEORY ........... 82
3.9.1 The making of meaning in 360, with reference to Hall ............... 83
3.9.2 Encoding/Decoding ................................................................... 84
3.9.2.1 Preferred reading ................................................................. 87
3.9.2.2 Oppositional reading ............................................................ 90
3.9.2.3 Negotiated reading ............................................................... 90
3.10 SUMMARY .................................................................................... 91
4 – RESEARCH DESIGN AND METHODS ........................................... 93
4.1 INTRODUCTION ............................................................................. 93
4.2 THE RESEARCH DESIGN: THREE CASE STUDIES ................... 94
4.3 CORPORATE CASE STUDIES ...................................................... 96
4.3.1 Case study guidelines ................................................................ 97
4.3.2 Choice of methods within the corporate case studies ................ 98
4.3.3 Interviews ................................................................................ 100
4.3.4 Interview sample ...................................................................... 103
4.3.5 Roles in the 360 process ......................................................... 104
4.3.5.1 The consultant ................................................................... 104
4.3.5.2 Vice President – Company A ............................................. 104
4.3.5.3 Central manager of the company’s 360 – Company B ....... 105
4.3.5.4 HR Managers ..................................................................... 105
- 6 -
4.3.5.5 Learning & Development Managers and Leadership Coach –
Company B .................................................................................... 105
4.3.5.6 Managers (who had received 360 and who had rated others)
....................................................................................................... 105
4.4 AUTOETHNOGRAPHIC CASE STUDY ....................................... 106
4.4.1 Reasons for choosing autoethnography .................................. 106
4.4.2 Boundarying the autoethnography ........................................... 107
4.4.3 Development of autoethnography ............................................ 108
4.4.4 Principles of analytic autoethnography .................................... 110
4.4.4.1 Complete member researcher (CMR) status ..................... 110
4.4.4.2 Uses analytic reflexivity ...................................................... 110
4.4.4.3 Has a visible narrative presence in the written text ............ 111
4.4.4.4 Engages in dialogue with informants beyond the self ........ 111
4.4.4.5 Is committed to an analytic research agenda focused on
improving theoretical understandings of broader social phenomena.
....................................................................................................... 112
4.4.5 Analytic versus evocative autoethnography ............................. 112
4.4.6 Design of the autoethnographic case study ............................. 113
4.5 ANALYSIS OF THE DATA ........................................................... 115
4.5.1 Transcription of the interviews ................................................. 115
4.5.2 Reflexivity ................................................................................ 116
4.5.3 Thematic analysis .................................................................... 117
4.6 ETHICS ......................................................................................... 121
4.7 EVALUATION CRITERIA ............................................................. 122
4.7.1 Contribution to the literature .................................................... 123
4.7.2 Development of theory and epistemological integrity .............. 123
4.7.3 Data collection and analysis .................................................... 125
4.7.3.1 Validity and reliability ......................................................... 125
4.7.3.2 Rigour ................................................................................ 126
4.7.3.3 Credibility and trustworthiness ........................................... 127
4.7.3.4 Persuasiveness ................................................................. 128
5 – ANALYSIS OF CASE STUDY A .................................................... 130
6 - ANALYSIS OF CASE STUDY B ..................................................... 131
6.1 INTRODUCTION ........................................................................... 131
6.1.1 Attitudes to research, security and creativity ........................... 131
6.1.2 Themes and subthemes .......................................................... 133
6.2 PURPOSE AND VALUE OF 360 .................................................. 134
6.2.1 Internally developed and owned .............................................. 136
6.2.2 A tool for relationship development ......................................... 142
6.2.3 A ‘snapshot’ ............................................................................. 145
6.2.4 360 for measurement............................................................... 147
- 7 -
6.3 ANONYMITY ................................................................................. 152
6.3.1 Safety ...................................................................................... 152
6.3.2 Pseudo-anonymity ................................................................... 153
6.3.3 Accountability .......................................................................... 157
6.4 NUMERICAL RATINGS ............................................................... 163
6.4.1 Numbers drive clarity ............................................................... 163
6.4.2 Graphical representation of the data ....................................... 164
6.4.3 The meaning of ‘3’ ................................................................... 165
6.4.4 Averaging ................................................................................ 167
6.4.5 The meaning of agreement ...................................................... 167
6.4.6 The meaning of difference ....................................................... 170
6.4.7 Interpreting the numbers .......................................................... 172
6.5 NARRATIVE COMMENTS ........................................................... 173
6.5.1 The relationship between comments and numbers ................. 173
6.5.2 The meaning of agreement ...................................................... 177
6.5.3 The meaning of difference ....................................................... 179
6.5.4 A message of respect .............................................................. 180
6.5.5 Interpreting text comments ...................................................... 182
6.6 SUMMARY AND CONCLUSION .................................................. 189
7 – AUTOETHNOGRAPHY OF A COACH USING 360 ....................... 192
7.1 INTRODUCTION ........................................................................... 192
7.1.1 My work with 360 ..................................................................... 192
7.1.2 Themes and subthemes .......................................................... 192
7.2 PURPOSE AND VALUE ............................................................... 193
7.2.1 Conversations .......................................................................... 194
7.2.2 Relationships ........................................................................... 196
7.2.3 Credibility ................................................................................. 197
7.2.4 Practice .................................................................................... 198
7.2.5 Conversations with myself ....................................................... 200
7.3 ANONYMITY ................................................................................. 203
7.3.1 Frustration ............................................................................... 203
7.3.2 Identifying the raters ................................................................ 207
7.3.3 Changing raters ....................................................................... 210
7.3.4 Follow-up and accountability ................................................... 213
7.4 NUMERICAL RATINGS ............................................................... 215
7.4.1 Graphical representation of the data ....................................... 216
7.4.2 Drive for simplicity ................................................................... 217
7.4.3 Agreement and difference ....................................................... 218
7.4.4 Interpreting the numbers .......................................................... 220
7.5 NARRATIVE COMMENTS ........................................................... 223
7.6 SUMMARY AND CONCLUSION .................................................. 224
- 8 -