Individual Differences in Judgment Accuracy in Personnel Selection

208 Pages·2015·3.31 MB·English

INVITATION
You are cordially invited
to the PhD defence of
François S. de Kock

Venue:
Senaatzaal
(Woudestein complex),
Erasmus University,
Rotterdam
Promotors:
Prof.dr. M.Ph. Born
and Prof.dr. F. Lievens
Date and time:
3:30 pm, Thursday
17 December 2015
Paranymph:
dr. G.(Gera) Noordzij
E: [email protected]
Tel:[phone]
The defence ceremony
is followed by refreshments
in Cum Laude.
Proefschrift_FranciosCover+Bladwijzer.indd 4 12-11-15 23:16

Individual Differences in Judgment
Accuracy in Personnel Selection
What Makes the ‘Good Judge’?

François de Kock
The research presented in this dissertation was supported in part by the Andrew W.
Mellon Foundation, National Research Fund (NRF), University of Cape Town
Research Office, Police Academy of the South African Police Services, Military
Psychological Institute of the South African Military Health Services, and the
University of Stellenbosch.

© 2015 Individual Differences in Judgment Accuracy in Personnel Selection: What
Makes the ‘Good Judge’?, François S. de Kock, Erasmus University Rotterdam

ISBN 978-94-6299-245-0

Cover designed by Linda van Zijp, StudioLIN, Rotterdam
Lay-out by François S. de Kock
Printed by Ridderprint B.V., Ridderkerk
Individual Differences in Judgment Accuracy
in Personnel Selection: What Makes the ‘Good Judge’?

Individuele verschillen in beoordelaarsnauwkeurigheid
bij personeelsselectie – Wat kenmerkt de ‘goede beoordelaar’?

Proefschrift

ter verkrijging van de graad van doctor aan de
Erasmus Universiteit Rotterdam
op gezag van de
rector magnificus

Prof.dr. H.A.P. Pols

en volgens besluit van het College voor Promoties.
De openbare verdediging zal plaatsvinden op

donderdag 17 december 2015 om 15:30 uur

door

François Servaas de Kock
geboren te Uitenhage, Zuid-Afrika
Promotiecommissie
Promotoren: Prof.dr. M.Ph. Born
Prof.dr. F. Lievens

Overige leden: Prof.dr. A.B. Bakker
Prof.dr. R.E. de Vries
Prof.dr. H.T. van der Molen
Contents

Chapter 1: Introduction: Individual differences in judgment accuracy 7
in personnel selection: What makes the ‘good judge’?

Chapter 2: The profile of the ‘good judge’ in HRM: A systematic review 29

Chapter 3: The internal factor structure of dispositional reasoning 61

Chapter 4: An in-depth look at dispositional reasoning and interviewer 89
judgment accuracy

Chapter 5: Does it take one to know one? Interviewer personality, 115
chronically accessible traits, and trait judgment accuracy

Chapter 6: Summary and discussion 151

Nederlandse samenvatting [summary in Dutch] 165

References 173

Curriculum Vitae 201

Dankwoord [Acknowledgements in Afrikaans] 203

Introduction 7

Chapter 1
Introduction: Individual differences in judgment accuracy in
personnel selection – What makes the ‘good judge’?

The judgment accuracy of assessors has been an enduring research topic in personnel
selection studies. Assessors produce ratings that affect the quality of personnel selection
decisions. Although it is well known that assessors differ in judgment accuracy, we do not yet
understand why this is so. This dissertation drew on social cognition literature and judgment
accuracy models (Funder, 1999) to study assessor constructs that may predict their judgment
accuracy in personnel selection. In order to advance contemporary practices designed to select
and train assessors, an integrative profile of the ‘good judge’, informed by empirical evidence,
is needed. The dissertation therefore presents four studies – one systematic review and three
empirical studies – that investigated individual difference constructs in judgment accuracy
within a personnel selection context. First, a systematic review of empirical literature was
conducted, which, in addition to determining what we know and do not know about the good
judge, identified focal constructs for further empirical research. In the subsequent empirical
investigations, the role of specific individual difference constructs in judgment accuracy was
explored. The dissertation advances an understanding of how dispositional reasoning (the
complex knowledge of traits, behaviors, and situations’ potential to elicit traits into manifest
behaviors) and personality trait chronic accessibility (the degree to which individuals differ in
the readiness with which constructs are utilized in information processing of behavioral
stimulus input) may be characteristics of the good judge in personnel selection. The general
project goal was to determine the extent to which assessor individual differences are able to
explain judgment accuracy in personnel selection ratings.
8 Chapter 1

The search for the good judge of personality is the oldest pursuit in the accuracy
literature and was nearly its sole concern during the early incarnation from the
1930s…the prey proved to be unexpectedly elusive. Despite the research attention
it has received, the good judge is the potential moderator concerning which the
accuracy literature has the sparsest data and fewest firm findings to report.
Funder (1999, p. 142)
Modern personnel selection approaches rely heavily on assessors as judges of
applicants’ characteristics. Assessors may include interviewers, assessment centre
observers, or line managers observing work-sample performances. Assessors are
typically called upon to produce subjective ratings of applicants’ performances in
selection procedures. Eventually, these ratings form the lifeblood of important
personnel selection decisions about offers of employment or promotions. Given their
centrality in human resource management, it is surprising that our understanding of
rater characteristics, which may affect rating quality in personnel selection, has not
developed correspondingly to other factors in personnel selection (cf. Guion &
Gibson, 1988; Hough & Oswald, 2000; Sackett & Lievens, 2008).
Over the last few decades, consistent evidence shows that individual assessors
appear to differ in their judgment accuracy (e.g., Borman, Eaton, Bryan, & Rosse,
1983; Dipboye, Gaugler, & Hayes, 1990; Heneman, Schwab, Huett, & Ford, 1975;
Kinicki, Lockwood, Hom, & Griffeth, 1990; Pulakos, Schmitt, Whitney, & Smith,
1996; Ryan & Sackett, 1989; Sackett & Wilson, 1982; Schneider & Bayroff, 1953; Van
Iddekinge, Sager, Burnfield, & Heffner, 2006; Zedeck, Tziner, & Middlestadt, 1983).
However, the reasons for these individual differences in judgment accuracy are not
well known. The present dissertation reports on assessor constructs as potential
explanatory variables for the variance in judgment accuracy outcomes. In this
dissertation, assessors constructs are treated as ‘individual difference’ variables that
“are linked to differences in job [criterion] performance” (Salgado, Viswesvaran, &
Ones, 2001, p. 166), referring in this case to assessors’ ability to produce accurate
ratings in personnel selection. Individual differences between assessors that are
relevant and which find easy expression in behavior are abilities (cognitive ability
and physical ability), personality (including social skills, emotional intelligence, and
dark traits), interests and self-evaluations (Murphy, 2012). This research on which
this dissertation reports, employed a broad framework (See Figure 1.1) of
psychological individual difference constructs (adapted from Farr & Tippins, 2010)
as a lens to study the assessor. More specifically, this dissertation sought to advance
understanding of how assessor constructs predict judgment accuracy in personnel
selection.