The Construct Validity of the NEO PI-R Personality Inventory in High Stakes Employee Selection

335 Pages·2017·2.96 MB·English

The Construct Validity of the NEO PI-R
Personality Inventory in High Stakes
Employee Selection

By Gerry Fahey, B.E., B.A., M.B.A., M.A., M.Sc.

Dublin City University Business School

July 2017

Research Supervisors: Dr. Finian Buckley
Dr. Janine Bosak

A Thesis Submitted to Dublin City University Business School
in Partial Fulfilment of the Requirements for the Degree of
Doctor of Philosophy.
Declaration

I hereby certify that this material, which I now submit for assessment on the programme
of study leading to the award of Doctor of Philosophy is entirely my own work, and that I
have exercised reasonable care to ensure that the work is original, and does not to the best
of my knowledge breach any law of copyright, and has not been taken from the work of
others save and to the extent that such work has been cited and acknowledged within the
text of my work.

Signed ID No: 12212420 Date: 28.07.17

i
Acknowledgements

First of all I would like to thank my supervisor Dr. Finian Buckley for his
direction and advice on this journey. Indeed there were times when I had firmly
resolved to end the journey before completion but a conversation with Finian i nhis
office always had a calming effect on me. Secondly, and just as important, without
Dr. Janine Bosak’s invaluable assistance in keeping me on track and focussed I would
have gotten lost on the journey. The input from both was essential to get me to the
finishing line. Gerry Coynygham also provided me with invaluable help in dealing
with the methodological issues that arose during the journey .I owe a special word of
thanks to Margaret Galuszyska who helped me deal with the admin issues as they
arose. I should also mention that there are a number of academics that I had the good
fortune to encounter during my years of third level education that influenced my
choice of research topic and the approach that I took. To those un-named individuals I
am also very grateful.
At a more personal level my three children– Paul, Julie, and Frances – each of
whom in their own special way encouraged me on this path. Welcome breaks in
Escorial with Paul and his family as well as the yearly family skiing holiday with my
two daughters didn’t go amiss. Interacting with my four grandchildren I found to be a
great antidote to the tendency to the danger of becoming overly obsessive with my
research topic. Helping Frances prepare for her Leaving Certificate Maths and
Physics, while doing my research, was as much a help to me as it was to her. There
are also three people of some decades of lasting friendship to whom I also owe much
in getting to this journey’s end – Bob Roeder, Eric McGrath, and Chantal Ladias.

ii
Table of Contents

Declaration i

Acknowledgements ii

Table of Contents iii

List of Figures vii
List of Tables viii

Abstract x

Chapter 1 Introduction 1

Chapter 2 The Concept of Construct Validity 9

2.1 The pre 1955 Fragmented Model 10
2.2 The Unified Model 12

2.2.1 Cronbach and Meehl on Construct Validity 13
2.2.2 Loevinger on Construct Validity 16
2.2.3 Campbell and Fiske and the Multitrait-Multimethod
Approach 19
2.2.4 Messick’s and Embretson’s Contributions 21

2.3 The Importance of Construct Validation 26

Chapter 3 The Dimensions of Personality 32

3.1 The Big Five Dimensions of Personality 33
3.2 Personality and Behaviour in the Workplace 35
3.3 The Higher Order Structure of Personality 40

3.3.1 What are the Higher Order Factors? 42
3.3.2 The Higher Order Structure is Unbalanced 46

3.4 The Lower Order Structure of the Big Five 48

3.5 The Evidence for a Hierarchical Structure 50

3.5.1 Review of the Research in support of a GFP 51
3.5.2 Relevance to the Research programme 56

iii
3.6 Additional Measurement Issues 59

Chapter 4 Socially Desirable Responding 64

4.1 The Effect of Socially Desirable Responding on the
Measurement of the Big Five 65

4.1.1 The Case against a Socially Desirable
Responding Effect 68

4.1.1.1 The Case for Socially Desirable
Responding in the form of Faking Good 72

4.1.2 To what extent does Faking Good occur? 76
4.2 Faking Good is a form of Moral Hypocrisy 82

4.2.1 Batson’s Research on Moral Hypocrisy 84
4.2.2 Mazar’s Dishonesty Research Paradigm 90
4.2.3 Behavioural Economics Research and Moral Hypocrisy 95

4.3 Remedies for Dealing with Faking Good 98
4.4 Conclusions for the Research Reviewed 102

Chapter 5 Accounting for Impression Management 109

5.1 Lie and Related Scales 110

5.1.1 Unidimensional Lie Scales 112
5.1.2 Paulhus’s Socially Desirable Responding Measure 114
5.1.3 Construct Validity and the BIDR 115

5.2 Rank Order Selection Effects 123

5.2.1 Empirical Evidence for a Rank Order Effect 126

Chapter 6 Methodology Issues 131

6.1 Separating Substantive and Method Effects 133

6.1.1 Review of Extant MTMM Studies on the Higher
Order Structure of Personality 138

6.1.1.1 Extant MTMM Studies 138

6.2 Factor Analytic Considerations 145
6.3 The Construct Validity of the BIDR IM Scale 152

iv
6.3.1 Restriction of Range Issues 153
6.3.2 The Item Transparency of the Impression
Management Measure 159
6.3.3 The Context Effect on the Construct Validity of
the Bespoke BIDR-IM Measure used 161

6.4 Monte Carlo Simulation 167
6.5 The Research Hypotheses Tested 173

Chapter 7 Research Methods 177

7.1 Participants 178
7.2 Measures 178
7.3 Procedure 182
7.4 Analyses 183

7.4.1 Factor Analysis 184
7.4.2 IM Measure Cut Off Score 185
7.4.3 Cluster Analysis 187
7.4.4 CFA Invariance Analysis 189
7.4.5 Monte Carlo Simulations 190
7.4.6 Comparison with the Rosse, Stecher, Miller, 193
and Levin (1998) Study

Chapter 8: Results 195

8.1 Managerial Field Study 197

8.1.1 Descriptive Statistics 197

8.1.1.1. Comparison of Managerial and Validation
Samples Descriptive Statistics 201

8.1.2 Exploratory Factor Analysis 203
8.1.3 Confirmatory Factor Analysis 207

8.1.3.1 Comments on CFA Models Tested 210
8.1.3.2 Just Identified Model Comparison 218
8.1.3.3 Invariance Analysis 221

8.2 Cluster Analysis 224
8.3 Monte Carlo Simulations 230

8.3.1 Descriptive Statistics 230
8.3.2 Simulation Results 232
8.3.3 Rosse, Stecher, Miller, and Levin Study comparison 236

v
Chapter 9: Discussion 240

9.1 Establishing the Construct Validity of the NEO-PIR 244

9.1.1 The Higher Order Structure of the Big Five 245
9.1.2 The Link between Faking Good and the Higher
Order Structure of the Big Five 253

9.2 Construct Validity of the Bespoke BIDR-IM Measure 258
9.2.1 The Practical Implications of Construct Validity 262
9.3 The Psychology of Faking Good 266
9.4 Limitations and Suggestions for future research 271
9.5 Conclusions 277

References 280

vi
List of Figures

Figure 1 Embretson’s Universal System for Construct Validity
Figure 2 Big Five Higher Order Putative Structure, with First Order Factors
Stability and Plasticity loading on a General Factor of Personality

Figure 3 Big Five Higher Order Putative Structure, withS tability and Plasticity,
and no First Order General Factor of Personality

Figure 4 Big Five Higher Order Putative Structure, with the Big Five Factors
loading on a First Order General Factor of Personality

Figure 5 Markon et al.’s (2005) Unbalanced Big Five Higher Order Structure

Figure 6 Nomological Net for the Paulhus BIDR IM Scale

Figure 7 Flow Chart for Results Section of Chapter 8

Figure 8 Illustration of Model 7 tested in Confirmatory Factor Analysis

Figure 9 Two Higher Order Factor Model – Model Predicted Item Covariance
Matrix (Hoffman, 2017)

Figure 10 Plot of Frequency of Occurrence against Impression Management
Scores for Cluster 1 compared to full sample of participants

Figure 11 Plot of Frequency of Occurrence against Impression Management
Scores for Cluster 2 compared to full sample of participant s

vii
List of Tables

Table 1 Comparison of NEO-PIR and HPI facet loadings
Table 2 Primary and secondary factor loadings of the Facets of Extraversion
and Conscientiousness

Table 3 Descriptive Statistics for the Full Sample of 443 Participant s
Table 4 Comparison of Big Five Mean Scores
Table 5 Intercorrelations between the Big Five dimensions
Table 6 Intercorrelations between the Big Five Dimensions of the Managerial
Sample and the Validation Sample for participants with IM scores < 12

Table 7 Means and Standard Deviations for the Big Managerial and the
Validation Samples

Table 8 EFA Variance Accounted for by Factors with an eigenvalue > 1
Table 9 EFA Factor Loadings on the Two Higher Order Factors

Table 10 Variance Accounted for by Factors with an eigenvalue > 1 in the
Validation and Managerial Samples for participants with an IM score
<12

Table 11 EFA Factor Loadings on the Two Higher Order Factors in the
Validation and Managerial Samples for participants with an IM score
<12

Table 12 CFA Goodness of Fit Indices for Participants with Impression
Management scores less than 12

Table 13 Comparisons of ‘Summed Score’ CFA models with ‘Facet Score’
models

Table 14 Comparison of expected and observed factor loadings

Table 15 Invariance tests of Managers and Validity samples

Table 16 Big Five Mean Scores comparison between Field Study samples

viii
Table 17 Mean Scores for the 2 Cluster Groups based IM, Agreeableness,
Conscientiousness

Table 18 Frequency Table of IM scores in Total and in Cluster 2

Table 19 Cohen’s ‘d’ Effect Sizes for Participants in Cluster 1 with IM Score of
12 or greater, compared with all other Participants.

Table 20 Correlations between the Predictor measures used in the assessments

Table 21 Simulation results of proportion of simulations containing Fakers, and
the proportion of times a Faker is selected

Table 22 Effect of number of Job Applicants in Selection Set on proportion of
simulations containing Fakers, and the proportion of times a Faker is
selected

Table 23 Effect of Predictor Set on Rank Order of Job Applicants
Table 24 Effect of Different Cut-off Hurdles on proportion of simulations
containing Fakers, and the proportion of times a Faker is selected

Table 25 Means and Standard Deviations of the bespoke BIDR-IM scale for
different groups

Table 26 Effect sizes for Mean Score Differences in Impression Management
scores between groups

ix