New Technology in Personnel Selection. The Validity and Acceptability of Multimedia Tests

163 Pages·2016·2.51 MB·English

by  Oostrom Janneke K.

Technology in
PPe el Selection
The Validity and
Acceptability of
Multimedia
Tests
Janneke K. Oostrom
New Technology in Personnel Selection:
The Validity and Acceptability
of Multimedia Tests

Janneke K. Oostrom
The research presented in this dissertation was supported in part by funding from
GITP International BV. The opinions expressed by authors are their own and do not
necessarily reflect the views of GITP International BV.

© 2010 New Technology in Personnel Selection: The Validity and Acceptability of
Multimedia Tests, Janneke K. Oostrom, Erasmus University Rotterdam

ISBN: 978-90-76269-85-6

Cover designed by Joost P. L. Modderman
Lay out by Janneke K. Oostrom
Printed by Ipskamp Drukkers
New Technology in Personnel Selection:
The Validity and Acceptability of Multimedia Tests

Nieuwe technologie in personeelsselectie:
de validiteit en acceptatie van multimedia tests

Proefschrift

ter verkrijging van de graad van doctor aan de
Erasmus Universiteit Rotterdam

op gezag van de
rector magnificus

Prof.dr. H.G. Schmidt

en volgens besluit van het College voor Promoties.

De openbare verdediging zal plaatsvinden op
vrijdag 1 oktober 2010 om 11.30 uur

door

Janneke Karina Oostrom
geboren te Krimpen aan den IJssel
Promotiecommissie
Promotoren: Prof.dr. M.Ph. Born
Prof.dr. H.T. van der Molen
Overige leden: Prof.dr. A.B. Bakker
Prof.dr. F. Lievens
Prof.dr. H. van der Flier
Copromotor: Dr. A.W. Serlie
Contents
Chapter 1 General introduction 7

Chapter 2 A multimedia situational judgment test with a constructed- 17
response item format: Its relationship with personality,
cognitive ability, job experience, and academic performance

Chapter 3 Webcam testing: Validation of an innovative open-ended 35
multimedia test

Chapter 4 Implicit trait policies in multimedia situational judgment 53
tests for leadership skills: Can they predict leadership
behavior?

Chapter 5 The role of individual differences in the perceived job 71
relatedness of a cognitive ability test and a multimedia
situational judgment test

Chapter 6 Pretest and posttest reactions to a paper-and-pencil and 93
a computerized in-basket exercise

Chapter 7 Summary and discussion 111

Samenvatting (summary in Dutch) 123

References 133

Dankwoord (acknowledgements in Dutch) 149

Curriculum Vitae 153

Kurt Lewin Institute dissertation series 157

Chapter 1

General introduction

The advances in technology of the last fifty years, specifically the advent of the

computer, its continuous improvements in functionality and capacity, and the growth
of the internet, have affected almost every aspect of psychological testing in person-
nel selection practices. Since the 1960s, traditional psychological tests with paper-
and-pencil formats are already being converted to computerized formats (Bartram,
1994). Yet, new technology provides more possibilities than simply changing the test
medium. For instance, it also provides the opportunity to dynamically select the items
to be presented and to use a variety of stimulus materials (Olson-Buchanan &
Drasgow, 1999). Recently, researchers and practitioners are using new technology for
the delivery of so-called multimedia tests, which include audio and video fragments
(Lievens, Van Dam, & Anderson, 2002).
The present dissertation presents five empirical studies on multimedia tests and is
aimed to address both theoretical and practical questions concerning their validity
and acceptability. In this introductory chapter, first, a short overview of the history of
computerized testing is given. Second, past research regarding multimedia testing
within the domain of personnel selection is discussed. Finally, the research aims of
the following five chapters of this dissertation are presented.

History of Computerized Testing
Already in the 1960s, some visionary test developers realized that psychological
tests could be efficiently and adequately administered via computers (Bartram &
Bayliss, 1984). During the late 1960s some of the earliest systems were designed to
automate the scoring procedures of psychological tests (Bartram, 2006).
In the 1970s, the first computer adaptive tests were launched (e.g., Brown & Weiss,
1977; Kreitzberg, Stocking, & Swanson, 1978; Weiss, 1973). These tests were admin-
istered via the computer and made use of item response theory to administer test
items that matches applicants’ ability level as determined by their performance on
previous items. However, early versions of computer adaptive tests were subject to a
number of constraints, such as high initial hardware and developmental software
costs (Kreitzberg et al., 1978).
In the 1980s, the first personal computers were introduced, which marks the be-
ginning of current approaches to psychological testing (Sands, Waters, & McBride,
1997). Since the introduction of personal computers, test developers started to
develop computerized versions of paper-and-pencil tests. However, these early
adaptations from paper-and-pencil tests to computerized tests were merely a change
in test medium. In order to take advantage of the potential of the computer for test
delivery, test developers tried to create an enhanced value through computerized
testing (McBride, 1997). Thus, the aim no longer was to simply transfer paper-and-
pencil tests to electronic page-turner versions, but to create so-called innovative
computerized tests. However, the possibilities for the development of innovative
computerized tests were restricted, as computers at that time were rather expensive,
and the storage, software, and multimedia capabilities still were limited.

9