The Wiley handbook of psychometric testing : a multidisciplinary reference on survey, scale, and test development

1032 Pages·2018·9.41 MB·English

by  Irwing

The Wiley Handbook
of Psychometric Testing
The Wiley Handbook
of Psychometric Testing
A Multidisciplinary Reference
on Survey, Scale and
Test Development
Volume One
Edited by
Paul Irwing
Tom Booth
David J. Hughes
Thiseditionfirstpublished2018
©2018JohnWiley&SonsLtd
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recordingor
otherwise,exceptaspermittedbylaw.Adviceonhowtoobtainpermisiontoreusematerialfromthis
titleisavailableathttp://www.wiley.com/go/permissions.
TherightofPaulIrwing,TomBooth,andDavidJ.Hughestobeidentifiedastheauthorsofthe
editorialmaterialinthisworkhasbeenassertedinaccordancewithlaw.
RegisteredOffices
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UK
EditorialOffice
TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UK
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproductsvisit
usatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontent
thatappearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorhaveusedtheirbesteffortsinpreparingthisbook,theymakeno
representationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisbook
andspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.Itis
soldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservicesandneither
thepublishernortheauthorsshallbeliablefordamagesarisingherefrom.Ifprofessionaladviceorother
expertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
LibraryofCongressCataloging-in-PublicationData
Names:Irwing,Paul,editor.
Title:TheWileyhandbookofpsychometrictesting:amultidisciplinaryreference
onsurvey,scaleandtestdevelopment/editedbyPaulIrwing,
ManchesterUniversity,UK,TomBooth,TheUniversityofEdinburgh,
Edinburgh,UK,DavidJ.Hughes,ManchesterBusinessSchool,Manchester,UK.
Description:FirstEdition.|Hoboken:Wiley,2018.|Includesbibliographical
referencesandindex.|
Identifiers:LCCN2017041032(print)|LCCN2017061203(ebook)|
ISBN9781118489826(pdf)|ISBN9781118489703(epub)|
ISBN9781118489833(cloth:alk.paper)|ISBN9781119121176(pbk.:alk.paper)
Subjects:LCSH:Psychometrics.|Psychologicaltests.
Classification:LCCBF39(ebook)|LCCBF39.W5452018(print)|
DDC150.28/7–dc23
LCrecordavailableathttps://lccn.loc.gov/2017041032
Coverimage:PrintedwithpermissionfromAnnaBrown
CoverdesignbyWiley
Setin10/12ptGalliardbySPiGlobal,Pondicherry,India
10 9 8 7 6 5 4 3 2 1
Contents
Notes on Contributors to Volume 1 vii
Preface xi
Introduction xiii
VOLUMEI
Part I Practical Foundations 1
1 Test Development 3
Paul Irwing and David J. Hughes
2 ClassicalTest Theory and Item Response Theory 49
Christine E. DeMars
3 Item Generation 75
Kristin M. Morrison andSusan Embretson
4 SurveySampling and PropensityScore Matching 95
Bo Lu and Stanley Lemeshow
5 Sample SizePlanning for ConfirmatoryFactor Models:
Powerand Accuracyfor Effects of Interest 113
Ken Kelley andKeke Lai
6 Missing Data Handling Methods 139
Craig K. Enders and Amanda N. Baraldi
7 Causal Indicators in Psychometrics 187
Aja L. Murray and Tom Booth
Part II Identifying and Analyzing Scales 209
8 Fundamentals of CommonFactor Analysis 211
Stanley A.Mulaik
vi Contents
9 Estimation Methods in Latent Variable Models for Categorical
Outcome Variables 253
Li Cai and Irini Moustaki
10 Rotation 279
Robert I.Jennrich
11 The Number of Factors Problem 305
Marieke E.Timmerman, Urbano Lorenzo-Seva, andEva Ceulemans
12 Bifactor Models in Psychometric Test Development 325
Fang FangChen and Zugui Zhang
13 Nonnormality in Latent Trait Modelling 347
Dylan Molenaarand Conor V. Dolan
14 Multidimensional Scaling: An Introduction 375
William G. Jacoby and David J. Ciuk
15 Unidimensional Item Response Theory 413
Rob R. Meijerand Jorge N. Tendeiro
Notes on Contributors to Volume 1
Amanda N. Baraldi is Assistant Professor of Psychology at Oklahoma State
University. Dr. Baraldi received her doctorate in Quantitative Psychology from the
Arizona State University in 2015. Dr. Baraldi’s current research interests include
missingdataanalyses,methodsforassessingmediation,longitudinalgrowthmodelling,
and health and prevention research.
TomBoothisLecturerinQuantitativeResearchMethodsintheDepartmentofPsy-
chology,UniversityofEdinburgh.Hisprimarymethodologicalinterestsareingeneral-
ized latent variable modelling. His applied work covers individual differences,
organizational, and health psychology.
LiCaiisProfessorintheAdvancedQuantitativeMethodologyprogramintheUCLA
Graduate Schoolof Education and InformationStudies. He also servesas Director of
the National Center for Research on Evaluation, Standards, and Student Testing
(CRESST). In addition, he is affiliated with the UCLA Department of Psychology.
Hismethodologicalresearchagendainvolvesthedevelopmentoflatentvariablemodels
that have wide-ranging applications in educational, psychological, and health-related
domains of study.
EvaCeulemansisProfessorofQuantitativeDataAnalysisattheFacultyofPsychology
andEducationalSciences,UniversityofLeuven,Belgium.Herresearchfocusesonthe
developmentofnewtechniquesformodellingmultivariatetimeseriesdataandmulti-
groupdata,andexploringindividualorgroupdifferencestherein.Tothisend,sheoften
combinesgeneralprinciplesofclusteranalysiswithdimensionreduction(principalcom-
ponent analysis, factoranalysis) and/or regression.
FangFangChenreceivedherM.S.inpsychologyfromPekingUniversity,herdoctoral
traininginSocialandQuantitativePsychologyfromArizonaStateUniversity,andcom-
pletedherpost-doctoraltrainingattheUniversityofNorthCarolinaatChapelHill.Her
methodologicalworkfocuses onmeasurementinvarianceandthe bifactormodel. Dr.
ChenwasanassistantprofessorofpsychologyattheUniversityofDelaware,andnowis
a senior research biostatistician at the Nemours Center for the HealthCare Delivery
Science.
viii Notes on Contributors to Volume 1
DavidJ.CiukisanassistantprofessorintheDepartmentofGovernmentattheFrank-
lin & Marshall College. His academic interests center on public opinion and political
psychology.Hisresearchaimstobuildabetterunderstandingoftheattitudeformation
processinthemasspublic.Morespecifically,helooksathowmoralsandvalues,policy
information,andpoliticalidentityaffectpoliticalattitudes.Heisalsointerestedinsurvey
experimental designs, measurement, and various public health issues.
ChristineE.DeMarsservesatJamesMadisonUniversityasaprofessorinthedepart-
mentofgraduatepsychologyandaseniorassessmentspecialistintheCenterforAssess-
mentandResearchStudies.SheteachescoursesinItemResponseTheory,ClassicalTest
Theory,andGeneralizabilityTheory,andsupervisesPh.D.students.Herresearchinter-
estsincludeappliedandtheoreticaltopicsinitemresponsetheory,differentialitemfunc-
tioning, test-taking motivation, and other issues in operational testing.
ConorV.DolanisProfessorattheVUUniversity,Amsterdam.Hisresearchinterests
include: covariance structure modelling, mixture analyses, modelling of multivariate
intelligence test scores, and modelling genotype-environment interplay.
SusanEmbretsonisProfessorofPsychologyattheGeorgiaInstituteofTechnology.
She hasbeenrecognizednationallyandinternationally for herprogrammaticresearch
onintegratingcognitivetheoryintopsychometricitemresponsetheorymodelsandinto
thedesignofmeasurementtasks.ShehasreceivedawardsfromtheNationalCouncilon
Measurement and Education; American Educational Research Association; and the
American Psychological Association Division for research and theory on item genera-
tion from cognitive theory.
CraigK.EndersisaProfessorintheDepartmentofPsychologyatUCLAwhereheisa
member of the Quantitative program area. Professor Enders teaches graduate-level
courses in missing data analyses, multilevel modelling, and longitudinal modelling.
The majority of his research focuses on analytic issues related to missing dataanalyses
andmultilevelmodelling.Hisbook,AppliedMissingDataAnalysis,waspublishedwith
Guilford Press in2010.
DavidJ.HughesisanOrganisationalPsychologistatManchesterBusinessSchool.His
research interests centre on individual differences and can be broken down into three
mainareas:thetheoryandmeasurementofindividualdifferences,individualdifferences
atwork,andindividualdifferencesinfinancialbehavior.Heisinterestedinpsychomet-
rictestevaluationandhisstatisticalinterestsrevolvearoundgeneralizedlatentvariable
models, in particular structural equation modelling, factor models, and other models
appropriate for multivariate data with complex structures.
Paul Irwing is Professor of Psychometrics at the Manchester Business School. He
chairedthePsychometricsatWorkResearchGroupandisadirectorofthepsychometric
publishingcompanyE-metrixx.Hehasauthoredtworesearchandtwocommercialpsy-
chometricmeasures.Heisknownforresearchonsexdifferencesandpioneeringwork
onthegeneralfactorofpersonality.Hiscurrentresearchconcernsthe11+FactorModel
ofPersonality,andthenewlyproposedindividualdifferenceofPersonalityAdaptability.
William G. Jacoby is Professor of Political Science at Michigan State University and
Editor of the American Journal of Political Science. He is the former Director of the
Inter-university Consortium for Political and Social Research (ICPSR) Summer Pro-
gram in Quantitative Methods of Social Research and former Editor of the Journal
Notes on Contributors to Volume 1 ix
ofPolitics.ProfessorJacoby’sareasofprofessionalinterestincludemasspoliticalbehav-
ior and quantitative methodology (especially scaling methods, measurement theory,
and statistical graphics).
Robert I. Jennrich began his work with the development of the first internationally
used statistical software package BMD. His main field is statistical computing. He
contributed to the development of nearly half of the 25 programs in this package.
Because of this, he was a member of the 1972 Soviet-American Scientific Exchange
DelegationonComputing.Sincetheseearlydays,hehaspublishedpapersonstepwise
linearandnon-linearregression,stepwisediscriminantanalysis,goodness-of-fittesting
forcovariancestructureanalysis,andanumberofpapersinfactoranalysis,primarilyon
rotation. Hehas recently beennominated for alifetime achievement award,whichhe
unfortunately didn’tget.
KenKelleyisProfessorofInformationTechnology,Analytics,andOperations(ITAO)
andtheAssociateDeanforFacultyandResearchintheMendozaCollegeofBusinessat
theUniversityofNotreDame.ProfessorKelley’sworkisonquantitativemethodology,
wherehefocusesonthedevelopment,improvement,andevaluationofstatisticalmethods
andmeasurementissues.ProfessorKelley’sspecialtiesareintheareasofresearchdesign,
effect size estimation and confidence interval formation, longitudinal data analysis, and
statisticalcomputing.Inadditiontohismethodologicalwork,ProfessorKelleycollabo-
rateswithcolleaguesonavarietyofimportanttopicsapplyingmethods.ProfessorKelleyis
anAccreditedProfessionalStatistician™(PStat®)bytheAmericanStatisticalAssociation,
associateeditorofPsychologicalMethods,andrecipientoftheAnneAnastasiearlycareer
award by the American Psychological Association’s Division of Evaluation, Measure-
ment,&Statistics,andafellowoftheAmericanPsychologicalAssociation.
KekeLaiisAssistantProfessorofQuantitativePsychologyatUniversityofCalifornia,
Merced. His research interests include structural equation modelling and multilevel
modelling.
StanleyLemeshowearnedhisPh.D.atUCLA,andhisMSPHatUNC.Hehascoau-
thored three textbooks: Applied Logistic Regression; Applied Survival Analysis; and
Sampling of Populations – Methods and Applications. His honors include: the Wiley
Lifetime Award (2003); UCLA School of Public Health Alumni Hall of Fame
(2006); Fellow of the AAAS (2003); Distinguished Graduate Alumnus (Biostatistics)
– UNC Graduate School Centennial (2003); Fellow of the ASA (1995); and the Sta-
tistics Section Award of the APHA (1995).
UrbanoLorenzo-SevaisaprofessorintheDepartmentofPsychologyatUniversitat
RoviraiVirgili,Spain.HeisthecoauthorofFACTOR,afree-sharedsoftwaretocom-
puteexploratoryfactoranalysis,andhaspublishednumerousarticlesrelatedtothissub-
ject. His research interests include the development of new methods for exploratory
dataanalysis,andappliedpsychometricresearch.Hehastaughtdataanalysisatuniver-
sity level and inshort courses for many years.
BoLuearnedhisPh.D.inStatisticsfromtheUniversityofPennsylvania.Heisanasso-
ciate professor of Biostatistics at the Ohio State University. His research expertise
includescausalinferencewithpropensityscorebasedadjustmentforobservationaldata,
surveysamplingdesignandanalysis,statisticalmodelsformissingdata.HehasbeenPIs
for multiple NIH and AHRQ grants and served as the lead statistician for the Ohio
MedicaidAssessment Survey series since 2008.
x Notes on Contributors to Volume 1
Rob R. Meijer is Professor in Psychometrics and Statistics at the University of Gro-
ningen,theNetherlands.Hisworkisinitemresponsetheoryandapplicationsoftesting.
Heisalsointerestedineducationalassessmentandeducationalandpersonnelselection.
DylanMolenaarisAssistantProfessorattheDepartmentofPsychology,Universityof
Amsterdam. His research interests include: item response theory, factor analysis, and
response time modelling.
KristinM.Morrison isanadvanced Ph.D.studentattheGeorgiaInstituteofTech-
nology. She has published and presented work on item generation at various confer-
ences and meetings. Other research interests include computer adaptive testing,
multistage testing, educational research, and cognitive complexity.
IriniMoustakiisProfessorofSocialStatisticsattheLondonSchoolofEconomicsand
Political Science. Her research interests are in the areas of latent variable models and
structural equation models. Her methodological work includes treatment of missing
data, longitudinal data, detection of outliers, goodness-of-fit tests, and advanced esti-
mationmethods.Furthermore,shehasmademethodologicalandappliedcontributions
in the areas of comparative cross-national studies and epidemiological studies on rare
diseases.Shehascoauthoredtwobooksonlatentvariablemodels.ShewaselectedEdi-
tor inChief of the journal Psychometrikain November 2014.
StanleyA.MulaikisEmeritusProfessorintheSchoolofPsychology,GeorgiaInstitute
ofTechnology.Hisworkhasbroadlyfocussedonlatentvariablemodelsandtheunder-
lying philosophy of causality. He is the author of the influential texts, Foundations of
Factor Analysisand LinearCausal Modeling with Structural Equations.
Aja L. Murray is a Research Associate in the Violence Research Centre, Institute of
Criminology, University of Cambridge. Her research interests include psychometrics
and childhood and adolescent mental health developmental.
JorgeN.TendeiroisAssistantProfessorattheUniversityofGroningen,theNether-
lands.Hehasresearchinterestswithinitemresponsetheory(IRT),withalargefocuson
person-fit analyses. Besides (co)authoring several papers on this topic, he is also the
author of the PerFit R package. He is currently extending person-fit approaches to a
broader type of IRT, which include the unfolding model.
Marieke E. Timmerman is a professor in multivariate data analysis at the Heymans
InstituteforPsychologicalResearchattheUniversityofGroningen,TheNetherlands.
Herresearchfocusesonthedevelopmentofmodelsformultivariatedatawithcomplex
structures, to achieve an understanding of the processes underlying these, mainly
psychological,data.Herresearchinterestsarelatentvariablemodelling,datareduction
methods,includingmultisetmodels,andclassification.Sheservesasanassociateeditor
of Psychometrika.
ZuguiZhangobtainedhisPh.D.inbiostatisticsfromtheUniversityofIowa.Heisthe
lead Biostatistician of the Value Institute of Christiana Care Health System, Assistant
ProfessoratThomasJeffersonUniversity,andJointProfessorattheUniversityofDel-
aware.Hehasabroadbackgroundinbiostatistics,healthoutcomesresearch,qualityof
life,publichealth,epidemiology,andhealtheconomics.Hehasworkedextensivelyon
large international and national, multicenter, randomized clinical trials, and observa-
tional studies.