Table Of Content

BAYESIANANALYSISOFITEMRESPONSEMODELSFORBINARYDATA By ATALANTAGHOSH ADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF DOCTOROFPHILOSOPHY UNIVERSITYOFFLORIDA 1996 Tomyparentsandteachers ACKNOWLEDGEMENTS IwouldliketoexpressmysinceregratitudetoProfessorsMalayGhoshandAlan Agrestiforbeingmyadvisors. Withouttheirenormouspatience,encouragementand guidance, itwouldnothavebeenpossibletocompletethework. Iconsidermyself extremelyluckytogetthemasmydissertationadvisors. Iwouldliketothankmy friendsfortheirgreathelp,especiallyduringthefinalstageofthisdissertation. Also IwouldliketowholeheartedlythankmywifeSofiaPaulforherinvaluablesupport andhelpallthroughoutthework. iii TABLEOFCONTENTS ACKNOWLEDGEMENTS iii ABSTRACT vi CHAPTERS 1 INTRODUCTION 1 1.1 LiteratureReview 1 1.2 TopicofthisDissertation 6 2 AUNIFIEDBAYESIANANALYSISOFITEMRESPONSEMODELSFOR BINARYDATA 8 2.1 Introduction 8 2.2 ChoiceOfPriors 9 2.3 ImplementationOfBayesProcedures 23 2.4 AnalysisBasedonDataForPlacementTests 26 2.5 ExampleonMatchedPairsData 31 2.5.1 EffectofDiagonalElements 39 2.6 Uniformapproximationofimproperpriorsbyproperpriors 44 2.7 ConsistencyofMarginalMaximumLikelihoodEstimator 57 3 HIERARCHICALBAYESIANANALYSISOFITEMRESPONSEMODELS FORBINARYDATA 75 3.1 Introduction 75 3.2 BayesProcedureswithHierarchicalPriors 76 3.3 ImplementationOfBayesProcedures 81 3.4 AnExample 82 4 AUNIFORMBAYESIANANALYSISOFTWO-PARAMETERITEMRE- SPONSEMODELSFORBINARYDATA 86 4.1 Introduction 86 iv 4.2 ChoiceOfPriors 87 4.3 ImplementationOfBayesProcedures 94 4.4 AnExample 96 5 SUMMARYANDFUTURERESEARCH 101 BIBLIOGRAPHY 104 BIOGRAPHICALSKETCH 108 v AbstractofDissertationPresentedtotheGraduateSchool oftheUniversityofFloridainPartialFulfillment oftheRequirementsfortheDegreeof DoctorofPhilosophy BAYESIANANALYSISOFITEMRESPONSEMODELSFORBINARYDATA By AtalantaGhosh December1996 Chairpersons: AlanAgresti MalayGhosh MajorDepartment: Statistics WepresentaunifiedBayesianapproachfortheanalysisofone-andtwo-parameter itemresponsemodels,withspecialemphasisonlogit,probit,andlog-loglinks. Nec- essaryandsufficientconditionsarefoundfortheproprietyoftheposteriorsunder improperpriorsforthesemodels. Bayesestimation is implemented usingMarkov Chain MonteCarlointegrationtechnique, andisillustratedwithanexamplefrom educational testing. Somerelationships between thefrequentist and the Bayesian procedurearealsodiscussed. vi CHAPTER 1 INTRODUCTION 1.1 LiteratureReview Itemresponsemodelsarewidelyusedfortheanalysisofpsychometricdata. Their origin can betraced back tomid-thirties andearly forties (cf. Richardson, 1936; Tucker, 1946). Asystematicdevelopmentofitemresponsetheoryfromtheclassical pointofviewowesmuchtothepioneeringworkofLord(1952,1953a,1953b),Rasch (1960,1961)andtheirassociates. Amongthemanynoteworthyrecentcontributions inthesamevein, wemayciteAndersen (1970, 1972, 1973), Bockand Lieberman (1970),MislevyandBock(1984),SwaminathanandGifford(1981),andHambleton andSwaminathan(1985). Tomotivateitemresponsemodels,considerasaspecificexample,abilitytestsor attitudetestswhereeachindividualanswersabatteryofquestions. LetX{jdenote theresponseoftheithindividualtothejthquestion (i= 1,...,n; j= 1,...,k). Associatedwiththezthindividual(orsubject)isasubjectparameter^thatexpresses thecapacity,abilityortheattitudeofthatindividualinagivencontext. However, thedistributionsofthe willdependnotonlyonQ{butalsoonsomeparameteraj where-aijrepresentsthedifficultylevelofquestionj. Itemresponseanalysismodels thedistributionofXtjtakingintoaccountbothQ{anda,(i=1,...,n;j=1,...,Jb). Forsimplicity,throughoutthisdissertationweconsideronlythecasewhentheXij arebinaryrandomvariables. Thisis,forexample,thesituationwhennexamineesare 1 2 answering "True/False" questions. Alternately,theexamineesmayanswermultiple choicequestionswhereeachansweriscodedas "correct" or "incorrect". Letpjj = P{Xij = 1) (i = 1,...,n ; j = 1,...,k), where 1 denotes a correct response. Itemresponsetheoryforbinaryresponseisbasedonmodelswhichexpresstheptjas functionsofcertainparameters. Thisdissertationconcentratesonone-andtwo-parameteritemresponsemodels. Aone-parameteritemresponsemodelinitsmostgeneralformisgivenby pij=F{6i+ai), (1.1.1) where F is a distribution function. This is referred to as a one-parameter item responsemodel,sinceasafunctionof9{therighthandsideof(1.1.1)hastheformof adistributionfunctionwithlocationparameter-cty ThefunctionF"1(orsometimes Fitself)iscalledalinkfunction. Thethreemostcommonlyusedlinkfunctionsare thelogit,probitandlog-loglinks. Logitandprobitlinksaresymmetric, whilethe log-loglinkisasymmetric. Alogitlinkisbasedonthelogisticdistributionfunction, namely The resulting one-parameter item response model is the celebrated Rasch model (Rasch, 1960, 1961). A probit link is based on thenormal distribution function. Alog-loglinkisbasedontheextremevaluedistributionfunction. Intheitemresponseliterature,the0,arereferredtoas "subjectability" parameters, whiletheaj arereferredtoas "item" parameters. Despitethesimplicityof one-parameteritemresponsemodelsandinparticular,theoverwhelmingpopularity oftheRaschmodel (seeforexamplethecollectionofessays inFischerand Mole- naar,1995),thesemodelsareoftencriticizedonthegroundthattheyassumeallthe questionstohaveequalpowertodiscriminatebetween "good"and"poor"students, andthereforedonotinvolveanydiscriminationparametersinadditiontotheitem 3 parameters. Thisproblemisalleviatedbyintroducingtwo-parameteritemresponse modelsgivenby Pij=F(>yj9i+bj) (1.1.2) orequivalentlyby Pv=FhAh+ (bi=W)- (1-1-3) Thejjarereferredtoas"discrimination"parametersrangingbetween(0,oo)foreach j. Modelsofthistypearereferredtoastwo-parameteritemresponsemodels,since asafunctionof0j, hastheformofadistributionfunctionwithlocationparameter —a.jandscaleparameterl/7j. Onemaybeinterestedininferenceforthe0jorfortheojjorsimultaneouslyfor boththe6i andthea, inone-parameteritemresponsemodels. Similarlyfortwo- parameteritemresponsemodels,inferencemayinvolveoneormoreofthethreesets ofparameters {aj} and {7,-}. Ourdissertationwillprimarilyconcentrateon inferenceforthe {aj} inone-parametermodels andfor {7,,a,} intwo-parameter models. Weshallnowbrieflyreviewsomeoftheexistingestimationproceduresforone- andtwo-parameteritemresponsemodels. Classicalestimationforthe{a,}or{a,-,7^} usesmaximumlikelihoodestimationmethod. Amajordifficultythatarisesinthis contextisthatthemaximumlikelihoodestimators(MLE)oftheitemparametersare inconsistent. ThisisanexampleofthewellknownNeyman-Scottphenomenon,first notedbyNeymanandScott (1948). Theyprovedinthebalancedone-waynormal ANOVAmodelthatasthenumberofcellmeansgrowstoinfinity,theMLEofthe errorvarianceisinconsistent. Andersen (1970, 1972, 1973), inaseriesofarticles, hasdiscussedveryextensivelyfrequentistinferencefortheRaschmodel. Oneofhis findingsisthatwhenk=2,butn-»00,theMLEofax (ora2)isaninconsistent estimator. AproofoftheinconsistencyoftheMLEforgeneralkisrecentlygivenin Ghosh(1995). , 4 Andersen(1970)recommendsavoidingtheproblemofinconsistentMLE'sforthe RaschmodelbyfindingtheMLE'softheaj (j=1,...,k)conditionalonsufficient statisticsforthenuisanceparameters9i(i=1,...,n). Thisapproach,however,does notworkwhenseparatesufficientstatisticsdonotexistforthenuisanceparameters. Thisis, forinstance, evidencedintheprobitmodel,pi3 = +aj), <3>beingthe standardnormaldistributionfunction,wheresufficientstatisticsdonotexistforthe nuisanceparameters ...,#„). Infact,theonlyitemresponsemodelthatadmits nontrivialsufficient statisticsforthe6{ isthe Rasch model. Bock and Lieberman (1970) have advocated instead theuseofmarginalmaximum likelihood estimates ofthe aj (j = l,...,k) byassigningsome distribution to d\,...,Qn. Then inte- gratingoutwith respect to0\,...,9n, onefinds thejoint marginaldistribution of Xij(i=1,...,n; j=1,... k)whichinvolveonlya^,...,a*andtheparametersof thedistributionsofthe#j. Basedonthisdistribution, onefindstheMLE'softhe <*!,...,a*;, usuallyreferredtoasthemarginalMLE's. Similarlyfortwo-parameter modelsonecanemploymarginalmaximumlikelihoodestimationtechniqueforthe item parameters as well asdiscrimination parameters. In thiscase one finds the marginaldistributionofXij (i—1,...,n; j=1,...,k)involvingtheitemparame- tersax,...,afcand7i,..•,7jt- BockandAitkin(1981)proposedanEMalgorithmto obtainmarginalmaximumlikelihoodestimatesfortheseparametersinthecontext oftwo-parametermodels. Analternativeandattractiveestimationprocedureforitemresponsemodelsis theBayesianprocedure. Overtheyears,thisapproachhasreceivedconsiderableat- tention. However,eventheBayesianapproachadmitsvariationinactualuse. First, in aregularBayesian approach, inferenceisbasedonacompletely specified prior distributionforalltheunknownparameters. However, unlessoneusesfullynonin- formativepriors,suchprocedureswilllackrobustnessagainstmisspecifiedpriors. As analternative, onemayproceedbyestimatingsomeorallofthepriorparameters