Statistical Theories of Mental Test Scores

593 Pages·2008·38.049 MB·English

by  Frederic M. Lord, Melvin R. Novick

STATISTICAL THEORIES
OF MENTAL TEST SCORES
The Addison-Wesley Series in
BEHAVIORAL SCIENCE: QUANTITATIVE METHODS
FREDERICK MOSTELLER, Consulting Editor
This series has been undertaken for the publication of works that combine mathe
matical, statistical, and computational techniques with substantive material. The
series will help to integrate these theoretical techniques with disciplines that are con
cerned with actual problems in the real world. The approach of each book is therefore
interdisciplinary, and heavy emphasis falls on the handling of data, on the application
of mathematical and statistical devices to concrete issues, and on the development of
"cross-departmental" subject matter. Many kinds and many levels of mathematics
are used and the range of subjects treated is broad. This wide coverage will benefit
all who apply quantitative methods in the area of behavioral science.
FREDERIC M. LORD and MELVIN R. NOVICK
Educational Testing Service
STATISTICAL THEORIES
OF MENTAL TEST SCORES
with contributions by
ALLAN BIRNBAUM, New York University
ISBN: 978-1-59311-934-8
Copyright © 2008 lAP - Information Age Publishing Inc.
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system, or transmitted in any form or by any electronic or mechanical means,
or by photocopying, microfilming, recording or otherwise without written permission
of the publisher.
Printed in the United States of America
The writing of this book, together with some of the research reported herein, was supported in part by the Logistics
and Mathematical Statistics Branch of the Office of Naval Research under contract Nonr-4866(OO), NR 042-249,
and by the Personnel and Training Branch of the Office of Naval Research under contract Nonr-2752(OO),
NR 151-201. Reproduction, translation, publication, use, and disposal in whole or in part, by or for the United
States Government is permitted.
PREFACE
More than fifteen years have passed since the publication of Harold Gulliksen's
Theory of Mental Tests. During that period mental test theory has been develop
ing at an accelerating pace. As a result it has been clear for some time that a
new synthesis of the field would be desirable. Professor Gulliksen has recognized
this need for many years, and it is partly as a result of his suggestion and
encouragement that this book has been written. These years have also seen an
increased dependence of the theory on relevant mathematical statistical models,
and hence it was apparent to us that any up-to-date, comprehensive treatment
of test theory must be based on the statement and development of these models.
Our initial efforts in seeking a new synthesis of the field suggested that a
very careful mathematical statement of the relevant models would provide
additional insight into many well-studied problems. In preparing this book,
consequently, we have tried to exercise more care in this direction than has
previously been attempted. We have also tried, however, to avoid technical
mathematical discussions that do not seem to contribute to an understanding
of test theory. As a result we have been able to keep parts of the book at a
more moderate mathematical level than had at one point seemed possible.
A major part of our task has been to restate and refine the work of many
people into one integrated development. Our book, however, represents the
theoretical orientation and interests of its authors. The relative amounts of
space assigned to various topics is due, in part, to this. Published research that
is central to our purpose has been integrated into our treatment, and thus we
have encompassed much of the test theory literature. Some heretofore un
published research of our own has also been included. Published research that
is relevant but not central to our approach has usually been referred to but
not discussed. Some basic papers and many special ones have not been referred
to here, either because they do not fit into our approach or because their subject
matter is beyond the scope of our undertaking. Thus the inclusion or exclusion
of reference to any particular paper should not be interpreted as a comment on
the merit of that paper.
During the planning of this book, we recognized that one important piece of
research, which at that time was still unpublished, would need to be covered in
any comprehensive treatment of mental test theory. This research was Allan
Birnbaum's work in latent trait theory, including his logistic response model.
v
Vi PREFACE
We have been fortunate indeed that Professor Birnbaum has agreed to publish
this material for the first time in our book. Professor Birnbaum has had a very
free hand in developing his contribution in his own way. However, we have
worked closely with him in an endeavor to integrate his material into the general
stream of development of our book. The remaining chapters of the book were
written by the undersigned, and the responsibility for these chapters, the selec
tion and integration of the work of the contributor, the outline, format, and
point of view of the book are jointly ours alone.
Many of the ideas presented here were nurtured in the ongoing Seminar in
Mathematical Psychology of the Psychometric Research Group of Educational
Testing Service. Members of this Seminar at various times have been Allan
Birnbaum, Michael W. Browne, Karl Joreskog, Walter Kristof, Michael Levine,
Frederic M. Lord, William Meredith, Samuel Messick, Roderick McDonald,
Melvin R. Novick, Fumiko Samejima, J. Philip Sutcliffe, and Joseph L. Zinnes.
Much of our discussion of the relationship between statistical models and the
methodological and theoretical problems of psychology has benefited from
conversations with Norman Frederiksen, Nathan Kogan, and Samuel Messick.
Part III has benefited from a careful reading by Julian Stanley. We are in
debted to Professor Louis Guttman for making available to us a copy of his
unpublished 1953 manuscript, The Concepts and Algebra of Reliability and
Validity: A Critique. The manuscript as a whole has been reviewed by Frederick
Mosteller and Robert L. Thorndike, from whose suggestions we have profited
immensely and to whom we extend our gratitude, but whom we saddle with no
responsibility for any errors that may remain.
Preliminary versions of a number of chapters of this book have been used
during the past two years at the following universities as a text for courses and
seminars in test theory and for related courses.
University Instructors
R. Darrell Bock
University of Chicago David Wiley
Benjamin Wright
Harvard University John B. Carroll
University College, London Melvin R. Novick
University of North Carolina Murray Aitkin
Ontario Institute for
Ross E. Traub
Studies in Education
University of Pennsylvania Melvin R. Novick
Frederic M. Lord
Princeton University
Melvin R. Novick
Stanford University Lee J. Cronbach
University of Tennessee Edward Cureton
PREFACE vii
Comments received from those who have used this material, both instructors
and students, have been most helpful in preparing later drafts.
Mrs. Dorothy Thayer played a substantial role in checking the manuscript
for errors and ambiguities and has had a major responsibility for developing or
checking most of the exercises and for preparing many of the tables and charts.
Mr. Charles Lewis offered many penetrating comments which have impelled us
to sharpen our arguments in many places. Messrs. Carl Frederiksen, Jon
Kettenring, Philip Piserchia, and Larry G. Richards, working as summer
research assistants, have helped us with early chapter drafts by pointing out
errors and ambiguities. The very tedious job of typing the successive drafts of
this manuscript has been expertly performed by Mrs. Beatrice Stricklin and
Miss Kathleen Rohe, with occasional assistance from Mrs. Mary Evelyn Runyon
and Mrs. Eleanor Hibbs. Mrs. Fay F. Richardson assisted with the galley
proofs. Mrs. Ann King monitored the final page proofs and provided invaluable
editorial assistance during the entire period of this project. Mr. Michael
Friendly has carefully checked a number of chapters for typographical errors.
Mrs. Sara B. Matlack arranged for and monitored all of the extensive support
facilities required for the completion of this project.
We would like to thank all the people at Addison-Wesley who have given us
so much fine technical assistance in the completion of this project. Their un
failing sympathy with our many wishes and their constant efforts to comply
with these wishes whenever possible, has earned our very deep appreciation.
We are indebted to the Office of Naval Research for extensive support of
some of the research reported here. This test theory research has been partly
supported by the Personnel and Training Branch since 1952. The preparation
of this book was financed, in part, by support received over a three-year period
from the Logistics and Mathematical Statistics Branch.
A very large measure of support for this project was provided by Educa
tional Testing Service, to whom we are greatly indebted for making this book
possible. In particular, we wish to express our gratitude to William W. Turnbull,
Executive Vice-President, and Norman Frederiksen, Director of the Division
of Psychological Studies, for creating and maintaining an environment in the
Division of Psychological Studies in which basic research can flourish.
Princeton, New Jersey Frederic M. Lord
November 1967 Melvin R. Novick

CONTENTS
PART 1
The Foundations of Mental Measurement Theory
Chapter 1
Measurement in Psychology and Education
1.1 The need for a theory of mental testing 13
1.2 Psychological theory and its function . 15
1.3 Measurement as a basis of model construction 16
1.4 The place of measurement in psychology 19
1.5 Levels of measurement 20
1.6 The specification of interval scales. 21
1.7 Deterministic and probabilistic models 23
1.8 The assumptions underlying test theory models 24
Chapter 2
The Construction of True and Error Scores
2.1 Introduction 27
2.2 The distribution of measurements on a fixed person 29
2.3 True score as an expectation 30
2.4 The construction of the error random variable for a fixed person 31
2.5 The random selection of persons 32
2.6 Construction of the linear model 34
2.7 Derivation of the usual assumptions of the classical model. 36
2.8 What is error? 38
2.9 The many concepts of true score 39
2.10 Experimental independence. 44
2.11 Linear experimental independence. 45
2.12 Replicate measurements 46
2.13 Parallel measurements and parallel forms 47
ix