Validity and Validation

217 Pages·2013·1.298 MB·English

by  Catherine S. Taylor

validity and
validation
SERIES IN UNDERSTANDING STATISTICS
s. natasha beretvas Series Editor
SERIES IN UNDERSTANDING MEASUREMENT
s. natasha beretvas Series Editor
SERIES IN UNDERSTANDING QUALITATIVE RESEARCH
patricia leavy Series Editor
Understanding Statistics Fundamentals of Qualitative Research
Johnny Saldaña
Exploratory Factor Analysis
Leandre R. Fabrigar and Duane
Th e Internet
T. Wegener
Christine Hine
Validity and Validation
Duoethnography
Catherine S. Taylor
Richard D. Sawyer and Joe Norris
Understanding Measurement
Qualitative Interviewing  
Svend Brinkmann
Item Response Th eory
Christine DeMars
Reliability
Patrick Meyer
Understanding Qualitative
Research
Oral History
Patricia Leavy
catherine s. taylor
VA L I D I T Y A N D
VA L I DAT I O N

1
3
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide.
Oxford New York
Auckland Cape Town Dar es Salaam Hong Kong Karachi
Kuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto
With offi ces in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Th ailand Turkey Ukraine Vietnam

Oxford is a registered trademark of Oxford University Press
in the UK and certain other countries.
Published in the United States of America by
Oxford University Press
198 Madison Avenue, New York, NY 10016
© Oxford University Press 2013
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system, or transmitted, in any form or by any means, without the prior
permission in writing of Oxford University Press, or as expressly permitted by law,
by license, or under terms agreed with the appropriate reproduction rights organization.
Inquiries concerning reproduction outside the scope of the above should be sent to the
Rights Department, Oxford University Press, at the address above.
You must not circulate this work in any other form
and you must impose this same condition on any acquirer
Library of Congress Cataloging-in-Publication Data
Taylor, Catherine S.
Validity and validation / Catherine S. Taylor.
pages cm. — (Understanding statistics)
ISBN 978–0–19–979104–0
1. Social sciences—Statistical methods. 2. Social sciences—Methodology.
3. Research methods. I. Title.
HA29.T3237 2013
001.4′22—dc23 2013008389
9 8 7 6 5 4 3 2 1
Printed in the United States of America
on acid-free paper
For Laurie, Robin, and Courtney
Th ank you.
This page intentionally left blank
contents
Acknowledgments. . . . . . . . . . . . . . . . . . viii
CHAPTER 1 Validity and Validation in Research and Assessment . . . . . . . . . . 1
CHAPTER 2 Evidence for the Internal Validity of Research Results . . . . . . . . . . 2 4
CHAPTER 3 External Threats to Validity . . . . . . . . . . . . . . . . . 5 5
CHAPTER 4 Validity of Statistical Conclusions . . . . . . . . . . . . . . . . 6 5
CHAPTER 5 Construct-Related Evidence for Validity . . . . . . . . . . . . . . 8 2
CHAPTER 6 Interpretation, Use, and Consequences of Scores from Assessments . . . . . . 1 47
CHAPTER 7 Validity Theory and Validation Resources . . . . . . . . . . . . . 1 89
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
acknowledgments
I would like to thank Robert Abbott for his willingness to think
with me about how to use correlational models to control for
threats to validity and Natasha Beretvas for her careful review
of the chapters for their usefulness and accuracy. Th anks also
to the Washington State Offi ce of the Superintendent of Public
Instruction for the use of state assessment data to generate exam-
ples. I am grateful to the individuals who have thought deeply
about validity and the potential pitfalls that occur when its
meaning is reduced to a handful of correlations. Th anks to Lee
Cronbach, Michael Kane, Samuel Messick, Robert Linn, Robert
Mislevy, Pamela Moss, and Lorrie Shepard for their elegant and
thoughtful papers on assessment validity and validation. Th eir
work has had a profound impact on my thinking about validity,
my assessment development work, and the focuses of my research.
Th ey have deepened my understanding of validity and the eth-
ics of assessment as well as the critical importance of assessment
purpose in determining the range of research needed to validate
assessment score interpretation and use.
validity and
validation