The Handbook of Research Synthesis and Meta-Analysis PDF
Preview The Handbook of Research Synthesis and Meta-Analysis
HANDBOOK OF RESEARCH SYNTHESIS AND META-ANALYSIS 2ND EDITION HANDBOOK THE RESEARCH OF SYNTHESIS META-ANALYSIS AND 2ND EDITION Edited by HARRIS COOPER, LARRY V. HEDGES, AND JEFFREY C. VALENTINE RUSSELL SAGE FOUNDATION NEW YORK THE RUSSELL SAGE FOUNDATION The Russell Sage Foundation, one of the oldest of America’s general purpose foundations, was established in 1907 by Mrs. Margaret Olivia Sage for “the improvement of social and living con- ditions in the United States.” The Foundation seeks to fulfi ll this mandate by fostering the devel- opment and dissemination of knowledge about the country’s political, social, and economic prob- lems. While the Foundation endeavors to assure the accuracy and objectivity of each book it publishes, the conclusions and interpretations in Russell Sage Foundation publications are those of the authors and not of the Foundation, its Trustees, or its staff. Publication by Russell Sage, therefore, does not imply Foundation endorsement. BOARD OF TRUSTEES Mary C. Waters, Chair Kenneth D. Brody Kathleen Hall Jamieson Nancy Rosenblum W. Bowman Cutter, III Melvin J. Konner Shelley E. Taylor Christopher Edley Jr. Alan B. Krueger Richard H. Thaler John A. Ferejohn Cora B. Marrett Eric Wanner Larry V. Hedges Library of Congress Cataloging-in-Publication Data The handbook of research synthesis and meta-analysis / Harris Cooper, Larry V. Hedges, Jeffrey C. Valentine, editors. — 2nd ed. p. cm. Rev. ed. of: The handbook of research synthesis. c1994. Includes bibliographical references and index. ISBN 978-0-87154-163-5 1. Research—Methodology—Handbooks, manuals, etc. 2. Information storage and retrieval systems—Research—Handbooks, manuals, etc. 3. Research—Statistical methods— Handbooks, manuals, etc. I. Cooper, Harris M. II. Hedges, Larry V. III. Valentine, Jeff C. IV. Handbook of research synthesis. Q180.55.M4H35 2009 001.4(cid:1)2—dc22 2008046477 Copyright © 2009 by Russell Sage Foundation. All rights reserved. Printed in the United States of America. No part of this publication may be reproduced, stored in a retrieval system, or transmit- ted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Reproduction by the United States Government in whole or in part is permitted for any purpose. The paper used in this publication meets the minimum requirements of American National Stan- dard for Information Sciences—Permanence of Paper for Printed Library Materials. ANSI Z39.48-1992. RUSSELL SAGE FOUNDATION 112 East 64th Street, New York, New York 10065 10 9 8 7 6 5 4 3 2 1 To the authors of the Handbook chapters, for their remarkable cooperation and commitment to the advance of science and society CONTENTS Foreword xi About the Authors xv PART I INTRODUCTION 1 1. Research Synthesis as a Scientifi c Process 3 Harris Cooper and Larry V. Hedges PART II FORMULATING A PROBLEM 17 2. Hypotheses and Problems in Research Synthesis 19 Harris Cooper 3. Statistical Considerations 37 Larry V. Hedges PART III SEARCHING THE LITERATURE 49 4. Scientifi c Communication and Literature Retrieval 51 Howard D. White 5. Using Reference Databases 73 Jeffrey G. Reed and Pam M. Baxter 6. Grey Literature 103 Hannah R. Rothstein and Sally Hopewell PART IV CODING THE LITERATURE 127 7. Judging the Quality of Primary Research 129 Jeffrey C. Valentine vii viii CONTENTS 8. Identifying Interesting Variables and Analysis Opportunities 147 Mark W. Lipsey 9. Systematic Coding 159 David B. Wilson 10. Evaluating Coding Decisions 177 Robert G. Orwin and Jack L. Vevea PART V STATISTICALLY DESCRIBING STUDY OUTCOMES 205 11. Vote-Counting Procedures in Meta-Analysis 207 Brad J. Bushman and Morgan C. Wang 12. Effect Sizes for Continuous Data 221 Michael Borenstein 13. Effect Sizes for Dichotomous Data 237 Joseph L. Fleiss and Jesse A. Berlin PART VI STATISTICALLY COMBINING EFFECT SIZES 255 14. Combining Estimates of Effect Size 257 William R. Shadish and C. Keith Haddock 15. Analyzing Effect Sizes: Fixed-Effects Models 279 Spyros Konstantopoulos and Larry V. Hedges 16. Analyzing Effect Sizes: Random-Effects Models 295 Stephen W. Raudenbush 17. Correcting for the Distorting Effects of Study Artifacts in Meta-Analysis 317 Frank L. Schmidt, Huy Le, and In-Sue Oh PART VII SPECIAL STATISTICAL ISSUES AND PROBLEMS 335 18. Effect Sizes in Nested Designs 337 Larry V. Hedges 19. Stochastically Dependent Effect Sizes 357 Leon J. Gleser and Ingram Olkin 20. Model-Based Meta-Analysis 377 Betsy Jane Becker PART VIII DATA INTERPRETATION 397 21. Handling Missing Data 399 Therese D. Pigott 22. Sensitivity Analysis and Diagnostics 417 Joel B. Greenhouse and Satish Iyengar 23. Publication Bias 435 Alex J. Sutton CONTENTS iX PART IX TYING RESEARCH SYNTHESIS TO SUBSTANTIVE ISSUES 453 24. Advantages of Certainty and Uncertainty 455 Wendy Wood and Alice H. Eagly 25. Research Synthesis and Public Policy 473 David S. Cordray and Paul Morphy PART X REPORTING THE RESULTS 495 26. Visual and Narrative Interpretation 497 Geoffrey D. Borman and Jeffrey A. Grigg 27. Reporting Format 521 Mike Clarke PART XI SUMMARY 535 28. Threats to the Validity of Generalized Inferences 537 Georg E. Matt and Thomas D. Cook 29. Potentials and Limitations 561 Harris Cooper and Larry V. Hedges Glossary 573 Appendix A 585 Appendix B 591 Index 595
The list of books you might like
