Methodological Issues and Strategies in Clinical Research

723 Pages·2015·19.646 MB·English

by  Dr. Alan E. Kazdin PhD (editor)

Method 4th Title_Method 4th Title_Paper_8.5x11 10/19/15 12:39 PM Page 2
ethodological
M
I
ssues
&
trategies
S
in linical
C
esearch
R
Fourth Edition
Edited by
Alan E. Kazdin
American Psychological Association
Washington, DC
Method 4th Title_Method 4th Title_Paper_8.5x11 10/19/15 12:39 PM Page 1
ethodological
M
I
ssues
&
trategies
S
in linical
C
esearch
R
Fourth Edition

Copyright © 2016 by the American Psychological Association. All rights reserved. Except as permitted
under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed
in any form or by any means, including, but not limited to, the process of scanning and digitization, or
stored in a database or retrieval system, without the prior written permission of the publisher.
Chapter 31 was written by employees of the National Institutes of Health and is in the public domain.
Published by
American Psychological Association
750 First Street, NE
Washington, DC 20002
www.apa.org
To order
APA Order Department
P.O. Box 92984
Washington, DC 20090-2984
Tel: [phone]; Direct: [phone]
Fax: [phone]; TDD/TTY: [phone]
Online: www.apa.org/pubs/books
E-mail: [email protected]
In the U.K., Europe, Africa, and the Middle East, copies may be ordered from
American Psychological Association
3 Henrietta Street
Covent Garden, London
WC2E 8LU England
Typeset in Berkeley by Circle Graphics, Inc., Columbia, MD
Printer: Sheridan Books, Inc., Ann Arbor, MI
Cover Designer: Minker Design, Sarasota, FL
The opinions and statements published are the responsibility of the authors, and such opinions and
statements do not necessarily represent the policies of the American Psychological Association.
Library of Congress Cataloging-in-Publication Data
Names: Kazdin, Alan E.
Title: Methodological issues and strategies in clinical research / [edited by] Alan E. Kazdin.
Description: 4th edition. | Washington, DC : American Psychological Association, [2016] | Includes
bibliographical references and index.
Identifiers: LCCN 2015017881| ISBN 9781433820915 (hc : alk. paper) | ISBN 978-1-4338-2092-2
(pbk. alk paper) | ISBN 1433820919 (hc : alk. paper) | ISBN 1433820927 pbk. alk. paper
Subjects: LCSH: Clinical psychology—Research—Methodology.
Classification: LCC RC467.8 .M48 2016 | DDC 616.890072—dc23
LC record available at http://lccn.loc.gov/2015017881
British Library Cataloguing-in-Publication Data
A CIP record is available from the British Library.
Printed in the United States of America
Fourth Edition
http://dx.doi.org/10.1037/14805-000
Contents
Contributors ............................................................ xi
Preface ................................................................. xv
Part I. Introduction: Overview and Background ............................. 1
Chapter 1. Methodology: What It Is and Why It Is so Important .................... 3
Alan E. Kazdin
Part II. Beginning the Research Process .................................... 23
Research Ideas ..................................................... 27
Chapter 2. Beginning the Research Process: Key Concepts That Can Guide
a Study ........................................................ 29
Alan E. Kazdin
Chapter 3. Getting Out of Our Conceptual Ruts: Strategies for Expanding
Conceptual Frameworks .......................................... 55
Allan W. Wicker
Chapter 4. Translational Research ............................................ 69
Michael T. Bardo and Mary Ann Pentz
Foci of Research ................................................... 85
Chapter 5. In Defense of External Invalidity .................................... 87
Douglas G. Mook
Chapter 6. When Small Effects Are Impressive .................................. 99
Deborah A. Prentice and Dale T. Miller
Part III. Sampling and Assigning Participants to Conditions ................... 107
Samples and Selection of Participants ................................... 111
Chapter 7. Most People Are Not WEIRD ...................................... 113
Joseph Henrich, Steven J. Heine, and Ara Norenzayan
Chapter 8. The Neglected 95%: Why American Psychology Needs to Become
Less American .................................................. 115
Jeffrey J. Arnett
v
Contents
Chapter 9. Amazon’s Mechanical Turk: A New Source of Inexpensive,
Yet High-Quality, Data? ........................................... 133
Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling
Randomization and Group Equivalence ................................. 141
Chapter 10. Random Sampling, Randomization, and Equivalence of Contrasted
Groups in Psychotherapy Outcome Research ......................... 143
Louis M. Hsu
Research Design Options ............................................. 153
Chapter 11. Experimental and Observational Designs: An Overview ................. 155
Alan E. Kazdin
Part IV. Assessment .................................................... 181
Measurement Development, Reliability, and Validity ....................... 185
Chapter 12. Constructing Validity: Basic Issues in Objective Scale Development ....... 187
Lee Anna Clark and David Watson
Chapter 13. Selecting Measures for Research Investigations ........................ 205
Alan E. Kazdin
Chapter 14. Measurement and Assessment: An Editorial View ..................... 227
Cecil R. Reynolds
Measurement Among Diverse Samples .................................. 233
Chapter 15. Methodological Issues in Assessment Research
With Ethnic Minorities .......................................... 235
Sumie Okazaki and Stanley Sue
Part V. Data Analysis, Evaluation, and Presentation .......................... 249
Background and Underpinnings of Data Analyses ......................... 255
Chapter 16. On the Origins of the .05 Level of Statistical Significance ................ 257
Michael Cowles and Caroline Davis
Chapter 17. Things I Have Learned (So Far) ................................... 265
Jacob Cohen
Null Hypotheses Testing, Meta-Analysis, and Bayesian Statistics. . . . . . . . . . . . . . 277
Chapter 18. A Power Primer ................................................ 279
Jacob Cohen
Chapter 19. Statistical Significance Testing and Cumulative Knowledge
in Psychology: Implications for Training of Researchers ................. 285
Frank L. Schmidt
Chapter 20. An Effect Size Primer: A Guide for Clinicians and Researchers ........... 301
Christopher J. Ferguson
vi
Contents
Chapter 21. The Proof of the Pudding: An Illustration of the Relative
Strengths of Null Hypothesis, Meta-Analysis, and
Bayesian Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
George S. Howard, Scott E. Maxwell, and Kevin J. Fleming
Data Presentation .................................................. 331
Chapter 22. Designing Better Graphs by Including Distributional Information
and Integrating Words, Numbers, and Images ........................ 333
David M. Lane and Anikó Sándor
Part VI. Special Topics: Evaluation in Clinical Practice and Research ........... 355
Assessment and Evaluation in Clinical Work ............................. 359
Chapter 23. Collecting Client Feedback ....................................... 361
Michael J. Lambert and Kenichi Shimokawa
Chapter 24. Revisiting and Reenvisioning the Outcome Problem in
Psychotherapy: An Argument to Include Individualized
and Qualitative Measurement ..................................... 373
Clara E. Hill, Harold Chui, and Ellen Baumann
Evaluating Interventions ............................................. 387
Chapter 25. Beyond Efficacy and Effectiveness: A Multifaceted Approach
to Treatment Evaluation ......................................... 389
Timothy D. Nelson and Ric G. Steele
Chapter 26. Methodological Challenges in Treatment Outcome Research
With Ethnic Minorities .......................................... 403
Anna S. Lau, Doris F. Chang, and Sumie Okazaki
Part VII. Multiple Methodologies ......................................... 415
Chapter 27. The Epistemology of Mathematical and Statistical Modeling:
A Quiet Methodological Revolution ................................ 419
Joseph Lee Rodgers
Chapter 28. Qualitative Research and Its Place in Psychological Science .............. 437
Anna Madill and Brendan Gough
Chapter 29. Single-Case Experimental Research Designs .......................... 459
Alan E. Kazdin
Part VIII. Ethical Issues and Scientific Integrity ............................. 485
Guidelines and Codes ............................................... 493
Chapter 30. Ethical Principles of Psychologists and Code of Conduct ................ 495
American Psychological Association
Chapter 31. Research Ethics: How to Treat People Who Participate
in Research ................................................... 513
Ezekiel Emanuel, Emily Abdoler, and Leanne Stunkel
vii
Contents
Chapter 32. Research Ethics for Mental Health Science Involving Ethnic Minority
Children and Youths ............................................ 525
Celia B. Fisher, Kimberly Hoagwood, Cheryl Boyce, Troy Duster,
Deborah A. Frank, Thomas Grisso, Robert J. Levine, Ruth Macklin,
Margaret Beale Spencer, Ruby Takanishi, Joseph E. Trimble,
and Luis H. Zayas
Questionable Research Practices, Fraud, and Professional
Responsibilities .................................................... 545
Chapter 33. False-Positive Psychology: Undisclosed Flexibility in Data
Collection and Analysis Allows Presenting Anything
as Significant .................................................. 547
Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn
Chapter 34. Why Has the Number of Scientific Retractions Increased? ............... 557
R. Grant Steen, Arturo Casadevall, and Ferric C. Fang
Chapter 35. Best Practices for Allocating Appropriate Credit and Responsibility
to Authors of Multi-Authored Articles .............................. 569
Lucas D. Eggert
Part IX. Reproducibility of Findings: Replication and Improving
Research Practices .............................................. 577
Chapter 36. Shall We Really Do It Again? The Powerful Concept of Replication
Is Neglected in the Social Sciences ................................. 581
Stefan Schmidt
Chapter 37. The Empirical March: Making Science Better at Self-Correction .......... 597
Matthew C. Makel
Chapter 38. Recommendations for Increasing Replicability in Psychology ............ 607
Jens B. Asendorpf, Mark Conner, Filip de Fruyt, Jan De Houwer,
Jaap J. A. Denissen, Klaus Fiedler, Susann Fiedler, David C. Funder,
Reinhold Kliegl, Brian A. Nosek, Marco Perugini, Brent W. Roberts,
Manfred Schmitt, Marcel A. G. van Aken, Hannelore Weber,
and Jelte M. Wicherts
Part X. Publication and Communication of Research ......................... 623
Reporting Standards: What to Cover and Include in an Article ............... 627
Chapter 39. Reporting Standards for Research in Psychology:
Why Do We Need Them? What Might They Be? ...................... 629
APA Publications and Communications Board Working Group on
Journal Article Reporting Standards
viii
Contents
Preparing Articles for Publication ...................................... 645
Chapter 40. Publication and Communication of Research Findings ................. 647
Alan E. Kazdin
Chapter 41. Writing a Review Article for Psychological Bulletin .................... 663
Daryl J. Bem
Part XI. Perspectives on Methodology ................................... 673
Chapter 42. Methodology: Perspectives and General Lessons to Guide Research ....... 675
Alan E. Kazdin
Index .................................................................. 687
About the Editor ........................................................ 703
ix