Becoming A Behavioral Science Researcher: A Guide To Producing Research That Matters

379 Pages·2020·7.261 MB·English

by  Rex B. Kline

ebook
THE GUILFORD PRESS
Becoming a
BEHAVIORAL SCIENCE RESEARCHER
Also Available
Principles and Practice
of Structural Equation Modeling,
Fourth Edition
Rex B. Kline
Becoming a
BEHAVIORAL
SCIENCE
RESEARCHER
A Guide to Producing Research
That Matters
S E C O N D E D I T I O N
Rex B. Kline
THE GUILFORD PRESS
New York London
For Joanna, Julia, and Luke
Copyright © 2020 The Guilford Press
A Division of Guilford Publications, Inc.
370 Seventh Avenue, Suite 1200, New York, NY 10001
www.guilford.com
All rights reserved
No part of this book may be reproduced, translated, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical,
photocopying, microfilming, recording, or otherwise, without written
permission from the publisher.
Printed in the United States of America
This book is printed on acid-free paper.
Last digit is print number: 9 8 7 6 5 4 3 2 1
Library of Congress Cataloging-in-Publication Data
Names: Kline, Rex B., author.
Title: Becoming a behavioral science researcher : a guide to producing
research that matters / Rex B. Kline.
Description: Second edition. | New York : Guilford Press, [2020] | Includes
bibliographical references and index.
Identifiers: LCCN 2019010917| ISBN 9781462538799 (pbk.) |
ISBN 9781462541287 (hardcover)
Subjects: LCSH: Psychology—Research.
Classification: LCC BF76.5 .K54 2020 | DDC 150.72—dc23
LC record available at https://lccn.loc.gov/2019010917
Preface and
Acknowledgments
There have been major developments in the behavioral sciences since
the publication of the first edition of this book in 2009. To summa-
rize:
1. The replication crisis expanded from mainly an academic debate
within psychology to a much broader public issue that now involves
other disciplines, too. The crisis involves the growing problem of
failed reproducibility of published research in psychology and
other areas.
2. Awareness of the potential for p hacking, or how basically any
result can be presented as statistically significant through deci-
sions that are not always disclosed, has increased to the point
where the credibility of our research literature is cast in doubt.
That p hacking is an enabler of scientific fraud also contributes to
a credibility crisis.
3. The use of significance testing was banned by a major journal in
social psychology, and increasing numbers of journals, especially
in health- related fields, basically forbid the use of significance
testing without estimating effect sizes.
v
vi Preface and Acknowledgments
4. Reforms from the open- science movement, including open access,
open data, open peer review, and preregistration of analysis plans,
are being implemented by more and more journals and govern-
ment agencies that fund research.
5. The American Psychological Association has published revised
journal article reporting standards for quantitative research and
a new set of standards for qualitative research. Hundreds of other
reporting standards for research in other areas have also been
published by additional groups or associations. Such standards
address a reporting crisis where insufficient information about
the analysis is presented in too many journal articles.
Today’s thesis students need to know about these five developments. This
is why there are new or extensively revised chapters in this second edition
that deal with research crises (Chapter 3), reporting standards (Chapter
4), open science (Chapter 5), statistics reform (Chapter 6), effect size esti-
mation (Chapter 7), and psychometrics (Chapter 8).
Some challenges for thesis students are relatively unchanged; that is,
they are still germane. To summarize:
1. Even after a few introductory courses in statistics and research
design, thesis students often feel ill prepared when analyzing data
from their own projects. This is because practical analysis skills
are typically not emphasized in such courses, and there are often
surprising gaps in knowledge of basic statistical principles among
many, if not most, new graduate students. That such knowledge
gaps are also apparent among even established researchers may
be less well known but is just as real.
2. Many thesis students struggle with writing, in part because uni-
versity students are not assigned meaningful writing tasks in too
many courses at the undergraduate level. This means that stu-
dents in even specialization or honors programs can enter a the-
sis course with essentially little or no refinement in their writing
skills since high school.
3. Most PowerPoint presentations seen in classes, colloquia, or other
contexts in educational or business settings are awful. This means
that students see mainly bad examples of PowerPoint presenta-
tions, which makes it difficult for them to develop a sense of how
to do better.
Preface and Acknowledgments vii
Chapter 2 deals with the trinity of research: design, measurement, and
analysis. As in the previous edition, students are encouraged to (1) think
about research, design, and measurement as parts of an integrated whole,
and (2) understand that flaws in any individual part can affect the quality
of results. Still emphasized in Chapter 9, about practical data analysis, is
the idea of a minimally sufficient analysis, or the advice to use the simplest
statistical technique that both addresses the hypotheses and is understood
by the student. Readers are also informed of freely available computer
tools for statistical analyses, including R, JASP, and PSPP. Chapter 10, on
writing, has been updated to reflect the increasing reliance on reporting
standards in the review process for manuscripts submitted to journals.
Finally, there is an even stronger emphasis in Chapter 11, about presenta-
tions, on designing slides that are simple, effective, and free of the usual
distracting clutter seen in so many PowerPoint presentations that annoy
more than enlighten due to flawed organization, preparation, or visual
aids.
Earlier drafts were much improved based on extremely helpful com-
ments and suggestions by a total of seven reviewers. The names of six
reviewers were revealed to me only after the writing was complete, and
their original comments were not associated with their names. The sev-
enth reviewer contacted me about the status of the second edition and
then graciously volunteered to review chapter drafts. A big “thank you”
to all reviewers:
••Julie Combs, College of Education, Educational Leadership, Sam
Houston State University
••Anne Corinne Huggins- Manley, College of Education, School of
Human Development and Organizational Studies in Education,
University of Florida
••Michael Karcher, College of Education and Human Development,
Counseling, University of Texas at San Antonio
••Craig D. Marker, College of Health Professions, Clinical Psychol-
ogy, Mercer University
••Diane Montague, School of Arts and Science, Psychology, LaSalle
University
••James Schuurmans- Stekhoven, School of Psychology, Charles Sturt
University, Bathhurst, New South Wales, Australia
viii Preface and Acknowledgments
••Rachel A. Smith, College of the Liberal Arts, Communication Arts
and Sciences, Pennsylvania State University
It was both a pleasure and an honor to work again with C. Debo-
rah Laughton, Research Methods and Statistics Publisher at The Guil-
ford Press. An author could not hope to have a better ally, but “guardian
angel” is probably a better term here. Issues of permissions were dealt
with quickly— and also with good style and wry humor— by Robert Sebas-
tiano, Permissions Coordinator at Guilford. It was good to work with
Guilford Senior Production Editor Laura Specht Patchkofsky and Art
Director Paul Gordon, who designed the terrific book cover. The original
manuscript for the second edition was expertly copyedited with many
helpful suggestions by Betty Pessagno.
I am grateful to the many psychology honors program students in my
sections of our year- long thesis course at Concordia. They have shared with
me their aspirations and frustrations in becoming more skilled research-
ers, and their experiences provide the background for many chapters in
this book. It has been rewarding for me to have played even a small role
in helping them to build the foundations for later professional careers.
As always, my deepest gratitude goes to my wife, Joanna, and our
children, Julia and Luke. Thank you for all your love and support during
the writing of this book.