Assessing competence in medicine and other health professions

432 Pages·2019·8.979 MB·English

by  Claudio Violato

Assessing Competence
in Medicine and
Other Health Professions

Assessing Competence
in Medicine and
Other Health Professions
Claudio Violato
Professor and Assistant Dean
University of Minnesota Medical School
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2019 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
International Standard Book Number-13: 978-1-138-59634-4 (Hardback)
International Standard Book Number-13: 978-1-4987-8508-2 (Paperback)
This book contains information obtained from authentic and highly regarded sources.
Reasonable efforts have been made to publish reliable data and information, but the author and
publisher cannot assume responsibility for the validity of all materials or the consequences of
their use. The authors and publishers have attempted to trace the copyright holders of all m aterial
reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write
and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, repro-
duced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now
known or hereafter invented, including photocopying, microfilming, and recording, or in any
information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC),
222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that
provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks,
and are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Names: Violato, Claudio, author.
Title: Assessing competence in medicine and other health professions / by
Claudio Violato.
Description: Boca Raton : Florida : CRC Press, [2019] | Includes
bibliographical references and index.
Identifiers: LCCN 2018032562 | ISBN 9781138596344 (hardback : alk. paper) |
ISBN 9781498785082 (pbk. : alk. paper) | ISBN 9780429426728 (e-book)
Subjects: | MESH: Clinical Competence | Professional Competence | Educational
Measurement | Formative Feedback.
Classification: LCC RA399.A1 | NLM W 21 | DDC 616—dc23
LC record available at https://lccn.loc.gov/2018032562
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Dedication
LeRoy Douglas Travis (October 14, 1940–August 9, 2017)
A dear friend and inspirational teacher

Contents
Preface ix
Author xi
SECTION I FOUNDATIONS 1
1 Introduction 3
2 Competence and professionalism in medicine and the health
professions 35
3 Statistics and test score interpretation 55
4 Correlational-based methods 83
SECTION II VALIDITY AND RELIABILITY 109
5 Validity I: Logical/Face and content 115
6 Validity II: Correlational-based 139
7 Validity III: Advanced methods for validity 163
8 Reliability I: Classical methods 191
9 Reliability II: Advanced methods 213
SECTION III TEST CONSTRUCTION AND EVALUATION 231
10 Formats of testing for cognition, affect, and psychomotor skills 235
11 Constructing multiple-choice items 257
12 Constructed response items 285
13 Objective structured clinical exams 301
14 Checklists, rating scales, rubrics, and questionnaires 319
15 Evaluating tests and assessments: Item analyses 341
16 Grading, reporting, and standard setting 365
Glossary of Terms 395
Index 409
vii

Preface
Assessing competence in medicine and other health professions continues to be a
challenge. The assessment and evaluation of competence and learning have their
origins in antiquity but are only now beginning to emerge as a unified field. This
is due to developments in statistical and mathematical theory, test theory, and
advances in computer hardware and software, as well as the developing internet.
Testing and assessment are the acts of quantifying an educational or psychologi-
cal dimension or assigning a numerical value to it. Evaluation requires a value
judgment to be made based on the measurement. While the history of testing
dates back two millennia, it has emerged on a large scale only in the 20th century,
coincidental with mass education.
The current status of testing is paradoxical: it is on the increase and receives
overwhelming public support, and yet most teachers, instructors, and professors
(who construct, administer, and interpret the vast majority of tests) have little or
no formal education in testing. Tests can be used to motivate students, enhance
learning, provide feedback, evaluate educational programs, and for research.
Other functions include curriculum revisions, selection and screening, certifica-
tion, and guidance and counselling. Tests can be used in a variety of ways and
for several evaluation functions. Testing is likely to expand in modern society.
The improvement of healthcare professional competence depends, in part, on the
improvement of testing and assessment of medical competence. The alarming
rate of medical errors that currently result in death or negative outcomes may be
reduced.
My purpose in writing this book is to provide a resource for teachers, a ssessors,
administrators, pedagogues, generally, and students in health professions educa-
tion for understanding, implementing, and critically evaluating testing, assess-
ment, and evaluation. This is a comprehensive book on how to assess competence
in medicine, medical education, and healthcare throughout the clinician’s career.
It is organized into three main sections: (1) Foundations, (2) Validity and reli-
ability, and (3) Test construction and evaluation consisting of 16 chapters. Each
chapter begins with an advanced organizer and contains summaries, reflections,
and exercises.
The book also contains a glossary of testing and statistical terms—e.g.,
reliability, validity, standard error of measurement, biserial correlation,
ix