Cobb, Kate (2015) The impact of assessment on constructive alignment of a modern veterinary ...

374 Pages·2017·4.16 MB·English

by  Christine Middleton

THE IMPACT OF ASSESSMENT ON THE
CONSTRUCTIVE ALIGNMENT OF A MODERN
VETERINARY CURRICULUM

KATE COBB

Thesis submitted to the University of
Nottingham for the Degree of Doctor of
Philosophy

July 2015

1
Abstract

Constructive alignment (CA) describes an approach to education where
teaching, learning and assessment are aligned to allow the learner to
achieve the intended learning outcomes (ILOs) of the course. Assessment
has a strong influence on learning and therefore the potential to have
either a positive or negative impact on CA. The aim of the research in this
thesis is to explore the effects of assessment on CA.
The context for the research is the final year of study within the School of
Veterinary Medicine and Science (SVMS), at the University of Nottingham.
Five mixed methods studies were conducted utilising questionnaires,
semi-structured interviews and focus groups. In study one the ILOs of the
course were defined and subsequently used in study two as a framework
for an alumni survey to determine how well prepared graduates feel for
clinical practice. Studies three and four investigated the impact of
assessments on learning behaviour, namely multiple-choice questions
(MCQ), directly observed procedural skills (DOPS) and the script
concordance test (SCT). Finally in study five the influence of the transition
to practice on learning behaviour during final year was explored.
Graduates felt well prepared by the SVMS curriculum for a career in
practice with respect to all ILOs. However, assessments were not rated so
highly when considered alone. DOPS and to some extent the SCT are
considered to be authentic assessment formats and encourage a deep
approach to learning. The MCQ in this context results in surface learning
strategies being adopted. The imminent transition to practice has a
2
positive effect on learning behaviour, however this conflicts with the
preparation required for final year examinations.
Elements of the assessment strategy that have a positive and negative
impact on CA are identified and discussed. Changes to the curriculum are
proposed to enhance CA and ease the transition to practice.

3
For Mum and Dad

4
Acknowledgements

I would like to thank the University of Nottingham for funding this PhD
and my supervisors Professor Richard Hammond and Professor Sarah
Freeman for their advice and support. I am especially grateful to Dr Liz
Mossop not only for agreeing to supervise my research but also for her
friendship and patience, thank you Liz.
I would also like to thank Professor Debbie Jaarsma for her advice in the
early stages of the research and Professor George Brown whose help and
guidance has been greatly appreciated throughout this project.
Thank you to all the practitioners, SVMS staff, students and alumni who
volunteered to participate in each of the research studies.
Finally I would like to say a massive thanks to my family, Mal, Steph and
Jack who have all been supportive and at times extremely tolerant of me
during this PhD.

5
“For every evaluative action there is an equal (or greater)
(and sometimes opposite) educational reaction”
Lambert Schuwirth, the “law” of educational cause and effect
6
Table of Contents
Abstract ......................................................................................... 2
Acknowledgements ........................................................................ 5
List of Figures .............................................................................. 13
List of Tables ................................................................................ 15
Abbreviations: .............................................................................. 17
Chapter 1: Introduction ............................................................... 18
1.1: The history of OBE within veterinary curricula ...................................................... 21
1.2: The SVMS curriculum ........................................................................................... 24
1.3: Implications for Research ..................................................................................... 30
Chapter 2: A review of the relevant literature .............................. 31
2.1: Introduction ........................................................................................................ 31
2.2: Outcome-Based Education (OBE) .......................................................................... 31
2.2.1: An introduction to OBE: ......................................................................................... 31
2.2.2: Defining the principles of OBE ............................................................................... 32
2.2.3: OBE within the healthcare professions .................................................................. 34
2.2.4: Criticisms of OBE .................................................................................................... 35
2.2.5: Defence of OBE in the context of a veterinary curriculum .................................... 38
2.2.6: Constructing effective outcomes ........................................................................... 41
2.3: Constructive alignment ........................................................................................ 44
2.3.1: The principles of constructive alignment ............................................................... 44
2.3.2: Examples of CA in curriculum design and development ....................................... 45
7
2.3.3: To what extent does constructive alignment exist within higher education?....... 48
2.4: Outcomes assessment ......................................................................................... 50
2.5: Indirect outcomes assessment ............................................................................. 53
2.6: Direct assessment of student performance ........................................................... 56
2.6.1: Assessment of clinical competence. ...................................................................... 56
2.6.2: The role of formative and summative assessment ................................................ 67
2.6.3: Assessing clinical competence in veterinary students ........................................... 70
2.7: Learning behaviour .............................................................................................. 84
2.8: Workplace-based learning.................................................................................... 93
2.8.1: Experiential learning .............................................................................................. 94
2.8.2 Socio-cultural perspectives ..................................................................................... 95
2.8.3 Adult learning theory .............................................................................................. 96
2.8.4 Benefits of WPBL ..................................................................................................... 97
2.8.5 Challenges of WPBL................................................................................................. 98
2.9: Transition to practice ......................................................................................... 100
2.9.1: What challenges do new graduates face? ........................................................... 101
2.9.2: What makes a successful transition to practice? ................................................. 102
2.10: Summary of Chapter Two ................................................................................. 104
Chapter 3: Research design ....................................................... 106
3.1: Mixed methods research .................................................................................... 106
3.2: Methods of data collection and analysis used within this thesis .......................... 112
3.2.1: Focus Groups ........................................................................................................ 112
3.2.2: Interviews ............................................................................................................. 113
3.2.3: Surveys ................................................................................................................. 115
3.2.4: Thematic analysis ................................................................................................. 118
3.2.5: Sampling ............................................................................................................... 119
8
3.3: Ethical approval ................................................................................................. 122
3.4: Summary of Chapter 3 ....................................................................................... 122
Chapter 4: Indirect Outcomes Assessment ................................ 124
4.1: Introduction ...................................................................................................... 124
4.2: Study 1: Defining the ILOs of the SVMS graduate ................................................ 126
4.2.1: A systematic review of the literature................................................................... 127
4.2.2: The focus group ................................................................................................... 131
4.2.3: The staff survey .................................................................................................... 132
4.2.4: Results of study 1: defining the ILOs of the SVMS graduate. .............................. 140
4.3: Indirect outcomes assessment ........................................................................... 142
4.3.1: The content of the Graduate Survey ................................................................... 142
4.3.2: Graduate Survey distribution ............................................................................... 145
4.3.3: Data analysis ........................................................................................................ 146
4.3.4: Results from the graduate survey ........................................................................ 147
4.3.5: Overview and implications of chapter 4 .............................................................. 164
Chapter 5: The Educational Impact of Assessment on Final Year
Students .................................................................................... 168
5.1: Study 3: The educational impact of assessment: a comparison of workplace based
learning and multiple choice question formats. ......................................................... 170
5.1.1 Introduction .......................................................................................................... 170
5.1.2: Methods ............................................................................................................... 171
5.1.3: Results .................................................................................................................. 173
5.1.4: Overview and implications of study 3 .................................................................. 189
5.2: Study 4: The students’ perceptions of the Script Concordance Test and its impact on
learning behaviour ................................................................................................... 193
5.2.1: Introduction ......................................................................................................... 193
9
5.2.2: Methods ............................................................................................................... 193
5.2.3: Results .................................................................................................................. 199
5.2.4: Overview and implications of study 4 .................................................................. 214
5.3: Discussion of studies in chapter 5 ....................................................................... 218
Chapter 6: What constitutes a successful transition to practice?
.................................................................................................. 222
6.1: Introduction. ..................................................................................................... 222
6.2: Methods ............................................................................................................ 224
6.2.1: The student survey............................................................................................... 224
6.2.2: The employer survey ............................................................................................ 226
6.2.3: Data Analysis ........................................................................................................ 226
6.3: Results .............................................................................................................. 227
6.3.1: Participants .......................................................................................................... 227
6.3.2: Comparison of students’ and employers’ perceptions of transition to practice and
employability .................................................................................................................. 228
6.3.3: Preparation for the first job in practice ............................................................... 235
6.3.4: The employers’ perception of a UK veterinary degree ........................................ 235
6.3.5: Support provided to new veterinary graduates .................................................. 236
6.3.6: Academic criteria and recruitment ...................................................................... 237
6.3.7: Non-academic criteria and recruitment .............................................................. 239
6.3.8: Attributes assessed at interview .......................................................................... 241
6.3.9: Influence of the type of practice on new graduate recruitment ......................... 243
6.4: Overview and implications of chapter 6 .............................................................. 246
Chapter 7: Discussion ................................................................ 253
7.1: What is the impact of assessment on CA within the SVMS curriculum? ............... 253
7.1.1: What are the intended learning outcomes of the veterinary medicine course? 253
10