Assessing the Viva in Higher Education: Chasing Moments of Truth

211 Pages·2018·2.478 MB·English

by  Stephen Dobson (auth.)

The Enabling Power of Assessment 6
Series Editor: Claire Wyatt-Smith
Stephen Dobson
Assessing the
Viva in Higher
Education
Chasing Moments of Truth
The Enabling Power of Assessment
Volume 6
Series editor
ClaireWyatt-Smith,FacultyofEducationandArts,AustralianCatholicUniversity,
Brisbane, Queensland, Australia
Thisseriesheraldstheideathatnewtimescallfornewanddifferentthinkingabout
assessmentandlearning,theidentitiesofteachersandstudents,andwhatisinvolved
inusingandcreatingnewknowledge.Itsscopeisconsistentwithaviewofassessment
as inherentlyconnected withcultural, social practices and contexts. Assessment isa
shared enterprise where teachers and students come together to not only develop
knowledge and skills, but also touse and create knowledge and identities. Working
from this position, the series confronts some of the major educational assessment
issuesofourtimes.
More information about this series at http://www.springer.com/series/13204
Stephen Dobson
Assessing the Viva in Higher
Education
Chasing Moments of Truth
123
StephenDobson
Division of Education, Arts andSocial
Sciences
University of SouthAustralia
Adelaide, SA
Australia
ISSN 2198-2643 ISSN 2198-2651 (electronic)
TheEnabling Power of Assessment
ISBN978-3-319-64014-3 ISBN978-3-319-64016-7 (eBook)
DOI 10.1007/978-3-319-64016-7
LibraryofCongressControlNumber:2017948221
©SpringerInternationalPublishingAG2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Acknowledgements
I owe my gratitude to a number of people: my wife Sonia and my mother;
Professors Back, Stobart, Klenowski, Engh, Johnson and Crotty; friends and col-
leagues Egil Hartberg, Vegard Meland, Ola Johan Sjøbakken and Eli Kari
Høihilder. Earlier versions of some of the arguments made in this book have been
published in a number of journals: Assessment and Evaluation in Higher
Education, Social Kritik, Norsk Pedagogisk Tidsskrift, Sosiologisk Årbok, www.
seminar.net and the Journal of Nordic Studies in Education. For editorial assis-
tance, my thanks are to Kate Leeson and Tiffany Muir. Last but not least, I would
also like to thank participants in this research, who remain anonymous but
remembered. All translations from Norwegian to English are my own.
TheworkonthisbookhasbeensupportedbyascholarshipfromtheNorwegian
Non-fiction Writers and Translators Association (NFF).
v
Contents
1 The Life and Death of the Viva. .... .... .... .... .... ..... .... 1
Typology or Genealogy?... ..... .... .... .... .... .... ..... .... 2
The Viva in the Time of the Greeks and Greco-Romans ... ..... .... 4
The Death of the Viva as Public Disputation .... .... .... ..... .... 8
The Rebirth of the Viva in the Middle Ages. .... .... .... ..... .... 10
The Viva in Modern Times. ..... .... .... .... .... .... ..... .... 12
The Contemporary Debate on the Viva. .... .... .... .... ..... .... 13
The Postmodern Viva. .... ..... .... .... .... .... .... ..... .... 16
ForegroundingDebateontheFutureoftheAcademicandProfessional
Viva .. .... .... .... .... ..... .... .... .... .... .... ..... .... 18
References.. .... .... .... ..... .... .... .... .... .... ..... .... 19
2 The Global Under-Theorisation of the Viva ... .... .... ..... .... 23
The Professional Viva. .... ..... .... .... .... .... .... ..... .... 24
The Academic Doctoral Viva .... .... .... .... .... .... ..... .... 25
Comparing the Research Frames for the Professional Viva
and the Academic Doctoral Viva . .... .... .... .... .... ..... .... 30
Researcher Frames on the Viva in North America
and Non-English-Speaking Countries .. .... .... .... .... ..... .... 32
Summary... .... .... .... ..... .... .... .... .... .... ..... .... 34
References.. .... .... .... ..... .... .... .... .... .... ..... .... 34
3 A Method to Theorise and Investigate the Social Practice
of the Viva. .... .... .... ..... .... .... .... .... .... ..... .... 37
The Viva as a Functional Event .. .... .... .... .... .... ..... .... 38
The Viva as a Social Practice.... .... .... .... .... .... ..... .... 40
The Viva as an Existential Experience . .... .... .... .... ..... .... 43
Methodology.... .... .... ..... .... .... .... .... .... ..... .... 49
A Qualitative Approach ... ..... .... .... .... .... .... ..... .... 50
Selecting the Vivas to be Filmed and Generalisation .. .... ..... .... 52
vii
viii Contents
Transcription.... .... .... ..... .... .... .... .... .... ..... .... 53
Analysis of the Empirical Data: Stage 1.... .... .... .... ..... .... 54
Analysis of the Empirical Data: Stage 2.... .... .... .... ..... .... 55
Analysis of the Empirical Data: Stage 3.... .... .... .... ..... .... 58
Summary... .... .... .... ..... .... .... .... .... .... ..... .... 63
References.. .... .... .... ..... .... .... .... .... .... ..... .... 63
4 The Master Degree Viva ‘Talked into Being’ .. .... .... ..... .... 67
What Evidence Was There of Language Games? . .... .... ..... .... 77
Why Bother with Conversation Analysis?... .... .... .... ..... .... 81
The Coordination of Talk with Tools and Corporeal Movements.. .... 83
Narratives in the Final Stage of the Analysis .... .... .... ..... .... 87
Extrinsic Narratives in Different Phases of the Viva... .... ..... .... 87
Summary... .... .... .... ..... .... .... .... .... .... ..... .... 89
References.. .... .... .... ..... .... .... .... .... .... ..... .... 90
5 Challenging the Genre of the Doctoral Viva... .... .... ..... .... 93
Clarifying the Research Design... .... .... .... .... .... ..... .... 97
What Have You Learnt?... ..... .... .... .... .... .... ..... .... 98
Challenging the Ontological Space of the Doctoral Viva ... ..... .... 100
The Relationship Between the Textual Dissertation
and Its Defence in a Viva.. ..... .... .... .... .... .... ..... .... 103
Reconceptualising the Ontological Space of Being
of the Dissertation and Viva..... .... .... .... .... .... ..... .... 106
Summary of the Chapter. ..... .... .... .... .... .... ..... .... 109
References.. .... .... .... ..... .... .... .... .... .... ..... .... 110
6 A Validity Argument for the Social Practice of the Viva. ..... .... 113
A Short Exegesis on the Theory of Assessment Validation.. ..... .... 114
Theoretical Rationales as Constructs... .... .... .... .... ..... .... 117
Generalisability and Construct Validity. .... .... .... .... ..... .... 118
Values. .... .... .... .... ..... .... .... .... .... .... ..... .... 120
Social and Personal Consequences .... .... .... .... .... ..... .... 121
Operationalising Messick Towards a Validity Argument
for the Viva. .... .... .... ..... .... .... .... .... .... ..... .... 121
Judgments.. .... .... .... ..... .... .... .... .... .... ..... .... 124
What Goes on When Judgments Are Made?. .... .... .... ..... .... 126
Can We Say in More Detail What Is Taking Place When
Examiners in the Viva Make Their Judgments? .. .... .... ..... .... 126
A Validity Argument for the Viva .... .... .... .... .... ..... .... 131
References.. .... .... .... ..... .... .... .... .... .... ..... .... 132
Contents ix
7 Refining the Validity Argument Model: Examples
from Bachelor and Master Degree Vivas.. .... .... .... ..... .... 135
Refining the Validity Argument Model: Some Preliminary
Points . .... .... .... .... ..... .... .... .... .... .... ..... .... 136
Case 1: A Travel and Tourism Viva... .... .... .... .... ..... .... 138
Case II: The Major in Education.. .... .... .... .... .... ..... .... 142
Case III: The Postgraduate Viva in Education.... .... .... ..... .... 146
Refining the Validity Argument Model of the Viva
as a Social Practice... .... ..... .... .... .... .... .... ..... .... 149
References.. .... .... .... ..... .... .... .... .... .... ..... .... 152
8 (Re-)Theorising the Viva.. ..... .... .... .... .... .... ..... .... 155
Questions .. .... .... .... ..... .... .... .... .... .... ..... .... 155
Theorising the Viva... .... ..... .... .... .... .... .... ..... .... 157
Core Concepts... .... .... ..... .... .... .... .... .... ..... .... 159
Social Practice. .... .... ..... .... .... .... .... .... ..... .... 160
Narratology... .... .... ..... .... .... .... .... .... ..... .... 161
Judgments.... .... .... ..... .... .... .... .... .... ..... .... 162
Constructs.... .... .... ..... .... .... .... .... .... ..... .... 164
Protective, Auxiliary Concepts ... .... .... .... .... .... ..... .... 165
Ethnicity, Gender and Socio-economic Background . .... ..... .... 166
Validity.. .... .... .... ..... .... .... .... .... .... ..... .... 166
Evidence. .... .... .... ..... .... .... .... .... .... ..... .... 167
Learning . .... .... .... ..... .... .... .... .... .... ..... .... 167
Contextualising the Theory of the Viva .... .... .... .... ..... .... 170
Future Directions and Policy Implications... .... .... .... ..... .... 173
References.. .... .... .... ..... .... .... .... .... .... ..... .... 177
Appendix A: Interview Guides..... .... .... .... .... .... ..... .... 181
Appendix B: Grading Sheet.. ..... .... .... .... .... .... ..... .... 183
Appendix C: Transcription Convention . .... .... .... .... ..... .... 185
Appendix D: Example of a Viva Assessment Rubric in Advanced
Course on Assessment Practices.... .... .... .... .... .... ..... .... 187
Glossary.. .... .... .... .... ..... .... .... .... .... .... ..... .... 191
Introduction
The candidates for the exams are the yearly lambs, some of
whomhavetobesacrificedattheexaminationtable,inorder
tomaintainthesociallynecessarycommonknowledge.
Kvale(2000:10)
For six years, I researched refugee work in a Norwegian municipality. The dis-
sertation was submitted to a UK university in the spring of 1999 and a date in the
early autumn was set for the viva. As the day of the viva drew closer, and I began
my journey by boat to Newcastle from Bergen, I felt a certain anxiety: I had no
guarantee that I would pass the viva. All the work might be in vain. My appre-
hensiononthewaytotheroomwherethetwoexaminerswaitedwasfurtherraised
whenmysupervisorsaid,‘95%ofcandidatesdonotgetthroughontheirfirsttry’.
It was a strange comment; had he already conferred with the examiners and knew
my fate?
Myvivabeganat2pmandfinishedat4.45pm.Iwastoldtowaitinthecorridor
whilethetwoexaminersarrivedattheirfinaldecision.TheyawardedmethePh.D.
on the condition that I made some minor revisions. I had been ‘grilled’ for almost
3 hwithoutanintermission.Theywereconsensualintheirdislikeofthemannerin
which I had used and interpreted the work of certain theorists (e.g. Foucault), my
methodology and my narrowing of the research topic to refugees in a small
(Scandinavian) country. In the course of the viva, I felt an adrenalin kick as I
answeredquestionafter question,arguingrationally andself-critically. Irefusedto
buckle under the pressure.
DidIresentthiskindofviva?Ontheonehand,thiswasnotaceremonialkindof
viva,whereImighthavebeentoldonenteringtheroomthatIhadpassed,andthat
a pleasant academic conversation would follow. In the viva for my Magister in
Sociology in Norway in 1990 (defending a 247-page dissertation after a public
lecture and two 10-hour written examinations), the first sentence of one of the
examinershadbeen,‘Ihopewecanbefinishedin45 minbecauseIhaveatrainto
catch’. The result in his opinion was already a foregone conclusion. In my Ph.D.
xi