Designing Listening Tests: A Practical Approach

225 Pages·2017·3.303 MB·English

by  Rita Green (auth.)

Designing
Listening
Tests: A
Practical
Approach
RITA GREEN
Designing Listening Tests: A Practical Approach
‘Rita Green has spent many years at the coalface of language test develop-
ment and training in a variety of international contexts; this book is the
sum of this experience. This book is a fantastic resource for anyone look-
ing to develop listening tests: a highly practical, theoretically-grounded
guide for teachers and practitioners everywhere. Green covers a range of
important principles and approaches; one highlight is the introduction
to the “textmapping” approach to working with sound files. This book is
highly recommended for anyone involved in the development of listen-
ing tests.’
—Luke Harding, Senior Lecturer, Lancaster University, UK
Designing Listening Tests:
A Practical Approach
Rita Green
Rita Green
UK
ISBN 978-1-137-45715-8 ISBN 978-1-349-68771-8 (eBook)
DOI 10.1057/978-1-349-68771-8
Library of Congress Control Number: 2016950461
© The Editor(s) (if applicable) and The Author(s) 2017
The author(s) has/have asserted their right(s) to be identified as the author(s) of this work in accordance
with the Copyright, Designs and Patents Act 1988.
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and trans-
mission or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made.
Printed on acid-free paper
This Palgrave Macmillan imprint is published by Springer Nature
The registered company is Macmillan Publishers Ltd.
The registered company address is: The Campus, 4 Crinan Street, London, N1 9XW, United Kingdom
Preface
Who is this book for?
This book is primarily aimed at teachers who would like to develop listen-
ing tests for their students but who have little, if any, training in the field
of assessment. It is also designed for test developers who have some expe-
rience of assessing the listening ability of test takers, but who would like
a practical reference book to remind them of the procedures they should
follow, and of the many do’s and dont’s that litter the field of task develop-
ment. Those who are engaged in MA studies, or other types of research,
should also find Developing Listening Tests (DLT) of interest as the book
raises many issues which would benefit from further investigation.
DLT offers a systematic approach to the development of listening tasks,
starting with a discussion of what listening involves, and the importance
of drawing up test specifications. It also explores how to exploit sound
files and investigates a range of issues related to task development. The
book concludes with a look at the benefits of trialling and data analysis,
and how to report test scores and set pass marks.
Not everyone reading this book will be able to carry out all of these
recommended stages. In many cases, even where test developers would
like to do this, the demands and limitations of their assessment contexts
make some stages very difficult to achieve. What is of importance is to
attempt to do as many as possible.
v
vi Preface
The organisation of this book
Each chapter focuses on one major aspect of the task development cycle.
Chapter 1 starts with an overview of the issues which a test developer
needs to consider when developing a listening test. These include looking
at the processes which are involved in real-life listening, how the spoken
and written forms of the language differ and what makes listening dif-
ficult. The chapter ends with a discussion on why listening is important
and introduces the reader to the task development cycle.
Chapter 2 discusses the role that test specifications play in assisting the
test developer to define the construct underlying the test, and to describe
the conditions under which the test taker’s performance will be measured.
Chapter 3 introduces the reader to a procedure called textmapping which
helps test developers to determine the appropriateness of the sound files
they would like to use in their task development work and explores how
those sound files can be exploited.
Chapter 4 focuses on task development, investigates many of the deci-
sions that need to be made at this stage, and provides a set of item writing
guidelines to help in this process. The chapter also discusses the role of
peer review in task development and provides an outline of how this feed-
back could work. Chapter 5 consists of a range of sample listening tasks
taken from a number of different testing projects. Each task is discussed
in turn providing insights into the listening behaviour, the sound file and
the task. Links to the sound files are also provided.
Chapter 6 focuses on the benefits to be gained from trialling the listen-
ing tasks and carrying out data analysis. Chapter 7 explores the different
ways test scores can be reported and how pass marks (or cut scores) can
be calculated. Readers are provided with insights into how a standard
setting session can be run and the importance of producing a post-test
report is discussed.
Good luck with the book and the task development process!
Rita Green
UK
Acknowledgements
I would like to start by thanking my colleagues and friends for their
feedback on previous versions of these chapters. A special mention goes
to Karmen Pižorn, Irene Thelen-Schaefer, Caroline Shackleton, David
Gardner, Heidi Ford-Schmidt and Astrid Dansoko.
I would also like to express my thanks to the following people
and organisations who have provided me with copyright permis-
sion to include the tasks and/or sound files used in this book: Graham
Hyatt, Länderverbundprojekt VerA6, Germany; Julia Grossmann
& Linnet Souchon; Walter Indra; The BundesInstitut, Zentrum für
Innovation und Quälitatsentwicklung (Bifie), Austria; The Institut zur
Qualitätsentwicklung im Bildungswesen (IQB), Humboldt-Universität
zu Berlin, Germany; Devawongse Varopakarn Institute of Foreign
Affairs (DVIFA), Ministry of Foreign Affairs, Thailand; Paul Vogel;
The Department of Foreign Affairs and Trade (DFAT), Australian
Government; iPod traveller: www.ipodtraveller.net; Star Radio,
Cambridge, UK; Nathan Turner, Centro de Lenguas Modernas, Granada
University, Spain; and Luke Harding, Lancaster University, UK.
Reprint of SPSS screen images courtesy of International Business
Machines Corporation, © SPSS, Inc., an IBM Company.
vii
Contents
1 What is involved in assessing listening? 1
1.1 W hat the listening process involves 2
1.2 H ow listening differs between contexts and listeners 5
1.3 H ow listening input varies 7
1.4 H ow the spoken and written forms of the language differ 8
1.5 W hat makes listening difficult? 11
1.5.1 Nature of listening 11
1.5.1.1 No permanent record 11
1.5.1.2 Lack of ‘real’ gaps 12
1.5.1.3 Lack of redundancy 12
1.5.2 Complexity of processing 13
1.5.2.1 Multi-tasking 13
1.5.2.2 Controlled versus automatic processing 14
1.5.3 Input 14
1.5.3.1 Content 14
1.5.3.2 Topic 14
1.5.3.3 Sound quality 15
1.5.3.4 Mode of delivery 16
1.5.4 Task 16
1.5.5 Listening environment 17
ix
x Contents
1.5.6 Speaker characteristics 17
1.5.6.1 Speed of delivery 17
1.5.6.2 Number and type of voices 18
1.5.7 Listeners’ characteristics 18
1.6 W hy is assessing listening important? 19
1.7 S ummary 20
1.7.1 Task development cycle 21
2 How can test specifications help? 27
2.1 W hat are test specifications? 27
2.2 Purpose of the test 28
2.3 Target test population 28
2.4 The construct 29
2.5 Performance conditions 34
2.5.1 Input 35
2.5.1.1 Source 35
2.5.1.2 Authenticity 37
2.5.1.3 Quality 38
2.5.1.4 Level of difficulty 39
2.5.1.5 Topics 40
2.5.1.6 Discourse type 40
2.5.1.7 Nature of content 41
2.5.1.8 Number of sound files needed 41
2.5.1.9 Length of sound files 42
2.5.1.10 Mode of delivery 43
2.5.1.11 Number of times heard 43
2.5.1.12 Speaker characteristics 45
2.5.2 Task 46
2.5.2.1 Instructions and the example 46
2.5.2.2 Test method 46
2.5.2.3 Number of items 47
2.5.2.4 Number of tasks 48
2.5.3 Criteria of assessment 48
2.6 W hy do we need test specifications? 49
2.7 S ummary 51