Challenges in Language Testing Around the World: Insights for Language Test Users PDF
Preview Challenges in Language Testing Around the World: Insights for Language Test Users
Betty Lanteigne Christine Coombe James Dean Brown Editors Challenges in Language Testing Around the World Insights for language test users Challenges in Language Testing Around the World · · Betty Lanteigne Christine Coombe James Dean Brown Editors Challenges in Language Testing Around the World Insights for language test users Editors BettyLanteigne ChristineCoombe LCCInternationalUniversity HigherCollegesofTechnologyDubai Klaipeda,Lithuania DubaiMen’sCollege Dubai,UnitedArabEmirates JamesDeanBrown UniversityofHawai‘iatMa¯noa Honolulu,HI,USA ISBN978-981-33-4231-6 ISBN978-981-33-4232-3 (eBook) https://doi.org/10.1007/978-981-33-4232-3 ©SpringerNatureSingaporePteLtd.2021 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Foreword The themes of most edited collections in language assessment address traditional areasoftestdevelopment,scoringmodels,statisticalanalyses,scoreinterpretation, research, and policy. In addition, they typically provide guidelines for idealized situationswithnearperfectsolutions.Thereisalmostnevermentionofthingsthat couldgetoverlooked,misapplied,misinterpreted,misused,andareerroneousintest development,practice,research,andpolicy.Thisiswherethiscollectioncomesin— itrevealshumaninfallibilitiesandmisinterpretationsinthedevelopment,practice, and research in a variety of contexts from many countries. It then offers reflective chaptersonhowtoresolveorreconcilethesematters.Thisisanuntoldstoryinthe fieldoflanguageassessment. Thethirty-sevenchaptersinsixpartsinthiscollectionaredevotedtoexperience- basedaswellasdata-basedissuesbutarecenteredonreflectingonmistakes,issuesof washback,fairness,andconstruct-irrelevance.Chaptersreflectonfailingtoconsider theeffectsofdescriptivestatisticsininterpretingotherimportanttestingstatisticslike reliability estimates, correlations, criterion-related validity, and failing to account for the assumptions that underlie higher-order statistics. Other chapters reflect on importantchallengesrelatedtofairnessofscoreuse,washback,andconsequences aswellastheuseoflocaltestdevelopment,high-stakesassessments,andplacement andclassroomassessments. Oneofthetravestiesinthefieldhasbeenthatinsightsintotestdevelopment,prac- tice,research,andpolicyhavedependedlargelyoninsightsfromWesternEnglish- speakingworldcountries.Suchaperspectiveislimitinginvisionandlackingmean- ingfulnessandapplicabilitytoothercontexts.Thus,itisrefreshingforacollection toincludechaptersonassessinglanguagevarieties(British,Australasian,andNorth American),indifferentlanguages(Arabic,Hebrew,andSlovenian)fromavarietyof countries(Japan,Ukraine,France,Iran,MexicoaswellastheUSA).Furthermore,it isencouragingtohavechaptersthatdiscussassessingtraditionalskillareas(listening, speaking,reading,andwriting)fromtheperspectiveoflocalhigh-stakesplacement testsandclassroomtests.Ofcourse,thebackboneofassessmentinschools,colleges anduniversitiesislanguageteachers.Itiscriticalthereforethattheyhavetherequi- site understanding and methods to deal with routine as well as challenging high- stakescontexts.Thisburgeoningarea,nowreferredtoasassessmentliteracy,isalso v vi Foreword includedinthiscollectionwithperspectivesfromteachersfromTurkey,SriLanka, Bangladesh,Malta,Iran,theUnitedArabEmirates,andtheUSA. Another delightful aspect of this collection is the inclusion of authors who are probablyfirst-timeauthorsorlesspublishedresearchersalongwithwell-knownand famousauthors.Thisinclusionisanintelligentchoiceastheformergroupofauthors couldofferaviewthatisnewandeye-openingwhilethelattergroupmightseekto consolidatetheirwell-knownpositionsonemoretime.Thisideaofmixingauthors from different regions can also be seen in the choice of editors of this collection: Betty Lanteigne (from Europe), Christine Coombe (from the Asia/Middle-East), and James Dean Brown (from the USA). Such collaboration can surely bring to theforemultipleperspectivesinchoosingauthors,themes,andchapters.Weseean outstandingexampleofthisinthisextraordinaryvolume. SanGabriel,CA,USA AntonyJohnKunnan October2020 Acknowledgments First and foremost we would like to thank all of the contributors for sharing their observations about, experiences with and investigations of challenges in language testing in many parts of the world. We also appreciate their patience throughout theentirepublicationprocess.GratitudeisalsoextendedtotheSpringerreviewers, whoseinsightfulcommentshelpedshapeChallengesinLanguageTestingAroundthe World.SpecialthanksgototheSpringerteam:LawrenceLiu,SeniorEditor,whose initiative was the spark that began this project; Lay Peng Ang, Senior Editorial Assistant; Sriram Srinivas, Project Coordinator; and Grace Liyan Ma, Editor, for theirorganizedandpracticalassistancewithproducingthiseditedvolume. Wededicatethisvolumetoourstudentsandcolleagueswhohavehelpeduswork throughassessmentchallengesandturnedthemintolearningopportunities. vii Contents 1 IntroducingChallengesinLanguageTestingAroundtheWorld ..... 1 BettyLanteigne,ChristineCoombe,andJamesDeanBrown PartI LearningfromLanguageTestInterpretationProblems, NegativeEffects,orMisuse 2 Problems Caused by Ignoring Descriptive Statistics inLanguageTesting ........................................... 15 JamesDeanBrown 3 Disregarding Data Due Diligence Versus Checking andCommunicatingParametricStatisticalTestingProcedure Assumptions .................................................. 25 PhillipB.Rowles 4 Washback of the Reformed College English Test Band 4 (CET-4) in English Learning and Teaching in China, andPossibleSolutions ......................................... 35 FeifeiHan 5 FairnessinCollegeEntranceExamsinJapanandthePlanned UseofExternalTestsinEnglish ................................ 47 YukoGotoButlerandMasakazuIino 6 (Mis)UseofHigh-Stakes Standardized TestsforMultiple PurposesinCanada?ACallforanEvidence-BasedApproach toLanguageTestingandRealignmentofInstruction ............. 57 Li-ShihHuang 7 Testing in ESP: Approaches and Challenges in Aviation andMaritimeEnglish ......................................... 67 NeilBullockandCarolynWestbrook 8 AConceptualFrameworkonthePowerofLanguageTests asSocialPractice .............................................. 79 TuçeÖztürkKaratas¸andZuhalOkan ix x Contents 9 TheWashbackEffectoftheVietnamSix-LevelsofForeign LanguageProficiencyFramework(KNLNNVN):TheCase oftheEnglishProficiencyGraduationBenchmarkinVietnam ..... 99 ,, PhuongHoaÐinhThi.andHandoyoPujiWidodo 10 AvoidingScoringMalpractice:SupportingReliableScoring ofConstructed-ResponseItemsinHigh-StakesExams ............ 127 KristinaLeitnerandBenjaminKremmel 11 Score Changes with Repetition of Paper Version(s) oftheTOEFLinanArabGulfState:ANaturalExperiment ...... 147 BettyLanteigneandHanaSulieman PartII LearningfromTestsofWorldLanguages 12 WhoseEnglish(es)AreWeAssessingandbyWhom? ............. 169 LiyingCheng,Gwan-HyeokIm,andRubaiyatJabeen 13 Challenges in Developing Standardized Tests for Arabic Reading Comprehension for Secondary Education intheNetherlands ............................................. 181 AnnekedeGraaf 14 TheConflictandConsequencesofTwoAssessmentMeasures inIsrael:GlobalPISAvs.theNationalMEITZAV ................ 191 OfraInbar-LourieandElanaShohamy 15 How to Challenge Prejudice in Assessing the Productive SkillsofSpeakersofCloselyRelatedLanguages(theCase ofSlovenia) ................................................... 201 InaFerbežarandPetraLikarStanovnik PartIII LearningfromProgram-LevelLanguageTests 16 EFLPlacementTestinginJapan ................................ 223 JamesCarpenterandSawakoMatsugu 17 TEFLTestPracticesataUkrainianUniversity:Summative TestDesignThroughTeacherCollaboration ..................... 233 OlgaKvasova 18 Designing a Multilingual Large-Scale Placement Test withaFormativePerspective:ACaseStudyattheUniversity ofGrenobleAlpes ............................................. 243 CristianaCerviniandMonicaMasperi 19 TheRelationshipBetweenEnglishPlacementAssessments and an Institution: From Challenge to Innovation foranIntensiveEnglishProgramintheUSA .................... 255 NicholasSantavicca Contents xi 20 PlacementDecisionsinPrivateLanguageSchoolsinIran ......... 267 KioumarsRazavipourandTaherehFiroozi 21 Perceptions of (Un)Successful PET Results at a Private UniversityinMexico ........................................... 287 LuisAlejandroFigueroaandKrisztinaZimányi PartIV LearningfromTestsofLanguageSkills 22 CompletingtheTriangleofReadingFluencyAssessment: Accuracy,Speed,andProsody .................................. 307 JihyeShin 23 (Re)Creating Listening Source Texts for a High-Stakes Standardized English Test at a Vietnamese University: AbandoningtheSearchinVain ................................. 317 XuanMinhNgo 24 The Oral Standardized English Proficiency Test: Opportunities Provided and Challenges Overcome inanEgyptianContext ........................................ 327 DeenaBoraieandRamyShabara 25 Opening the Black Box: Exploring Automated Speaking Evaluation .................................................... 333 NahalKhabbazbashi,JingXu,andEvelinaD.Galaczi 26 Developing a Meaningful Measure of L2 Reading ComprehensionforGraduateProgramsataUSAResearch University:TheRoleofPrimaryStakeholders’Understanding oftheConstruct ............................................... 345 AhmetDursun, NicholasSwinehart, JamesMcCormick, andCatherineBaumann 27 ChallengingtheRoleofRubrics:PerspectivesfromaPrivate UniversityinLebanon ......................................... 357 ChristineSabieh 28 AMixed-MethodsApproachtoStudytheEffectsofRater Training on the Scoring Validity of Local University High-StakesWritingTestsinSpain ............................. 383 JuliaZabala-Delgado PartV LearningfromTests,Teachers,andLanguageAssessment Literacy 29 ACriticalEvaluationoftheLanguageAssessmentLiteracy ofTurkishEFLTeachers:SuggestionsforPolicyDirections ....... 411 ElçinÖlmezer-Öztürk,GökhanÖztürk,andBelginAydın
The list of books you might like
