ERIC EJ1128073: The Influence of Peer Assessment and the Use of Corpus for the Development of Speaking Skills in In-Service Teachers

2016·0.22 MB·English

by  ERIC

http://dx.doi.org/10.19183/how.23.1.142
TheInfluenceofPeerAssessmentandtheUseof
CorpusfortheDevelopmentofSpeakingSkillsin
In-ServiceTeachers
Lainfluenciadelaco-evaluaciónyuncorpuseneldesarrollo
delahabilidadoraldedocentesenejercicio*
Mary MilyGómez Sará
[email protected]
UniversidaddeLaSabana,Bogotá,Colombia
Thisarticlereportsonhowtheuseofpeer-assessmentandacorpusinfluencethedevelopmentof
thespontaneousinteractivespeakingof14adultswithanA1Englishlevel.Thedata,thatwerecollected
through video recordings, two peer-assessment forms, and a teacher’s journal, evidenced the
developmentofthreeenhancementstrategies(willingnesstoimprove,useofcompensatorystrategies,
andconstructionofapersonalizedversionofthecorpus)andtwodetrimentaltraits(underassessment
anddependencyonthecorpus).Theresultsoftheinquiryevincedsomelimitationsinthepedagogical
intervention.
Keywords:Corpus,in-serviceteachers,peerassessment,spontaneousinteractivespeaking.
Esteartículoinformasobrecómoelusodelaevaluaciónporparesyuncorpusinfluyeneneldesa-
rrollodelhablaespontáneainteractivade14adultosconnivelA1deinglés.Losdatos,quefueronreco-
gidos a través de videograbaciones, dos formatos de evaluación por pares y un diario del docente,
evidenciaroneldesarrollodetresestrategiasdemejoramiento(voluntaddemejorar,usodeestrategias
compensatoriasyconstruccióndeunaversiónpersonalizadadelcorpus)ydosrasgosperjudiciales(in-
* Received:July17,2015.Accepted:February22,2016.
Howtocitethisarticle(APA6thed.):
GómezSará,M.M.(2016).Theinfluenceofpeerassessmentandtheuseofcorpusforthedevelopmentof
speakingskillsinin-serviceteachers.HOW,23(1),103-128.http://dx.doi.org/10.19183/how.23.1.142.
This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
InternationalLicense.LicenseDeedcanbeconsultedathttp://creativecommons.org/licenses/by-nc-nd/4.0/.
HOWVol.23,No.1,April/September2016,ISSN0120-5927.Bogotá,Colombia.Pages:103-128 103
MaryMilyGómezSará
fravaloraciónydependenciaalcorpus).Losresultadosdelainvestigaciónevidenciaronalgunaslimita-
cionesenlaintervenciónpedagógica.
Palabrasclave:corpus,docentesenejercicio,evaluaciónporpares,hablaespontáneainteractiva.
Introduction
This paper describes a research study conducted with a group of in-service school
teachers in different content areas who were studying English to enrich their professional
performance. The research question that guided the development of the study was: “How
mighttheuseoftwopeerassessmentstrategiesandacorpusinfluencethedevelopmentof
thespontaneousinteractivespeakingofagroupof14adultswithanA1Englishlevel?”Inthe
subsequentpages,thereaderwillfindtheprocessthattheteacher-researcherwentthrough
whenundertakingandconductingthestudy.
Identification of Participants’Learning Needs
Identifyingthelearningneeds ofparticipantswasrelevantinorder tosettheirlearning
objectivesanddecideonanactionplan.Twoquestionnairesandafocusgroupwereusedto
exploreparticipants’linguistics,affective,andcognitiveneedsasfollows:
(cid:129) ThroughQuestionnaire1(Appendix1)andfocusgroup(Appendix2),participants
reportedthattheyhaddifficultiesproducingorallanguage,especiallyinspontaneous
situations that entail interacting with others. This was interpreted as their linguistic
need.
(cid:129) Through the Questionnaire 2 (Appendix 3), participants reported that comforting
students whowerehavingproblemsorweregoingthroughdifficultcircumstances,
was the most common communicative situation that they faced in their work as
schoolteachers when interacting with their students outside the classroom context.
Thissuggestedthattheiraffectiveneedhadtodowithestablishingrapportwiththeir
studentsbysupportingthem.
(cid:129) Through Questionnaire 1, participants also reported that the class time was not
enoughforpracticingandimprovingtheirEnglish.Thissuggestedthattheircognitive
needhadtodowithdevelopingautonomouslearningskillsthatwouldallowthemto
extendtheirlanguagepracticebeyondtheface-to-faceclasses.
Strategies Proposed to Address Participants’ Needs
To attend to participants’ cognitive need, two peer-assessment strategies (checklist
[Appendix4]andPlus,minus,andwhat’snext?[Appendix5])wereselectedinordertofoster
autonomouslearning.
104 HOW
TheInfluenceofPeerAssessmentandtheUseofCorpus
fortheDevelopmentofSpeakingSkillsinIn-ServiceTeachers
Peerassessmentchecklist.Thepeerassessment(PA)checklistwasusedtoencourage
participants to reflect on their strengths and weaknesses in speaking by contrasting their
productionswithcriteriaofsatisfactoryperformance.Thus,thestrategywasintendedtohelp
participantsraiseawarenessoftheirEnglishlanguagelearningprocess,whichisnecessaryto
fosterautonomouslearning.ThePAchecklistwasalsousedasadatacollectioninstrument.
Plus,minus,andwhat’snext?The“Plus,minus,andwhat’snext?”(PMWN)wasused
to encourage participants to reflect on their strengths, weaknesses, and improvement
opportunitiesusingtheirownwordsinsteadofpre-establishedcriteriaasinthechecklist.The
strategywasintendedforparticipantstobecomemorecriticalwhenreflectingontheiroral
performance,whichisalsonecessarytofosterautonomouslearning.AccordingtoGlasson
(2009), the PMWN is a strategy in which learners reflect on what was done well (plus) and
wrong (minus) in regard to the development of a particular task. Then, they generate a
personallearningtarget(what’snext?).Forthepurposeofthepresentstudy,participantshad
to write their perceptions about their peers’ spoken strengths and weaknesses when
developingspeakingtasks.Furthermore,insteadofproducingapersonallearningtarget,they
hadtowriteimprovementpiecesofadvicetotheirclassmates.ThePMWNwasalsousedasa
datacollectioninstrument.
Corpus.Toattend toparticipants’linguisticandaffective needs theteacher-researcher
selectedtoteachparticipantsacorpus(Appendix6)withthemostcommonexpressionsused
to comfort someone. According to Bennett (2010) and Suzuki (2008, 2009a, 2009b, 2010,
2011),a“corpus”isatoolthatinformsthefrequencyofuseofwordsand/orcombinationof
words. Thornbury (2008), Moudraia (2001), and Suzuki (2008, 2009a, 2009b, 2010, 2011)
believe that learning prefabricated multi-word units (also known as chunks) help learners
produce more fluent and accurate language because they do not have to think about
sentenceswordbyword,butinlongermeaningfulunits.AccordingtoMoudraia(2001),“the
lexical approach to second language teaching [which consists of teaching prefabricated
multi-wordunitstoL2learners]hasreceivedinterestinrecentyears”(p.1),soitdeservesto
bestudiedinordertoexploreitsinfluenceandfeatures.Moreover,teachingparticipantsto
comforttheirstudentsinEnglishwouldenrichtheirprofessionalperformance.
LiteratureReview
Peer Assessment for Developing Oral Skills
Theteacher-researchercontrastedthreestudiesinwhichPAtofosterthedevelopmentof
speakingskillsinEnglishlanguagelearnerswasused.ThefirststudywasconductedinIran
(Ahangari,Rassekh-Alqo,&Akbari,2013),thesecondonewascarriedoutinSpain(Serrano
&CebriándelaSerna,2011),andthelastonewasdevelopedinColombia(Gómez,2014).As
HOWVol.23,No.1,April/September2016,ISSN0120-5927.Bogotá,Colombia.Pages:103-128 105
MaryMilyGómezSará
aresultofcontrastingthestudies,theteacherresearchercouldhaveanideaofwhathavebeen
donearoundtheglobeinregard totheuseofPAtofosterspeakingdevelopment.Table1
summarizesthefindingsofthethreestudiesintermsofpositiveeffects,negativeeffects,and
otherfindingsandrecommendations.
Table1.PeerAssessmentforDevelopingOralSkills
PositiveEffectsofPA NegativeEffectsofPA
(cid:129)It fosters involvement, responsibility, and (cid:129)Studentsfoundevaluatingtheirpeers’speakingand
commitmentinstudents. learningabilitiesdifficult.
(cid:129)Students raiseawarenessoftheirlearningprocess (cid:129)Underandoverassessmentwerefrequent.
andskills. (cid:129)It requires the support of other techniques to
(cid:129)Studentsbecomeabletoassesstheirpeerssimilarto encouragereflection.
theirteachers.
(cid:129)Itfostersautonomyandcriticalthinking.
(cid:129)Learnerslikedandenjoyedit.
(cid:129)Itpositivelyinfluencesself-assessment.
(cid:129)Students praise and reinforce their peers’ good
performance.
OtherFindingsandRecommendations
(cid:129)TheintermediatelevelofstudentswasfoundtobeanadvantagewhenapplyingPA.
(cid:129)Discussionandreachingagreementsontheassessmentcriteriaarerecommendedtohelplearnersinternalize
thecriteria.
Teaching Through Prefabricated Chunks
ChuandWang(2011)reviewtherolethatthelexicalchunkmethod,whichconsistsof
teachinglearnersprefabricated lexicalchunks,hashadonthedevelopmentoftheoraland
written competences of Chinese students who learn English as a foreign language. Their
review reveals that the method contributes to the enhancement of learners’ pragmatic
competence, helps students understand the discourse structures and speech rules, and
promotesfluencyandaccuracyinoralandwrittenEnglish.However,ChuandWang(2011)
pointed out that a disadvantage isthat chunksare learned as unanalyzed units that are not
availabletobecombinedwithotherstructuresorpartsandthislimitstheiruse.
ResearchDesign
The study followed the qualitative approach, which according to Creswell (2009) and
McMillanandSchumacher(2009)ischaracterizedbyin-settingparticipationandobservation,
andhighlydescriptivedata.Inaddition,theinquiryusedtheactionresearchmethod,which
106 HOW
TheInfluenceofPeerAssessmentandtheUseofCorpus
fortheDevelopmentofSpeakingSkillsinIn-ServiceTeachers
according to Koshy (2005), Lim (2007), Sagor (2000), and Valcarcel (2009), consists of
in-service teachers studying class situations in order to improve their pedagogical practice.
Hence,teachersassumeadoubleroleasteachersandresearchers.
Participants and Context
This study was conducted with 14 adults, from 26 to 50 years old, who worked as
schoolteachersindifferentcontentareas(Christianeducation,mathematics,music,physics,
Spanish,socialscience,accounting,biology,andpre-schooleducation)ataprivatenonprofit
school located in the northeast of Bogota, Colombia. Participants’ English level was A1
accordingtotheCommonEuropeanFrameworkofReference(CEFR)(CouncilofEurope,
2014).Inordertobecomeabilingualinstitution,theschoolprovidedtwohoursofEnglish
training per week for the teaching staff. The 14 teacher-students signed a consent form
throughwhichtheyauthorizedtheirparticipationintheresearchstudy.
Data Collection Instruments
Datawerecollectedthroughfourinstruments,namely,videorecordings(Appendix7),PA
checklist(Appendix4),PMWN(Appendix5),andateacher-researcher’sjournal(Appendix8).
The video recordings were used to gather data on participants’ speaking behaviors, the PA
checklistandPMWNwereusedtocollectdataonparticipants’perceptionstowardstheirpeers’
spoken productions, and the journal was used to collect data on the teacher-researcher’s
perceptions on the participants’ responses to the pedagogical implementation. Immediately
aftereachsession,theteacher-researcherwroteanentryinthejournalinordertoguaranteethat
thememorieswerestillfresh.
PedagogicalIntervention
Thepedagogicalinterventionwascarriedoutin11lessonsoftwohourseach(22hours).
Itwasdevelopedintwostages,namely,training(sixhoursdividedintothreesessions)andmain
implementation (16 hours divided into eight sessions). In the training stage, participants got
acquaintedwiththefeatures,scope,aims,andmethodologyoftheresearchstudy.Also,the
PAformsandthecorpuswereintroducedandsomepracticeonhowtousethemwascarried
out.Then,themainimplementationstagewasundertakenusingtask-basedinstruction(Willis
& Willis, 2012) which consists of learners using authentic language to do meaningful
situationaltasks.
Thus,atthebeginningofeachlesson,participantsdidactivitiestoappropriatethecorpus
of69prefabricatedchunks,whichwasareconstructionofSuzuki’s(2008,2010)comforting
corpus.Theseactivitiesservedthemaspreparationforsubsequenttasksinwhichtheyhadto
produceSISinrole-playsaboutcomfortingsituationssuchasbreakup,difficultsituations,
HOWVol.23,No.1,April/September2016,ISSN0120-5927.Bogotá,Colombia.Pages:103-128 107
MaryMilyGómezSará
unfavorable events, sickness or injury, failure on a test, accident, or death of a loved one,
whichSuzuki(2008,2010)recognizedasthemostcommoncomfortingsituationsoccurring
intheeducationalenvironment.
DataAnalysisandResults
Datawereanalyzedusingthegroundedtheoryapproach,whichconsistsofcodingdatain
three consecutive steps, namely, open, axial, and selective (Corbin & Strauss, 2008). In open
coding, data were extracted from the instruments, contrasted, and triangulated. From this
procedure,theteacher-researcherobtainedalistofrecurrenttopics(patterns).Then,inthe
axialcodingthepatternsweregroupedobtainingcategoriesandsubcategories.Finally,inthe
selectivecoding,thecategoriesweregroupedandrefinedinordertoobtainacorecategory
thatanswereddirectlytheresearchquestionandpredictedhowPAandcorpuswouldoperate
indiversecontexts.Table2showsthestepscarriedoutinthedataanalysisprocess.
Table2.EmergingPatterns,SubcategoriesandCategories
EmergingandRecurrentTopics
Subcategories Categories Corecategory
(Patterns)
(cid:129)Recognitionofspeaking
strengthsandweaknesses
(cid:129)Identificationof
Willingnessto
pronunciation,accuracy,
improve
fluency,andlinking
wordsasweakness
(cid:129)Awarenessraising
Appearanceof
(cid:129)Complywiththe
Patternsthat Developmentof strategiestoenhance
interaction,volume,
improved Strategiesto SISaswellastraits
speakingtime,variety,
participants’SIS andorganizationcriteria Useof EnhanceSIS thatlimitthe
compensatory developmentofSIS
(cid:129)Useofcompensatory
strategies
strategiestointeract
(cid:129)Transferenceofabilities
fromL1toL2
(cid:129)Preferencesand Constructionofa
personalizationofthe personalizedversion
corpus ofthecorpus
108 HOW
TheInfluenceofPeerAssessmentandtheUseofCorpus
fortheDevelopmentofSpeakingSkillsinIn-ServiceTeachers
EmergingandRecurrentTopics
Subcategories Categories Corecategory
(Patterns)
(cid:129)Underassessment
(cid:129)Reluctanceand
difficultiestogivepieces
ofadvice Appearanceof
Patternsthat Underassessment strategiestoenhance
(cid:129)Feartothreatenpeers Emergenceof
limited SISaswellastraits
participants’SIS (cid:129)Lackofcriticalthinking DetrimentalTraits thatlimitthe
skillsandawarenessof developmentofSIS
thelearningprocess
(cid:129)Dependencyonthe Dependencyonthe
corpus corpus
Throughtheopenandaxialcoding,theteacher-researcherdiscoveredthattheuseof
PA and a corpus produced two main effects on participants’ SIS. On the one hand, it
encouragedparticipantstodevelopstrategiestoenhancetheirSISproductions(Category
1). These strategies were called willingness to improve (Subcategory 1.1), use of compensatory
strategies (Subcategory 1.2), and construction of a personalized version of the corpus (Subcategory
1.3).Ontheotherhand,itprovokedtheemergenceoftwodetrimentaltraits(Category2)
thatwerenamedunderassessment(Subcategory2.1)anddependencyonthecorpus(Subcategory
2.2).Subsequently,thecategoriesandsubcategoriesareexplainedindetailandsupported
throughexcerptstakenfromtheinstruments.
Category 1: Development of Strategies to Enhance SIS
TheuseofPAandcorpusencouragedparticipantstodevelopthreestrategiestoimprove
theirSISperformance.
Subcategory 1.1: Improvement willingness. The teacher-researcher observed a
change in participants’ assessment patterns by the end of the implementation process.
Thischangehadtodowiththeemergenceofwillingnesstoimprovethepeerassessment
practice.Bytheendoftheimplementation,participantsshowedanemergentmotivation
to express their ideas, produce more objective and less emotional assessment, identify
their peers’ mistakes and difficulties, and provide specific reasons to support their
assessment.
At the beginning of the implementation process, participants were reluctant to
express their opinion, especially through the “observations” column of the checklist.
Nonetheless,attheendoftheimplementation,theyshowedanemergentdesiretowrite
HOWVol.23,No.1,April/September2016,ISSN0120-5927.Bogotá,Colombia.Pages:103-128 109
MaryMilyGómezSará
their opinions. Therefore, they moved from no comment in the checklists to at least two
comments,whichareshowninExcerpt1below:1
Excerpt1.EmergenceofParticipants’Comments
S10andS9:verygood!
S4:canbebetter.(Checklist)
Although these comments do not evidence deep reflection, they show participants’
willingnesstoreflectontheirpeers’performances.InthePMWN,theteacher-researcheralso
observedachangeinparticipants’productionofcomments,whichevidencedthattheywere
able to recognize their peers’ mistakes and difficulties. Excerpt 2 shows examples of
participantsidentifyingspecificproblemsintheirpeers’performance:2
Excerpt2.IdentificationofPeers’MistakesandDifficulties
Don’tclearthesituation.
Inmyopinionheshouldcalmwhenhasaconversation.
Theconversationbreakdown.(PMWN)
There were also positive comments that showed that participants identified good
performances,too.Excerpt3evidencesthis:
Excerpt3.IdentificationofPeers’GoodPerformances
Thetimewasapropiate.
Talkwithgoodvolume.
Shehaveagoodpronunciation.(PMWN)
In the PMWN, there were even some comments in which participants did not only
establishiftheperformancewasgoodorbad,buttheywentbeyondbyprovidingreasonsthat
explaintheirjudgment.Excerpt4showsexamplesofthis:
Excerpt4.EmergentArgumentation
Relevance:Isnecessarythatthesituationispunctual.
Organization:Thesentencessheusedhasclearandlogical.
Variety: He use a variety of vocabulary. He used pre-fabricated sentences in the performance.
(PMWN)
Inthefirstone,theparticipantdidnotconsiderhispeers’utterancesrelevantbecausethe
situation that framed the conversation was not well established. In the second one, the
1 ThenamesoftheparticipantswerereplacedbySandanumber.
2 Excerptswerewrittenasparticipantsproducedthem,sotheycontainmistakes.
110 HOW
TheInfluenceofPeerAssessmentandtheUseofCorpus
fortheDevelopmentofSpeakingSkillsinIn-ServiceTeachers
participant explained that the speech of the classmate approved the organization criterion
becauseitwasclearandorderedinalogicalway.Inthelastone,theparticipantarguedthat
the criterion of variety was accomplished because the speaker incorporated various
pre-fabricated sentences to enrich his speech, the use of the pre-fabricated sentences was
confirmedinthevideorecording.
Subcategory1.2:Useofcompensatorystrategies.Theteacher-researchernoticedthat
participantscouldcommunicatedespitethelanguagelimitationsassociatedwiththeircurrent
languagelevel.Theteacher-researcherreflectedonthisphenomenoninthejournalascanbe
seeninExcerpt5:
Excerpt5.UseofStrategiestoEnsureInteraction
Animportantaspectoftheirspokenproductionswasthatparticipantswereabletointeractinthe
conversation.Mostofthemwereabletoadapttheirspeechtotheemergingunexpectedsituations
suchaswhenthepartnerdidnotproducetheexactsentencethatwasplannedorwhenaclassmate
fromtheaudiencemadeajoke.(Teacher-researcher’sjournal)
Theteacher-researcherfoundthattoovercomeemergingchallengesincommunication,
participants used what Thornbury (2008) called compensatory strategies. According to this
author, compensatory strategies are actions that speakers undertake in order to ensure the
transmission of a message and maintain the interaction when they do not knowor do not
remembertheexactgrammarthattheyneed.Inthisinquiry,participantsusedsomeofthe
compensatory strategies described by Thornbury. Furthermore, the teacher-researcher
observedthatparticipantsusedotherstrategiestoavoidcommunicationbreakdown,soshe
includedtheminthelistofcompensatorystrategies.Table3showsthenumberoftimesin
which each compensatory strategy was used during the intervention. The ones with white
background are those described by Thornbury while the ones with grey background
correspond to other compensatory strategies that participants used in response to the
implementation.
TheuseofthecompensatorystrategiesallowedparticipantstoachievecommunicationinL2
byovercomingtheirlimitationsintheuseofthelanguage.ThiscanbeseeninExcerpts6and7:
Excerpt6.UseofApproximationinL2
S14:Oh,dearGod!Ilostfivesubjects.Ilostmath,IlostEnglish,Ilostchemistry,Ilostbiologyand
religion.(Video-recordingtranscription)
Excerpt6showsS14’suseoftheapproximationstrategy.Inthisexcerpt,S14produced
thewordlostinsteadoffailwhichwouldhavebeenmoreappropriateforthiscontext.Both
words were similar in meaning which enabled the communication to occur. Excerpt 7
evidences acombinationofthree compensatory strategies, namely, omission(three spaced
dots),avoidance(italics),andapproximationinL2(underline):
HOWVol.23,No.1,April/September2016,ISSN0120-5927.Bogotá,Colombia.Pages:103-128 111
MaryMilyGómezSará
Excerpt7.UseofOmission,AvoidanceandApproximationinL2
S7:Hewas...sincealongtime…Ihave10years…IrememberwhenIwasachildIhaveadogandnowhe.
..death.(Video-recordingtranscription)
Table3.UseofCompensatoryStrategies
ApproximationinL2 12
Translation 4
Omission 3
Appealingforhelp 3
ApproximationtoanL1expression 2
Wordcoinage 1
Avoidance 1
Paralinguistics 1
Foreignizing 0
All-purposewords 0
Circumlocution 0
Total 27
Note:Takenfromtheanalysisofthevideorecordings.
In Excerpt 7, S7 started conveying a message omitting the unknown words. Later, he
decided to reword the message which entailed the use of the avoidance strategy. Next, S7
approximatedtheverbhavethatshouldhavebeenconjugatedinpast.Finally,S7producedthe
worddeathtoapproximatetheappropriatewordingthatshouldhavebeenisdead.
Subcategory 1.3: Construction of a personalized version of the corpus. The
teacher-researcher found that participants modified some expressions of the corpus by
removingwords(thisphenomenonwascalled“simplification”).Participantsalsoadded
onecomfortingexpressiontothecorpus(thisphenomenonwascalled“addition”).Table
4 shows the new and simplified expressions that participants used in their spoken
productions.
112 HOW