ERIC EJ585978: A Review of the Woodcock Reading Mastery Test-Revised (WRMT-R).

8 Pages·1999·0.38 MB·English

by  ERIC

A Review of the Woodcock Reading Mastery
Test-Revised (WRMT-R)
Marybeth De Rose
The WoodcockReadingMastery Test-Revised (1997)isthelatesteditionofthe
Woodcock ReadingMastery Test, which was originallypublishedin1973.The
revisedversion,firstpublishedin1987,isnowinits12thyearofserviceand
remainsapopularchoiceamongavailablereadingtests.Theobjectiveofthis
review is to take a renewed critical look at the test in general, and in par
ticular to inquire whether test results mightbe useful in makinginferences
aboutESLlearnerreadingability.
The WRMT-R was designed to assess the reading levels of test-takers
within an age range of4 to 75 years of age and over. The manual suggests
that the WRMT-R may be used with ESL (English as a Second Language)
students, although provisions must be made for "ensuringthat the subject
understands the instructions, and providingadditional practice opportuni
ties, or reviewing the instructions" in order to "provide a valid measure of
theEnglishreadingskills" (p.18).
TheWRMT-R,likeitspredecessortheWRMT,consistsoftwoforms that
facilitate retesting at short intervals. The original test consisted offive sub
tests: Letter Identification, Word Identification, Word Attack, Word Com
prehension, and Passage Comprehension across two parallel forms (A and
B). The revised WRMT has retained the two forms, but they are no longer
parallel. Form H has been reduced to four subtests by eliminating Letter
Identification from the original, and FormG has beenexpanded to six sub
tests to improve readiness assessment. Form G consists ofthe five subtests
from the WRMT, plus an additional one takenfrom the Woodcock-Johnson
Psycho-EducationalBattery(Woodcock&Johnson,1977).Thesixsubtestsof
FormG,fourofwhichcompriseFormH,areasfollows.
First, the new subtest Visual-Auditory Learning assesses the ability to
learn26symbols and theirnames, aswellas twosymbolsfor wordendings
(-ing,-s),thenreadtheminsentenceformacrosssevenbriefteststories.
The Letter Identification subtest involves reading alphabet characters in
several fonts, including cursive writing. Supplementary lists have been
added to the revised WRMT, in upper and lower case, which also contain
severaldiphthongsanddigraphsforthepurposeofassessingsound-symbol
association. Thesetestsarenotnormedandarethereforeofdiagnosticvalue
only.
86 MARYBETHDEROSE
TheWord Identification subtestentails readingand pronouncingwords
in isolation from lists of increasing difficulty, "even though [the test taker
mayhave] hadnopreviouspersonalexperiencewiththeword." Essentially
the listed words must be decoded and pronounced in a manner consistent
withthearticulationguidelinesinordertobecountedascorrect.
The Word Attack subtest assesses phonetic decoding skills through the
reading aloud of increasingly complex nonsense words. Nonsense words
have been used in order to eliminate the possibility of test-takers knowing
howtopronouncetheminadvance,asmightbethecasewiththeoccasional
realstimulusword,nomatterhowobscure.
TheWordComprehensionsubtestconsistsofthreedistinctsections.The
first twosections entailreadingstimulus words and respondingwitheither
a synonym or an antonym, and the third section requires the test-taker to
read thestimuluswords, ascertainthe relationship betweenthem, and then
continuetheanalogytocompleteanotherstimuluspairofwords.Thevocab
ulary itemshave beencategorizedas eithergeneral reading,science-mathe
matics, social studies, or humanities. Separate category scores can be
calculatedfor diagnosticpurposes,butarenotnormed.
The last subtest, Passage Comprehension, resembles a cloze exercise,
which requires test-takers tofill inaseries ofblanks accordingto themean
ing of the surrounding sentences or phrases. Picture clues have been in
cludedfor approximatelyonethirdoftheeasiestitems.
The WRMT-R provides norms for subjects from kindergarten to college
seniorsintermsofgrade,andfor agesup to andbeyond75years.
The original normative data were gathered from 6,089 subjects in 60
geographically diverse United States communities. The kindergarten to
grade12samplecomprised4,201subjects,andthecollege/universitysample
contained 1,023 subjects. The adult sample of subjects aged 20-80 but not
enrolled in college comprised 865 subjects. The norming sample was struc
tured to resemble the distribution of variables included in the 1980 US
Census. Individual subject weighting was applied during data analysis to
obtain"exact"proportionmatchingwiththecensusdata.
The WRMT-R was renormed in 1995-1996. In Canada these norms have
justbeenmadeavailablecommerciallyinrecentmonths.
The current norms (WRMT-R NU) are based on a somewhat smaller
samplepopulationofapproximately3,700,varyingsomewhatfor eachsub
test/cluster, whose ratios are reflective of the 1994 American census. The
samplecontrolledfor age, gender, race, geographicregion,SES,parentedu
cation,andcommunitysize(adultnormsonly).A representativeproportion
of special education students were also included in the norm sample. The
new technicalchapterofthemanualstatedthatindividualsnotproficientin
Englishwerenotincludedinthenormsample;however,anoperativedefini
tionofproficiencywasnotincluded.
TESLCANADAJOURNAULAREVUETESLDUCANADA 87
VOL.16,NO.2,SPRING1999
TheWRMT-Rrenormingprocesswaspartofaprogramaimedatrenorm
ing the PIAT-R, KeyMath-R, and the K-TEA in addition to the WRMT-R.
Accordingto theupdatedWRMT-Rtechnicalchapter,thefour achievement
batteries included in the normative updating project measure a number of
achievement domains in common: word reading, reading comprehension,
mathematics computation, mathematics applications, and spelling. The Vi
sual-AuditoryLearning,WordAttack,andWordComprehensiontestsofthe
WRMT-R are not measures of any of the five shared domains; therefore,
these tests were normed individually with sample sizes controlled for
throughtheplan'stestingassignments.
The advantage of the domain-norming procedure is primarily the in
creasedcomparabilityoftestbatteriesintermsoftheircommoncross-battery
domains. Inaddition,smallersamplesizesareacceptable.
Raschscalingwasusedtocalibrateitemdifficultyofthesubtestsincluded
inthecommondomains. However,normsfortheVisual-AuditoryLearning,
Word Attack, and Word Comprehension tests were constructed from raw
scoreswithnoRaschanalysis.
Criticisms ofthe WRMT-R
From its debut in 1973 the WRMT has met with controversy. Critics were
eitherimpressedbyitspurportedprecisionandeaseofadministration(Ban
natyne, 1974;Allington, 1976),ordisappointedbyitstheoreticalframework
and questionable psychometric properties (Houck & Harris, 1976; Dwyer,
1978).
Current critical review continues to raise many of the same issues left
outstanding after the revision process. Both editions were criticized for the
Eurocentric and gender-stereotype-linked artwork (Dwyer, 1978; Jaeger,
1989).
Ofcontinuingconcernis the assessmentoffragmented skillsrather than
the reading act as a whole. The test author apparently views reading as a
collectionofsubskillsactinginconcert.Inreaction,Cooter(1989)arguesthat
"instruments like the WRMT-Rare useless and representa bygone age that
viewedthereadingactinafractured orsplinteredway" (p.910).
Jaeger (1989) and Eaves (1990) debated whether readiness skills such as
Letter Identification should be included in a reading test per se. The recent
renorming project pointed out that Letter Identification tended to correlate
positivelywithWord Recognitionscoresinallage groupsbuttheyoungest.
The lack of correlation in the early grades was attributed to a curriculum
effect, in that young childrenhave a greater portion oftheir program dedi
cated to the development of letter recognition skills as opposed to word
recognition. As well, both authors had difficulty accepting whether the
Auditory-Visual Learning subtest (part of the readiness cluster) actually
generalizes to the auditory-visualskills requiredbywrittenEnglish. Forthe
ss
MARYBETHDEROSE
purposes of the renorming project, Auditory-Visual learning was not con
sideredpartofeitherthewordreadingorreadingcomprehensiondomains.
Althoughtheauthorofthe testasserts thattheWordIdentificationitems
represent the culmination of a variety ofhigh frequency word lists, Jaeger
(1989)isoftheopinionthat"thewordscontainedinthetestrangefromthose
found in early elementary school readers to polysyllabic tongue-twisters
likely tobe familiar only to personswhose leisurehours arespentperusing
the Oxford English Dictionary [sic]" (p. 914). In addition, he states that
reading lists of words in isolation has been faulted as a useful measure of
sightvocabulary,becauseexamineescannotusesemanticcontextcuesfound
in authentic reading experiences. Tuinman (1978) adds that some subjects
willmanagetoreadanumberofthewordscorrectlyassightwords,whereas
others will rely on phonetic word attack skills. Consequently, this subtest
measuresdifferentskillswithdifferenttest-takers.Onemightanticipatethat
the person with superior decoding skills would have an advantage on this
subtest.
Both editions of the WRMT have drawn criticism for the Word Com
prehension subtest. Tuinman (1978) decided that this was more a test of
reasoning than word knowledge because of the necessity to ascertain the
relationship between the stimulus words in order to complete the analogy.
As well, Tuinman felt thatpoor readers were penalized twice because they
may miss additional items due to poor decoding skills, not necessarily be
cause they lackthe word knowledgenecessary to complete the item.Jaeger
(1989) wasunsure ofthe diagnosticandprescriptivevalueofalowscoreon
this subtest because of the number of possible constructs being measured.
This subtest was not considered part of the word reading or reading com
prehensiondomainsin therenormingproject.
Houck and Harris (1976), Dwyer (1978), and Tuinman (1978) were par
ticularly critical of the Passage Comprehension subtest because of the
modified cloze procedure. Houck and Harris (1976) revealed that the test
items do not appear to meet Bormuth's (1969) criteria for determining the
location of deletions as stated in the manual. In their opinion, "the items
resemble more closely a completion test where deletions are made using
subjectiveconceptssuchas keywords" (Houch&Harris,1976,p. 77).
Bachman (1985) concluded thatnotalldeletionsina givenclozepassage
measureexactlythesameabilities.AmongL2clozetest-takers,Turner(1989)
found that other factors in addition to knowledge in a second languagedo
contribute to successful performance on the cloze procedure. Therefore,
changing the type of deletions in any given cloze test affects the construct
validity of the test. Straying from Bormuth's (1969) criteria may have sig
nificantly altered what the Passage Comprehension subtest purports to be
measuring.
TESLCANADAJOURNAULAREVUETESLDUCANADA 89
VOL.16,NO.2,SPRING1999
Although the WRMT-Rhasbeenrenormed, itis importantto remember
thatmanyschools and institutionsmaybeslowin updatingtheir testpack
ages as a result of current funding limitations. It is, therefore, prudent to
considerthecriticismsofbothnormeditions.
Eaves (1990) and Jaeger (1989) have expressed concerns regarding the
sampleused for normative purposes in the originalWRMT-R. In particular
they had difficulty with the author's statementin the manual that selective
weightingwasusedinordertohavethesamplereflectthevariablesoutlined
inthe1980USCensus,butdidnotspecifytheprocedureorofferanydetails.
Eaves(1990)hadparticulardifficultywithhowWoodcock"indicatedthat
communities were selected according to socio-economic characteristics ina
mannerthatwouldobtainasamplematchingthe [1980]USCensusdata,yet
no evidence was provided to verify the outcome ofhis selection procedure
for communities or individuals" (p. 288). This evidence was similarly not
forthcominginthebackgroundinformationaccompanyingthenewnorms.
As the readermayrecall, theUSCensusin1980restricteditselftodistin
guishing between "white" and "non-white" ethnic groups. Only with the
1988 US Census did further details regarding ethnic background become
available. Therefore, the author used only age, socioeconomic status, and
white-non-whitevariablesinhisoriginalsample.Thecurrentnormsampleis
more inclusive with regard to race, gender, and community size (adult
normsonly).However,thereadermustbearinmindthatthe1994USCensus
was still limited to the categories White, Black, Hispanic, and other. It re
mains questionable whether thenormsampleis representative oftheCana
dianpopulationwithwhomitiswidelycompared.
MorerecentworkbyKlesmer(1994)hasfoundstrongevidencetosuggest
thattheacademic/linguisticdevelopmentofESLstudentsfollows adistinct
pattern. Atleastsix years are required for ESL students to approach native
Englishspeakers'norms ina variety ofareas, and itappears thateven after
sixyears,fullcomparabilitymaynotbeachieved.Withthisinmind,itwould
seem prudent to develop separate achievement criteria for these students
based on their age and length of residence. Klesmer's (1994) attempts to
renorm the Peabody Picture Vocabulary Test illustrated the difficulty and
complexity involved in establishing norms appropriate for individual ESL
populations.
Given Klesmer's (1994) findings, it would appear that the crude
unidimensional norms presented in the WRMT-R are hardly adequate to
account for the potential variability in performance among ESL test-takers.
Indeed, itis questionable whether the ESLpopulationis representedinany
meaningful way thatwould allow Woodcockto advocate the use ofthe test
withonlyminorprovisions.
Alsoofcurrentconcernis the validityevidencepresentedin themanual.
As with theformer edition, there is lack ofevidence ofcontentvalidity. No
90 MARYBETHDEROSE
information is offered regarding the possible match between the test and
variouscurricula,orthe "itemsand thepsychologicalprocessesinvolvedin
the readingactfor eachgrade/agelevel" (Cooter, 1989,p. 912). Themanual
suggests that the WRMT-R items were developed with the assistance of
"outside experts" and experienced teachers, but does notelaboratebeyond
this overtstatementregardingthespecificinputofthesesources. Toseveral
critics this information is far from sufficient for claiming that the WRMT-R
hasanykindofcontentvalidity(Cooter,1989;Jaeger,1989;Eaves1990).
Intermsofconcurrentvalidity(howthistestcompareswithothersinthe
field for measuring various reading skills), the authors compared the
WRMT-R only with the Woodcock-Johnson Reading Tests in the original
manual. "It seems implausible that this exercise constitutes a comparison
between independent criterion measures" (Cooter, 1989, p. 912). Jaeger
(1989)andEaves (1990)weresimilarlyunimpressedwithevidencesupplied
inthemanual.
The originalWRMT-Rmanual did present a table comparing the earlier
editionwith various other tests, arguing that it remained pertinentbecause
the "psychometric characteristics of the original WRMT (1973) and the
WRMT-Raresosimilarthatmanygeneralizationsfrom oneto theothercan
be made" (p. 100). This statementbegged the question: Why bother with a
neweditionthen?Theauthortookgreatpainsinthefirstchaptertoillustrate
thechangeseffectedfromtheoriginaltest,yethesuggestedatthatpointthat
thetwoweresosimilarthattheysharedpsychometricproperties.
By using the domain-norming approach in the renorming project,
WRMT-RconcurrentvalidityisestablishedwiththePIAT-Rand theK-TEA
insofaras thesubtests inthecommondomainareconcerned.
In response to the criticisms regardingvalidity levelled at the WRMT-R,
Eaves (1990) reported that the test's publisher,AmericanGuidance Service,
made available intercorrelations between the WRMT-R Form G and the
K-ABC (Kaufman AssessmentBatteryfor Children) and WISC-R (Wechsler
Intelligence Scale for Children-Revised) intelligence scales. In both cases
correlationswerefairlyhigh(range.75to.96).Eaves(1990)alsoreportedthat
the studies were without date. One might surmise that in making these
particular intercorrelations available, the publisher was making a brilliant
marketing move. Both the WISC and the K-ABC are widely used ineduca
tionandpsychology. ByaligningtheWRMT-Rwiththesepopulartools,the
publisher implies that they measure the same constructs. This makes the
WRMT-R especially attractive for use in diagnosing a variety of learning
difficultiesthatentailtheuseofanachievementmeasure. However,without
adateonthestudy,onehastoquestionthe credibilityofthestatistics.
Curiously,itmustbenotedthatseveralrecentstudiesusedtovalidatethe
WISC-III(WechslerIntelligenceScaleforChildren-ThirdEdition)involved
the WRMT-R, to establish credibility (Hishinuma & Yamakawa, 1993;
TESLCANADAJOURNAULAREVUETESLDUCANADA 91
VOL.16,NO.2,SPRING1999
Newby, Recht, Caldwell, & Schaefer, 1993; Schwean, Saklofske, Yakulic, &
Quinn, 1993). Itwould appear that the validation arrangement is mutually
beneficial.
The K-ABC and the Wechsler tests are not reading tests per se. They
purport to measure intelligence; therefore, the only conclusion that can be
drawnfromtheaboveinformationisthattheK-ABC,WISC-R,andWISC-III
maymeasurethesameundefinedconstructsasdoestheWRMT-RFormG.
The construct validity of the WRMT-R is not discussed in the original
manual. The renorming project, in definingthe domains and selectingsub
tests appropriate for those domains, effectively establishes the constructs
beingmeasured. However,threeoftheWRMT-Rsubtestsdidnotfallwithin
the confines ofthe definitions ofword readingand readingcomprehension
and therefore are measuring other, undefined constructs. The paucity of
validity evidence should again raise questions regarding whether the test
shouldbeusedasthebasisfordecision-makingofanykind.
WithregardtoESLpopulations,itshouldbeapparentthatreadingskills
might not be effectively assessed using this tool. The original manual does
notdefine"Englishreadingskills";therefore,anynumberofconstructsmay
bemeasuredbythis instrument.
Letter Identification might be of some use with persons whose L1 is
encoded with characters other than those of English; however, this skill
should not be interpreted as reading. The Word Identification subtest may
notbeavalidmeasureofacquiredEnglishsightwords iftheindividualhas
developedsufficientphoneticdecodingskills.WordComprehensionmayat
best define some sense of the test-taker's knowledge of synonyms and an
tonyms. However, becausewhattheanalogiessubtestis measuring(decod
ing?verbalreasoning?wordknowledge?)isunclear,itmaybedangerousto
ascribe the construct word comprehension to this subtest. The vocabulary
chosenforthetestasawholemaynotrealisticallyreflecteverydaylanguage
at a variety of age levels or lengths of residence. Last, the Passage Com
prehension subtest may not be measuring true comprehension as much as
the ability to fill in a blankadequately. Itwould appear that the US sample
used to establishnorms for the WRMT-Rmaynotadequately represent the
Canadian and/or North American newcomer; therefore, an ESL test-taker
may not be adequately represented in the norm population. Essentially,
undertheseconditions,theESLtest-taker'sperformancewouldbecompared
withamajoritywhiteAmericannorm. Thisishardlyfairtestingprocedure.
Toconclude, the Woodcock ReadingMaster Test-Revised is "unsafeatany
speed,"especiallyfor ESLtest-takers.
92 MARYBETHDEROSE
The Author
MarybethDeRoseisan18-yearveteranofOntarioclassrooms;forthepastdecadeshehasbeen
aspecialeducationteacherinapredominantlyESLcommunity.Sheiscurrentlycompletingthe
finalyearofherPhDprogramattheOntarioInstituteforStudiesinEducation.
References
Allington,RL.(1976).Woodcockreadingmasterytestsreview.JournalofReading,20,162-163.
Bachman,L.F.(1985).Performanceonclozetestswithfixed-ratioandrationaldeletions.
TESOLQuarterly,19,535-556.
Bannatyne,A.(1974).Woodcockreadingmasterytestsreview.JournalofLearningDisabilities,7,
398-9.
Bormuth,J.(1969).Factorvalidityforclozetestsasmeasuresofreadingcomprehensionability.
ReadingResearchQuarterly,4,358-365.
Cooter,R (1989).ReviewoftheWoodcockReadingMasteryTests-Revised.InJ.CConoley&J.J.
Kramer(Eds.),The10thmentalmeasurementsyearbook(pp.910-912).HighlandPark,NJ:
Gryphon.
Dwyer,CA.(1978).WoodcockReadingMasterytestsreview.InO.K.Buros(Ed.),The8th
mentalmeasurementsyearbook(pp.1303-1305).HighlandPark,NJ:Gryphon.
Eaves,RC(1990).WoodcockReadingMasteryTests-revised(WRMTR).Diagnostique,15,
277-297.
Hishinuma,E.S.,&Yamakawa,R (1993).Constructandcriterionrelatedvalidityofthe
WISC-IIIforexceptionalstudentsatrisk.JournalofPsychoeducationalAssessment:Advances
inPsychoeducationalAssessmentMonographSeries,94-104.
Houck,C,&Harris,L.(1976).WoodcockReadingMasteryTestsreview.JournalofSchool
Psychology,14(1),77-79.
Jaeger,RM.(1989).ReviewoftheWoodcockReadingMasteryTests-revised.InJ.CConoley
&J.J.Kramer(Eds.),The10thmentalmeasurementsyearbook(pp.913-916).HighlandPark,
NJ:Gryphon.
Klesmer,H.(1994).AssessmentandteacherperceptionsofESLstudentachievement.English
Quarterly,26(3),8-11.
Newby,RF.,Recht,D.R,Caldwell,J.,&Schaefer,J.(1993).ComparisonofWISC-IIIand
WISC-RIQchangesoveratwo-yeartimespaninasampleofchildrenwithdyslexia.
JournalofPsychoeducationalAssessment:AdvancesinPsychoeducationalAssessmentMonograph
Series,87-93.
Schwean,V.L.,Saklofske,D.H.,Yakulic,RA.,Quinn,D.(1993).WISC-IIIperformanceof
ADHDchildren.JournalofPsychoeducationalAssessment:AdvancesinPsychoeducational
AssessmentMonographSeries,56-60.
Tuinman,J.J.(1978).Woodcockreadingmasterytestsreview.InO.K.Buros(Ed.),The8th
mentalmeasurementsyearbook(pp.1306-1308).HighlandPark,NJ:Gryphon.
Turner,CB.(1989).TheunderlyingfactorstructureofL2clozetestperformancein
Francophone,university-levelstudents:Causalmodelingasanapproachtoconstruct
validation.LanguageTesting,6,172-197.
Woodcock,RN.(1973).WoodcockReadingMasteryTests.CirclePines,MN:AmericanGuidance
Service.
Woodcock,RN.(1987).WoodcockReadingMasteryTests-revisedexaminer'smanual.Circle
Pines,MN:AmericanGuidanceService.
Woodcock,RN.(1997).WoodcockReadingMasteryTests-revised/normativeupdate.Circle
Pines,MN:AmericanGuidanceService.
Woodcock,R,&Johnson,M.(1977).Woodcock-JohnsonPsychologicalBattery.Allen,TX:DLM
TeachingResources.
TESLCANADAJOURNAULAREVUETESLDUCANADA 93
VOL.16,NO.2,SPRING1999