Report on Development of USES Specific Aptitude Test - Eric

30 Pages·2011·1.13 MB·English

DOCUMENT RESUME
TM 004 350
ED 103 489
Technical
Drafter, Civil (Profess. & Kin.) 005.281.
TITLF
Aptitude Test
Report on Development of USES Specific
Battery.
Manpower Administration (DOL), Washington, D.C. U.S.
INSTITUTION
Training and Employment Service.
S-266R74
REPOFT NO
Oct 74
PUB DATE
NOTE
39p.
HC-$1.95 PLUS POSTAGE
MF-$0.76
EDRS PRICE
*Aptitude Tests; Criteria; *Cutting Scores;
DESCRIPTORS
*Draftsmen; Employment Qualifications; Evaluation
Criteria; Job Applicants; *Job Skills; Job Training;
Evaluation;
Norms; Occupational Guidance; Personnel
*Personnel Selection; Test Reliability
GATB; *General Aptitude Test Battery
IDENTIFIERS
ABSTRACT
Service
The United states Training and Employment
published in 1947, has
General Aptitude Test Battery (GATB), first
validate the
been included in a continuing program of research to
occupations. The GATB
tests against success in many different
General Learning
consists of 12 tests which measure nine aptitudes:
Spatial Aptitude; Form
Ability; Verbal Aptitude; Numerical Aptitude;
Finger
Perception; Clerical Perception; Motor Coordination;
standard
Dexterity; and Manual Dexterity. The aptitude scores are
for the general working population,
scores with 100 as the average
Occupational norms are established in
and a standard deviation of 20.
for each of the significant
terms of minimum qualifying scores
job performance.
aptitude measures which, when combined, predict
aptitudes which aid in
Cutting scores are set only for those
the experimental
predicting the performance of the job duties of
only for jobs with
sample. The GATB norms described are appropriate
description presented in
content similar to that shown in the job
validation sample is also included.
this report. A description of the
(RC)
S-266R74 (In
so
MM US"
ton
4".
October 1974
U.S. Employment Service
Technical Report
S-266R74
u S. DEPARTMENT OIL HE AL Tel
EDUCATION A WELFARE
NATIONAL INSTITUTE OF
EDUCATION
loi%t
T% kTPuO
:4'S
1
D1DrKA
.1 LI
NOV
i:,"0%
F nF 4. ^N
GIN
Jlr Ok LIgN ON
;.
V f
A T -4(.
111
NI.'
, ?ATI tt L
, V WI PWT
I
.114
s
;Ns! !((!t of
.0 A. %A.
..E N U/ I
( V
r t.% t-
I 0.
AT
PC.
Development of USES
APTITUDE TEST
BATTERY FOR
DRAFTER
O LABOR
-; OEPART
amtnicTr ATIOn
2
a
CO
Technical Report on Development of USES
1101
Specific Aptitude Test Battery
Cgif
SE
O
re\
For
r--4
Drafter, Civil (profess. & kin.) 005.281
W
C.)
Drafter, Geological (petrol. production) 010.281
Drafter, Mechanical (profess. & kin.) 007.281
Drafter, Structural (profess. & kin.) 005.281
S-266R74
Developed in Cooperation with the
Alabama, California, Connecticut, Georgia,
Illinois, Michigan, Nevada, New Jersey
and New York State Employment Services
U. S. DEPARTMENT OF LABOR
Peter J. Brennan, Secretary
Manpower Administration
William H. Kolberg
Assistant Secretary for Manpower
October 1974
nevelopment of PSES Soecific Aptitude Test Pattery S-2nro74
For
BEST COPY. AMAMI
Drafter, Civil (nrofess. i kin.) DDr.//1
Drafter, Cooler:If:al (petrnl. production) D7.^./P1
Drafter, 'lechanical (Profess. 5 !:in.) Dn'."1
Drafter, Structural (profess. 5 kin.) DD.'111
IFIEADN SPfq"!""(
This renort descrih:%s the research uhtch resulted in the develop-
for use In
lent of the follov,In- Snecific .1.ptitude Test Pattery
selectin!.. inexperienced or untrained individuals for training as
Drafters:
ruttipr. Scores
Artttiv'es
G - General Learning Phility
15
'PO
Antlturle
-
Inn
spatial Aptitude
F
on
1 - Clerical Perception
Salplq:
32C Civil, Geological, flechanical am! Struc-
ValldatiQn sample:
orth, South
from the
tural (rafters (/Dr males anA 7n females)
(4" Blacs
A total of 1/5 were minority group
and Vest.
Ariertcan Indians) and 7"1
ln Spanish Surnamed, 1^ nrtentals and
7,
were nonminority group menhers.
35 "echonical and Structural ('rafters
Cross-validltion saqinle:
A total of 1r were mi-
(3h -Isles anA 1 female) from the "Iorth.
1 oriental an4 1 !Iponish surnamed)
nority group neml)ers (1/ Placks,
anr" In were nonninority g,roun menhers.
CriteriQn:
Criterion data were collected during the
Supervisory ratings.
neriod In(Yr. through 1'73 for the valriation sample and during
1D73 fnr the cross-vat lrfation sample.
Concurrent (test anH criterion data were collected at aporoxi-
mately the sane time).
4
BEST COPY AVAILABLE
rouprren;
Validation Samnle:
Phi coefficient for total sample = .37 (P/2 <
.1(1(15)
Pt:i coefficient for Black subsample = .32 (P/2 <.A25)
Phi coefficient for Spanish Surnamed subsample = .4R (P/2 < .nos)
Phi coefficient for Oriental subsample = .13 (P/2 < .25)
Phi coefficient for nonminority subsample = .34 (P/2 < Jinn)
.37 (Pr! < ennns)
Phi coefficient for male subsanpie ra
Phi coefficient cor female subsample = .?1 (P /7 < sin)
Cross-valiAation Sample:
Phi coefficient cm- total sample = .53 (P/2 < .nn5)
LS.:tiMgiltra5Q.LBatteLY_i2Laff.QSALIA.11611.1.2a-1112141:
For the total vali4ation sample,
FrAl' of the nontest-selected
in,qviduals in this study were in the high criterion group; if
they had been test-selected, 77% woul' have been in the higi:
34' of the nontest-selected individuals were in
criterion -roue.
the low criterion group; if they had been test-selected 231 would
The effectiveness of the
have heen in the low criterion groun.
battery is shown in Table 1.
TABLE I
Effectiveness of Battery for Total Validation Sample
10.11MLLtatl With Tests
High Criterion
77;
Group
Low Criterion
235;
Group
Orou s for the Validation
Comnarison of iinorit
and 'Ionninorit
amnl
:
The dif-
ifforential validities for this battery were found.
Ito
ferences between the phi coefficients for minority and nonminority
Black-non-
groups (above) are not statistically significant (cn
.R4, CR Oriental -
minority =-.1?, CR Spanish Surnamed-nonminority =
nonminority = -1.1n).
AVAILABLE
BEST COPY.
The battery is fair to minority group members since the proportion
of Blacks, Spanish Surnamed and Orientals who met the cutting
scores approximated the proportion who were in the high criterion
60% of the Blacks met the cutting scores and 60% were in
group.
the high criterion group; 70% of the Spanish Surnamed met the cut-
ting scores and 63% were in the high criterion group; and 87% of
the Orientals met the cutting scores and 77% were in the high cri-
terion group.
The battery is fair to females since the proportion of females who
met the cutting scores approximated the proportion who were In the
high criterion group.
73% met the cutting scores and 67% were in
the high criterion group.
JOB ANALYSIS
A job analysis was performed by observation of the mIrkers' per-
formance on the job and in consultation with the workers' super-
A comparison
visors for each occupation included in the research.
of these job analysis schedules indicated that the critical job
duties for each of the Mur occupations were similar enough to per-
mit combination of the workers in these occupations.
On the basis of the job analyses, the job descriptions show in
Appendix 4 were prepared.
These job descriptions were used to
(1) select an experimental sample of workers who were performing
the job duties; (2) choose an appropriate criterion or measure of
job performance; (3) determine which aptitudes are critical, im-
portant or irrelevant to job performance (see Tables 2 and6 ); and
(4) provide information on the applicability of the test battery
resulting from this research.
TABLE 2
Qualitative Analysis
Ratjonale
Ai =Dada
Required in determining sctIle to be
General Learning Ability
G
used by analysis of specircations
and data and organizing an: arrang-
ing data into logical sequence for
drafting.
N - Numerical Aptitude
Required in determining scale to be
used through analysis of data.
Required to draw and plot detailed
S - Spatial Aptitude
graphic representations to scale in
conformity with specifications, com-
puted dimensions and spatial rela-
tionships.
BEST COPY AVAILABLE
Required to differentiate minor
P - Form Perc,eption
variances in shading.
Required to observe fine detail
Q - Clerical Perception
in
checking work to perceive errors.
EXPERIMENTAL TEST BATTERY
All 12 tests of the CATB, B-1002B, were administered during the
during 1873
Period from 1966 to 1973 to the validation sample and
to the cross-validation sample.
CRITERION
The ratings were
The immediate supervisor rated each worker.
obtained by means of personal visits of State test development
analysts who explained the rating procedure to the supervisors.
interval
Two ratings were obtained from each sunervisor with an
Since sample members' test
of two weeks between the ratings.
supervisors had no knowledge of
scores are confidential,
the test scores of the workers.
on Sample:
Vaj i Ala
The scale (see Appendix 3)
A descriptive rating scale was used.
Nine of these items cover different aspects
consists of ten items.
Drafter's
The tenth item is a global item on the
of job performance.
Each item has five alternative responses
"all-around" ability.
For the
corresponding to different degrees of job proficiency.
weights of 1 to 5 were assigned to
Purpose of scoring the items,
of
The total score on the rating scale is the sum
the responses.
The possible range is In-50.
the weights for the ten items.
A review of the job descriptions indicated that the subjects
important
covered by the rating scale were directly related to
aspects of job performance.
Satisfactory production must be achieved in
A - Amount of work:
order to maintain desirable progression of work projects.
Plans and drawings must be accurate in
B - Accuracy of work:
order to be acceptable.
Workmanship must be high quality in order to
C - Quality of work:
produce plans and drawings which are readily used and durable.
Drafter must have sufficient knowledge
D - Amount of knowledge:
to oroduce satisfactory plans and drawings.
MAME
COPY
BE%
Drafter must he able to integrate broad
E - Facility for work:
in order to produce
tend specific knowledge of principles
satisfactory work.
Drafter must have specific knowledge
F - Knowledge of mathematics:
acceptable plans and draw-
of mathematics in order to produce
ings.
Drafter must det^rmine validity of
G - Analysis of source data:
s for drafting.
into compor
source data and separate them
sound judg-
Drafter must analyze problems and make
H - Judgment:
ments without constant supervision.
Drafter must verify completed
finished work:
- Checking of
1
and accuracy without check-
plans and drawings for completeness
ing by supervisor.
Drafter's value to employer Involves a
J - "All-around" ability:
performance listed above.
combination of the aspects of job
obtained between the Initial
A reliability coefficient of .94 was
significant relationship.
racinrs and the re-ratings, indicating a
combined scores
final criterion score consists of the
the
Therefore,
criterion is
The possible range for the final
of the two ratings.
The mean is 65.2 and the stan-
The actual range Is 27 -'Vi.
20-100.
criterion and
The relationship between the
dard deviation is 15.3.
job experience is shown in Table
3.
age, education and
TABLE 3
Validation Sample
and Pearson
Means, Standard Deviaticns (SD)
Criterion (r) for
Product-Moment Correlations with the
Age, Education and Experience
Total Sample
Si/
9.7
.006
34.3
Age (years)
-.040
1.4
13.5
Education (years)
.161**
83.0
108.4
Total Experience
(months)
**Significant at the .01 level
- 6
MINABLE
BEST COPY
About one third of the workers
are considered to be marginal workers.
Therefore, the criterion distribution
was dichotomized so as to
include as close as possible
to one third of the sJmple In the low
criterion group and the remainder in the high
criterion group.
The
criterion cutting score was set at 39 which places
34% in the low
criterion group and 66% in the high criterion
group.
ri.C12.1=stadiktiCLILIABliall:
A descriptive rating scale
was used.
The scale (see Appendix 3)
consists of six performance items.
Five of these items cover
different aspects of job performance.
The sixth item is a global
item on the Drafter's "alt
- around" ability.
Each item has five
alternative responses corresponding
to different degrees of job
proficiency.
For the purpose of scoring the items,
weights of
I to 5 were assigned to the
responses.
The total score on the
rating scale is the
sum of the weights for the six items.
The
Possible range is G-30.
A review of the job descriptions indicated
that the subjects
covered by the rating scale
were directly related to important
aspects of job performance.
A - Amount of work:
Satisfactory production must be achieved
In order to maintain desirable progression
of work projects.
Quality of work:
B
Workmanship must be high quality in order
to produce plans and drawings which are readily used
as well
as durable.
C - Accuracy of work:
Plans and drawings must be accurate in
order to be acceptable.
D - Amount of knowledge:
Drafter must have specific mathematical
and design knowledge in order to produce
satisfactory plans
and drawings.
E - Variety of job duties:
Drafter should be able to handle
a
large variety of tasks without specific
instruction.
"All-around" ability:
F
Drafter's value to employer
Involves
a combination of the aspects of job performance listed above.
AVAILABLE
BEST COPY
A reliability coefficient of JO was obtained between the initial
ratings and the re-ratings, indicating a significant relationship.
The final criterion score consists of the combined scores of the
The mean score on the
The possible range is 12-61.
two ratings.
final criterion was 3/.8 with a standard deviation of R.n.
The relationship between the criterion and age, education and job
experience is shown in Table 4.
TABLE 4
Cross-validation Sample
Means, Standard Deviations (SD) and Pearson
Product-Moment Correlations with the Criterion (r) for
Age, Education and Experience
L
Lican
.137
10.4
Age (years)
29.7
-.285
13.5
Education (years)
1.4
62.1
70.2
.122
Experience (months
on current job)
About one-third of the workers are considered to he marginal
Therefore the criterion distribution was dichotomized
workers.
one-third of the
so as to include as close as possible to
high
sample in the low criterion group and the remainder in the
The criterion cutting score was set at 38 which
criterion group.
places 34% in the low criterion group and 66% in the high cri-
terion group.
SAMPLE
Alitlatimn.Agmala:
The validation sample consisted of 326 Civil, Geological, Mechan-
ical and Structural Drafters (2,16 males and 30 females) employed
Appendix
at various companies in the North, South and West (see
A total of 105 were minority group members (40 Blacks, 3n
2).
Spanish Surnamed, 30 Orientals and 5 American Indians) and 221
The means and standard deviations
were nonminority group members.
for age, education and experience of the sample members are shown
State Civil Service tests were used for selection of
in Table I.