Development of USES Specific Aptitude Test Battery

19 Pages·2007·0.48 MB·English

0
DOCUMENT-RESUME
\.?
TM 820 870
Et 223 722'.
Development of USES Sprific Aptitude Test Battery
TITLE
.
petrol. refin.)
for Reffnery Operator
549.260-010.
(DOL),
Employment and Training Administration
SPONS AGENCY
Washington, D.C.
/
S-68R82
REPOliT NO
RUB DATE
82
.19p.; Analysis arnd report by Northern Test,
NOTE
Devglopment Field Center,.Detroit, Mithigan.
Descriptive (141) -- Reports
Reports
PUB TYPE
..Research/Technical (143)
MF01/PC01 Plus Postage.
Ephs PRICE
,
,*Aptitude Testsr Employment Qualifications; Jot
DESCRIPTORS
Ahalysis; *Occupational Tests; Personnel Evaluation;
Predictive Measurement; Semiskilled Workers.; *Test
Construction; Test Use; Test Validity; *yocatkonal
Aptitude
*Refinery Operators; Test Batteries; USES Specific
_IDENTIFIERS
.
Aptitude Battery.
-
ABSTWT
(USES) Specific'
The United States Employment Service
is evaluated frOm
Aptitude Test Battery (SATB) for Refinery Operator
the research; (2)
three points of-view: (1) technical adequacy,of
battery to
fairnessl:to minoritieg; and (3) usefulness of the
selecting individuals for
Employment Service staff and employers'in
demonstrated a
training in refinery operator positions. Research
between proficiency
statistically significant"and useful relationship
SATB.-The SATB can be-expected to
as refinery operatdrs.and the
of highly proficient
produce, a usef41 increase in the proportion
validation sample, composed
workers. When eke SATB was applied.to the
considered competent,
at individuals who were employed/and theretore
highly
from.62'percent to 72 percdnt in the proportion of
an inCrease
foiind Or'the
proficieht workers was found. Similar results were
expected when the
e.cross-validation sample. A greater increase can be
of relevant abilities
\ battery is used with applicants, as. the range
"is wider amang applicants than among employed workers. The report
(3) analygis; and (4)
dncludeg: (1) research summary; -(2) procedure;
for subgroups ,of the
validity of the battery.-Descriptive statistics
job description are
validation sample; descriptive rating scale; and
contained in the appendices.4(Author/PN)
e
*************i*****************************
***************************
*.
_
be made
.
Reproductions supplied by EDRS are the best that can
i
*
from, the original document.
.
t
*********************,*****4*****************************t*************
Refinery Operator
41.
Development of USES._
Specific,Aptitude
(petrol. iefin.)
Test Batter)) S-68R82
549.260-010
U.S. Department of Labor
Employment and Training Administration
U.S. Employment Se'rvice
c.
182
("J.
4
U.S..DEPARTNIENT OF EDUCATION
NAT1OFIAL INSTITUTE-OF EDUCATION
EDUCATIONAL RESOURCES INFOqMATION
bEN I ER (ERIC)
This document has been reproduced as
person or Organization
rereleed
runt
(in
origindting it
Nov.!, 1,ove berm Inath! to itnprove
Minor
reproduction quality
Roint ot view on opinions statedn this docu
merit dbot necessarily represent allocral NIE
v
positron 01 HAL),
A
0
0
Av.
6
0
tar
a
1
,
DEVEWPMENT,OF USES SPECIFIC APTITUDE TEST BATTERY
for
('
REFINERY OFpATOR (petrol. refift%) 549.260-010
S-'68R82
5-
Deyeloped in cOopatatfonwith ehe California, Kansas,
Kentucky, Louisiana, Mich4an, New Mexico,, New York, Ohio,
South Carolina, and Tel,cas State EqplOyment Services
Analysis and Report
by
-
.
enter
Northern Test Developmerit Field
Detroit, Michigan
f
U.S. DEPARTMENT OF LABOR
Employment and Training Administration
United States Employment Service
P."
1982
:\
01(
4
'r
ACKNOWLEDGMENT
'Mt
EniProyment-SIrvice
The United States Departmeta of.Labor and affiliated State
.Agencies'express their since-re gratitude to the following organizations for
cooperating in this research.
Allied Chemical Company, Detroit, Michigan
Caroshna
Amoco Chemical Corporation, MtPleasant, South
ChannelAew,
Texas
ARCO Chemical Company,
!.
Louisviile, Kentucky.
Ashland drr
Chevron U.S.A. Inc.; Bdkersfield, California
Liquid Carbonic Corporation, Geismar; Louiiiana
Kansas
Mobil Oil, Corporation; Augusta
.
Mo '1 Oil Corporation., Buffalo, New Y
Placid Refining Company, Port Allen, Louisiana
Mexico
Plateau Refining Company, Farmington, New
SOHIO Refinery, Lima, Ohio
Mexico,
Southern Uniqn Refining Company, Hobbg, New
,
a
.
t
4
a,
ii
1
TABLE OF CONTENTS
.0-
PAQE
1'
ii
'ACXNOWLEDGMENT
0.
.,-,....k.....
.
.
1
..
,
SUMMARY
.1.
,
.
.
.
,
.
2
PROCEDURE
4
o
2
4
e
1 Job.Analicis
0
3
Experimental Test Battery
%
.
-
.
.
ValidationSample'DescriptIon
3'
.
t-,
.
-4,
.
.
3
Cross-validation Sample .Description
A
'
4
3
Criterion for Validation Study
,
,
5
Criterion for Cross-validation Study *
,
ANALYSIS
6
VALIDITY OF THE BATTERY
6
Criterion:Related Validity/Pr
7
Effectivpieis of the,BattiWy
I
8
Subgrod0 Analysis.
8
Prior Battery
-
APPENDIX 1
9
Descriptive Statilaics for Black and Nonminority Subgrotps
APPENDIX 2
11
Descriptive Raring Scale
APPENbIX 3
.
15.
Job Duties
.
V"'
.
S.
tJ
1.
9 0
(-1
44-
.
,,
,
,
3.GATB Study No. 3066
1r
,
,
.
.
,
-P
..
,
.
,
,
.
.
.
\
,
0
DEVELOPMENT OF USES SPECIFIC APTITUDE TEST BATTZRY S-68R82
1
for
REFINERY OPERATOR i(petol. refin.) 549.260-010
1.
SUMMARY
evaluate the
This report is designed,to provide the information required to
Spesific Aptityde Test Battery (SATB) for Refinery Operator froethree points
(1)_technical adequacyof the research; (2) fairness to minorities;
of view:
stpfrand.employers
and (3) usefulnesa*of the batfery.to_Employment Service
positions..
in selecting'fhdividuals for training in Refinery.Operator
Research demonstrated a statistically signiffcant and useful relationship
Aptitude
between 13roficiency as Refinery Operators 'and she follmAng Specific
Test Battery:
Cutting
ores
Sc
Aptitudes
.-
85
N - Numerical Aptitude\
I.
90
,--
S --Spatial Ap,titue
85
P - Form Perception
j-
The validation sample, on whichpthe SATB 4
use,d'in the research.
Two sam les were
workers (inclufling,41 blacks) fi.om 10
was de elopedr consisted of 194 em,ployed
The*tests used were those of tne
Data were collectea during 1974-1980.
state
.
Job proficiency was measured bST supervis
General Aptitude,Test Aaetery (GATB. .
ratings.'
('
Thi s sample consisted ofy
nfirmed or tross-validated the SATB.
second saMqe
\ A
The same eXberimental'tests were'used;'the criterion, or
1 03 Refinery, Opeftors.
The data were
measUregent Of jdb.proficiency,'consisted of supervisory rat?ngs.
collected in 1954
.....'4
,
.
.
.
--/
?
:
1
,
.
'
bladks and nonminorities was
No evidence,of..differencesin validity between
.
faA to:blaeks.and nonthkaorities hsing several
founds, ,,The 'SATB was found toZbe
Additidhal information is presented in Ehe Validity
definitgions.Of Eairness.
iAvendix 1;
4
of the Battery section and
the pr9portiovof
'0
f
a Useful increase in
The SATB can be expected to 'produce
When,the SATB was*applied to the Validation Sample,
highl; proficient workers--;
.
tterefore considered competent,
compoed of individualS who Were employed ahd
,
in the proportion.of highIY prd-ficient workers was
an &crease from 62% to 72%
A greater
sample:
found. 'Similar.resultS were found for the.croas-validation
with-applicants, as the range
increase can be expected when the battery is used
'aorkeA.
than among employed
of relevant abilities is wider ainong applicants
.°
.
.
-2-
Procedure
0
collected at about
A concurrent design was used (test and criterion data were
.
Rata for the validation .sample were collected,during 1974-
the..mame time).
1980.
q'
Job Analysis
A job analysis wai ,performed by observing'the !Winery Operators' performance
On the
Refinery Operators' s.uPervisors.
on the job andsby consulting with the
used top'
basis of the job analysis, ajob AellrFiption was prepared which.wa's
performing those
select an experimental sample of. Refikery Operators who were
_job dutieeand choose an appropriate criterion or measure of job performance.
job`dueies were compared
At each loeation listed, under ACKNOWLEDGMgNT, the
.
If Minor
with the job description and found to be essehtially the aame.
the job desCriptiOn
Sifferences were found, the job dearription was,modified.
used to provide
'shown In Appendix 3 is the resUlt of this process and may be
this re-
information on the applicability ofthe test,battery resulting from
r
search/
of performance,
In the job anarysis, each job duty was . rated for.frequency
Crittical job duties
percentage of; time sperit, end level pf difficulty.
rting.
were identified op the basis of thdse
the aptitudes a 4,trelevant, im-
At each location at least one analyst rated
.
A'synthesis of thes
job dutie53.
pottant or critical to the performance of the
ratings and their rationale follows:
°
Required to.learn and underegtadd principles
G - General Learning Ability
Re-
and protedures af refinery operation.
'quired"ta perCeive re1oria0 data present in
,
specification& and re,adj.ngs in order to
di'agnose-and portNt trouble.
ttequired to continuously dbserve instruments
P -.Fdrm Perception.
.andsvecordingAevides for deviations from
Required to prepare and in-
ecifiadtionsA
z'
4
:
instruments
set grpph p per into recording
arld to accur tely adjust knob'S, dials and
0
uired to, make visual comparisons
R
Levels.
ok prOduet with alar-.
'
Required ,to:accurately obser:re and record
Q - Clerical Perception
readings on instrruments and recording de-
vices, aneto accutately/compare'tabular
;data given in logs ands-ptables.-
-
Required`telmanip014ce knOps, buttons,
M - Manual Dexterity
.
..
evers oft4control panels.
switches Add
l
.
insertand adjust,charts,
p Required to-
graph paper,and tapes used in recording
L..
.
.
Requiredvp m nually opetate
devices.
vielves and manual dontrbls on units and
Auxilliary equiliment.
'
7
-3-
-
,
z
ExperLental Test Batteri
f
...
,
.
1
The eXperimental testbattery consisted of-ell 12 tests of tfie GATB-1002B.
Information on the coMposition and developmental research of the'GATB may be'
found in the Manual for the General Aptitude Test Battery, Section III,. Develop-
mént, available from the Government Printing Pffice:
Validation Sample Description
The validation sample consisted of 194 Refinery Operators (12 females and'182
males) employed in companies in the North; South and West (see ACKNOWLEDGMENT)..
A total...of 55 were minority group members (41 blacks', 8 Hispanics, 3 Orient.als,
_
2 French Canadians, and 1 AilericAn Indian) and 139 were-nonmoinority geoup
educationjand experi nce
members. ''The means and standard deviations for age,
of sample members are shown in Table 1.
Two employers p'articipating in the research used test's in their selection pro-
The &pie Occupatipnal Literary Test, a measure of basic 'reading and
cess.
The second employer used two tests, the
math skills was used by one emplOyer.
Survey of. Mechanical Insight and the California Capacity Questionnaite, hut did
The
not use the obtained scores to automatically eliminate any applicant.
All Refinery Operators had at.
remainder of the sample was not test selected.
least 6 months experience on a job which has duties similar to those found
Decriptive statistics for blacks and
in the job description in Appendix .3.
nonminority subgroups are shown in Apkatfix 1.
Cross-validation Sample Description
3 male refingu operators employed.in
The cross-validation sample consisted of
Data for this study were gathered in 1954, pi-Dor to the requirement
the North.
Therefore, ethnic group'composition
of providing minority group information.
The means and standard deviations for age,. education
of the sample is unknown.
/
and experience of sample members are Awn in Table la.
a
Ciiterion for Validation Study
.°
The
Thel criterion for the vi,lidätion sample consisted of supervisory ratings.
The ratings were obtained by means
immediate supervisor rated each worker.
State test development analysts who explained the rating
of personal viSits b
Two ratings were obtained from each supervisor
procedure tO the supervisor§.
Since sample mem-
with in interval of at least two weeks between the ratingb.
bers' test scores are Confidential, supervisors had no knowledge of the test
scores nf workers.
The:scale (see Appendix 3) consists of
descript,lve rating scale was used.
.
Five of'these items cover different aspects of job performance.
;ix items.
"all-arOund" ability.
The sixth iteM is a g obal it,em on the Refinery Operators'
degrees Of
..;Each item has fiVe al ernattve responses corresponding to different
F2s-t1Te purpose of scot'ing thd'items, weights of 1 to 5.;vere
'job proficiency:
o
The total scote on the rating scale is-the sum-of
assigned to the:reiPonses.
.
The possible range for,each rating is 6-10. -"
the weights for the,six items.
.
..
.
...A'review of the job description indicated that.the subjects coveted by the rat;
A
ing scale were.directly related to important aspects of job performance.
,
summary of these relationthips follow:.
'
t
A Refinery Operator must work quickly and efficiently
Quantity of Wark:
A
and othei control deVIces.
to Make timeli manipulations of valves, levers;
.
The.work of a Refineiy Operator must be of high quality
Quality of.Work:
..P.
his/her control meet strict-/-
to insure that the products and procigsses under
quality and safety specifications.
.
.
The work of a:Refinery Operator must be precise'in the
C -ACcuracy of Work:
loss of product or
measure of many process variables in order to prevent
.
the creation of hazardous working conditions.
'
2
;
The work of a Refinery Operator requites the acquisition
D - JacKnowledge:.
and use of knowlege of the mechanical and chemical processes that are
,
initiated and monitored in the manufacture of refinery products-.
,
The work of a Refinery Operator requires the capacity to
E - Job Versatility:
add effective operation of -
perform a va4ety of duties involved in the safe
'a refinery.
'
A Refinev Operator's value to the employer in-
F - "All-around" Job Ability:
above..
volves a combination of the aspects of job performance listed
the,initiai ratings and
A reliability coefficient of .83 wgs obtained between
4
Therefore, the final
the reratings, indicating a significant relationship.
The possi-
celterion score consists of the combined scores of the two ratings.
The actUal range ia 15=6D-. The
ble tinge for the final criterion is 12-60.
7:9-.---T1e relationshipbetween the
mean fs 42.1 with a standard deviation of
1, below.
criterion and age, education and experience is shown in Table
Table 1
Means, Standard Deviations (SD) 'and Pearson
Product-Moment-Correlations with the,Criterion (r) for
Age, Education,and Experience"
.
Mean
SD
,
i
.07
11.4
34.3
Age (
Ye
s)
:
1
.01
12.6
(years)
f 1.7
Educatio
1 1.3
85.8
Total Experience
(months)
.
.
t
.
.
9
**Signifitant at the .01 level
-5-
For the purpope of analysis, the criterion-distribution 4as dichotomiiednso as to
include, approximately one-third of the sample in the low criterion group and two-
This is the standard procedure for SATB
tbirds in the high criterion group.
The.caterion tuttlng score was set at 39 which placed 38% in the'low
studies.
,
criterion group and 62%,in the high criterion group.
Criterion for the Cross-:validation Stndy
The criterion used in the cross-validat.ion study consisted of supervisory ratings
For computational purposes the ratings Were converted to linear,
in/rank order.
The relationship between the criterion and age, education
sares and then averaged.
and experfence.is shown in Table la.
,
r
TABLE la'
(SD),-And Pearson Prodvt-Moment
- Means, .Standard Deviations
Correlations 'with the Criterfon (r) for
Age and Education
Cross-validation Sample'
N63
SD
Mean
4.5
46.4
-t*142
Age (years)
1
.236-
2.4
10 1
Educatfon (years)
'4
53.9
.148
Experience (months)
169.7.
.
ANAIXSIS
.
which show
The intitial,step in the analysis is to Identify those aptitudes
This evidence can be:
some evidence of validity and job relatedness.
Statistical evidence of the correlation (r) between the test and the cyiterion.
1.
Content validity as evidenced by a rating of "critical" based on the job
2.
analyais, or
r,
Any combination of the following:
3:
high mean."'
standard deviation (SD)
lo
rating of "importanp" based'on the job analysis
Statistical results for the validation Imre are shown in Table 2.