DOCUMENT ::',ESUBE TM 008 001 ED 164 579 Manual for the USES General Aptitude Test BaLtery. TITLE Section III: Development. Manpower Administration (DOL), Washington, D.C. INSTITUTION PUB DATE 70 423p.; For related documents, see TM 007 890 and 008 NOTE Small type may be marginally legible 281 ; Superintendent of Documents, U.S. Printing Office, AVAILABLE FROM Washington, D.C. 20402 (Stock Number 029-014-00018-3, $3.95) MF-$0.83 HC-$22.09 Plus Postage. ED FS PRICE Adults; Aptitude Tests; Bibliographic Citations; DESCRIPTORS Higher Education; Norms; *Occupational Tests; Occupations; Scoring Formulas; Senior High Schools; Test Bias; *Test Interpretation; Test Reliability; *Test Reviews; *Test Validity; *Vocational Aptitude *General Iptitude Test Battery; *Test Manuals IDENTIFIERS kBSTRACT This revised manual for the General Aptitude Test item (1) historical development; Battery (GATE) discusses: (2) (5) general (4) physical format; factor analysis; analysis; (3) (6) intercorrelations of raw working population norms (ages 18 -54) ; (7) development of GATB test scores and of GATB aptitude scores; data for various GATE norms for specific occupations (tables present validity studies on 446 occupations defined by the Dictionary of development of norms for specific Occupational Titles) ; (8) (9) validity of norms for specific occupations; (10) occupations; (11) validity development of Occupational Aptitude Pattern Structure; relationship of of occupational Aptitude Pattern norms; (12) (13) effectiveness of tests in aptitudes to college success; (14). correlations with 40 other selection and counseling; (15) reliability and effects of practice; (16) standardized tests; (17) effect of sex, minority group effect of training on scores; effect of aging on status, and cultural exposure on scores; (18) (20) 9th and 10th effect of disabilities on scores; scores; (1;) (22) use of disadvpltaged; use of the GATB with the grade norms; (21) a 706-item bibliography. (C2) test results; and (23) *********************************************************************** made Reproductions supplied by EDRS are the best that can be from the original document. *********************************************************************** 1910 Manual for the USES Section III: DEVELOPMENT O 0. UNITED STATES DEPARTMENT OF LABOR 0 MANPOWER ADMINISTRATION U s DEPARTMENT OF WEALTH, EDUCATION I. VIRLPARE 20210 Washington, D.C. NATIONAL INSTITUTE OF EDUCATION E. THIS DOCUMENT HAS BEEN REPRO- DuCED ExACTLy AS RECEIVED FROM THE PERSON OR ORGANIZATION ORIGIN- ATING IT POINTS OF VIEW OR OPINIONS STATED DO NOT NECESSARILY REPRE, flail 2 SENT OrFIC,AL NATIONAL INSTITUTE OF .K.- EDUCX,TION POSITION OR POLICY Foreword General Aptitude Test Battery (GATB) was developed by the United States Employment Servict and has been used since 1947 by State employ- ment service offices. Since that time the GATB has been included in a con- tinuing program of research to validate the tests against success in many different occupations. Because of its extensive research base, the GATB has come to be recognized as the best validated multiple aptitude test battery in existence for use in vocational guidance. Many schools and other organizations have been authorized to use the GATB for counseling and re:;oarch. information regarding release of the GATB tests for these purpos=2.s may be obtained from State Employment Services. The Manual for the General Aptitude Test Battery is published in four separate sections as follows: Section I, A dministr3tion card Scoring, contains the procedures for administration and scoring of the GATB, and conversion of the raw test scores to aptitude scores. Separate editions of Section I are published for B-1001 (the mark-in-booklet version) and B- :002 (separate answer sheet version) . Supplement to Section I for 8-1002 contains the modifications in pro- cedures in the administration and scoring of the GATB using the -Scan separate answer sheets. It is to be used in IBM 1230 or Section I for conjunction Section II, Nw.nis, ecru/dot/wd Aptitude Pattern Structure, shows the GATB occupationo.1 aptitude pattern structure which is used for counseling purposes. GATB norms for adults and 9th and 10th graders are shown for occupational families. Section III, Derelopmnt, contains technical information on the devel- opment of the GATB; procedures for GATB occupational valida- tion research; techniques used in developing the GATB occupa- tional aptitude pattern structure; statistics on the relationships between the GATB and other tests; information on the effects of age, sex, minority group status, cultural exposure, disabilities, and training on aptitude scores; information on the development of GATB -norms for 9th and 10th graders; and guidelines for using GATB results. Section IV, NOrinti, Specific Ocenpations, r -fnins GATB aptitude test norms used for selection for specific occupations. Also includes alphabetical and industrial indices of the many occupations for which norms have been developed. Copies of Sections II, III and IV of the Manual are available to the public from the Superintendent of Documents, Government Printing Office, Washington, D.C. 20402. Section I of the Manual is on restricted sale. I may be obtained from State Further information regarding Section Employment Services. III 3 Table of Contents Foreword Construction of Tests, 1. Item Analysis 5 Factor Ana lysi.- .J. 7 Composition of the GATI1 1. 15 General Working Population Norms 19 I ntercorrelat ions 6. .).9 Derivation of Aptitude Scores 7. Development of Norms for Specific Occupations K. .17 Validity of Norms for Specific Occupations 9. 63 ,.x-elopment of Occupational Aptitude Pattern Structure 'O. 179 . V;:!ity of Occupational Aptitude Pattern Norms 11. 183 Relationship of Aptitudes to College Success 205 12. Effetiven-sL; of Tests in Selection and Counseling 13. Correlations With Other Tests 229 1.1. Reliability and Effects of Practice 951 Effect of Training on Aptitude Scores .-)75 16. I;. Effect of Sex. iNlitiority Group Status, and Cultural Exposure On Aptitude Scores 277 Effect of Aging On Aptitude Scores 289 18. Effect of Disabilities un Aptitude Scores 19. 311 Norms for 9th and 1('`!1 Grades 90. 317 Use of the C AT1-1 With the Disadvantaged IT;;e of Test Results 359 Bibliography 371 1. Construction of Tests STEPS IN CONSTRUCTION HISTORICAL DEVELOPMENT OF TESTS Before the development of the GATB, sepa- of The procedures used in the development the rate tests had been constructed to measure separate-answer-sheet form of the GATB various abilities that seemed related to success (B-1002) were as follows: in different occupations. Each time the study -17-The construction of items for the IBM of a different occupation was undertaken, new 805 separate-answer-sheet form of the GATB tests would be devised if the job analysis indi- the (B-1002) involved two major .phases: (1) cated that some ability seemed to be important in revision of test items that had been included and the store of USES tests did not already to the original edition of the GATB, 13-1001, include such a test. Each test was constructed adapt them for use with a separate answer with items that were as homogeneous as possi- test sheet; and (2) the construction of new ble with regard to the abilities they seemed to items. This did not apply to the test measuring Over a period measure, but varied in difficulty. itself of time, about i00 tests, composed of items Motor Coordination, which does not lend to the use of a separate answer sheet. It was such as arithmetic, vocabulary, and surface de- number of new necessary to construct a large velopment were del eloped. In addition, certain apparatus tests were devised. By a process of test items to allow for the elimination of some analy- items as a result of the findings of item factor analysis (see Chapter 3 of this Section) sis studies which were to be conducted, and to 11 paper-and-pencil tests and 4 apparatus tests provide a sufficient number of items for the de- were selected from this group of tests as the of an alternate .form each of best measures of 10 factors or abilities. These velopment separate-answer-sheet test to be included in tests formed the first edition of the GATB, of P,--1002. The primary task in the revision B-1001. Thus, in about 21/a hours it was possi- the B-1001 items to adapt them for a separate ble to measure all of the major abilities repre- the multiple- answer sheet was conversion to sented in the entire stock of USES tests. Since choice type of all items that were not already 1945 the GATB has been used as the standard in this form. In developing new test items, an experimental battery in every aptitude study effort was made to construct items that would that has been undertaken. for the development abili- measure, as closely as possible, the same of occupational norms. Previously about 15 ties that were measured by the test items that tests would be chosen as the experimental bat- had been used in the original edition of the tery for the study of an occupation, as a result General Aptitude Test Battery, The items that of estimates derived from the job analysis, and use and the newly constructed had been in for a different experimental battery wes used items were arranged in the experimental form each study. Consequently, there was a possibil- of each test in apparent order of difficulty. The ity that some ability might have turned out to relative difficult!, had to be determined subjec- be sig-niilcant in the job when tests of that tively at this point because the performance of ability were not tried out in the experimental examinees on all of these items was not yet battery. Mcreover, the grouping of jobs into known. The experimental form of each test in- families, on the basis of similarities in abili- cluded approximately 20 percent more than ties, could not readily be accomplished, because twice the number of items to be included in all the occupational samples had not been given each final form of the test. For example, if for the same tests. 1 AIANITAI. FOR THE GATB, SECTION III 5. After the experimental data were col- was planned to have 50 a particular test it lected, item analysis studies were conducted to items. in the final form and 50 items in its al- dc4rmine the difficulty level and discriminat- ternate, the experimental form of that test in- ing value of each item (except for items on the cluded about 120 items. Motor Coordination test). The difficulty level 2. An IBM 805 separate answer sheet was number of cor- was determined by finding the constructed for the experimental form of each To rect responses that were made to each item. paper- and pencil test. except for the test meas- determine the discriminating power of each uring Motor Coordination. At this point, there item. each sample was divided into quartiles was an individual separate answer sheet for based on the distribution of the total scores on each test, even though plans called for the in- each test; subsequently. for each item, the sig- clusion of several tests on each answer sheet in nificance of the difference between the percent- the final form of the battery. An attempt was highest and age of correct responses in the made to. devise 'answer sheets Nv h MI would re_ lowest quartiles was determined. The signifi- suit in maximum clarity for the examinees and in each cance of the difference was determined would facilitatethe administration of the tests. instance by obtaining the critical ratio equal to An appropriate:scoring stencil was devised for the difference divided by the standard error of each of the ansWer sheets, and directions were the difference. (See Chapter 2 of this Section.) prepared for administration and scoring of the fa For the final form of each test (except tests for experimental purposes. the Motor Coordination test) items were se- The next step was to administer the ex- lected that met the criteria established for dis- perimental form of each test (except the Motor criminating- value and difficulty level. In gen- Coordination test), untimed. to a sample of ap- eral, items with the lowest critical ratios or proximately 200 people to obtain item analysis items that were too easy were eliminated or re- data and information such as the time required vised. (See Chapter 2 of this Section.) A few for the test to be completed, difficulties occa- very easy items were placed at the beginning sioned by use of the separate answer sheet and of each test for warm-up purposes. and all any problems encountered in test administra- items in the final form of the test were Ar- tion. The administration directions called for ranged in increasing order of difficulty. Alter- the examiner to record his observations on nate forms of each test were made as equal as these points. In most instances only one of the possible with respect to the difficulty levels and (;ATIt tests was administered to an experi- discriminating values of the items included on mental sample; hut, in a few cases, the experi- were long form. Time which limits, mental sample took two of the tests. The ex- each enough to enable obtaining a sufficient sample iwrimental administration of each test was of each examinee's performance with respect without a time limit in order to permit each to each test, and which were short enough to examinee to attempt all of the test items so insure that very few, if any. examinees would that item analysis data would he =ivailable for complete each test, were set in accordance with every item. and to collect data for the estab- the findings of the experimental studies. suitalde time limits for the tests lkhment Irf 7. The final forms of the IBM 805 answer 1. Some experimental work was also done sheets were constructed accordance with. with the motor tests to determine the time lim- in findings of the experimental studies and sug- it; and scoring' i}rocedures which would yield gestions made by the test examiners who par- optimum results with respect to the reliability '. Answer sheets wcre ticipated in these studi of the tests ;111(1 the time requ;red for adminis- devised so that each one included spaces for tering and scoring-. This involved administering the responses to several tests and at the same these tests to various samples for several test - time were arranged to facilitate the tasks for reliability studies retest different in NO) ich both the examinees and the test examiners. time limits and scoring techniques were em- Appropriate scoring- stencils were devised for ployed and the number of trials was test the answer sheets. varied. 3 CONSTRUCTION OF TESTS Aptitude (see to the measurement of Spatial S. IZe.suits (df the eXperilnentai studies On the It Chapter ;1 of this Section), and since Part limits motor tests were analyzed, :ind the time by itself had shown substantially high test- and scoring procedures were determined ac- it was concluded that retest reliability (.8.11, cordigly. As studies, these a of result :1 without Part F timid be omitted from B-1002 change \\-as made which simplified the :;coring battery. the or the effectiveness reducing the paper-and-pencil motor test. was of It. Therefore, only one measure of Spatial Apti- found that none of the experimental time lim- tude was included in B-1002. its ot. modifications in administration and scor- In 11-1001 there had been some overlapping ing procedures for the apparatus tests resulted A-- Aiming 0) among the measures of Aptitude sufficient increase in reliability over the reli- in Eye-Hand Coordination and Aptitude T--IJo- ability obtained with the procedures used in been tor Speed. Eye-Hand Coordination had to justify the necessary increase in I-1-1001 measured by the tests designated as Part C ltesting-4tinie that would be required if the new and Part K and Motor Speed had heen jyrp,,,Lidtirt,s were put into Ilse. Therefore, it was vali- ured by Part G and Part K. Since litany decided to use the same adlnillistlatinn and dation studies on occupational samples yielded 1(11)2 that had been scoring procedures for results which were quite similar for Eye-IIand used for the apparatus tests in 11-1001. (See it was con- Coordination and 11.1otor Speed, for llrr I of the .1Ion Section (;.-AT1.. ) it. was not necessary to have two cluded that 9. The directions for the administration Will separate measures of these aptitudes included scoring for the final form of B--1002 \ee pre- in the battery. Therefore, only Part K, which paed. The findings of all experimental studies Coordination :t measure of both Eye-Iland \Yene taken into account in the preparation of is and Motor Speed, was selected from the three these adniinist ation and scoring- procedures. pape -and-pencil tests of motor abilities (Parts (i:l7'1;.) (See Section I of the .11 ore mil few C, G, and K) that had been in B-1001 for in- to II). Several StUdieti ET(' also roll(111(1.e(1 E-1002. Further evidence of the determine the alternate form rliability clusion in :Intl overlapping of these two aptitudes was found in the separate-ans\er- comparability of each o tabulat- a study of 121 knitting mill workers, tests. These ste lies that sheet ilaVe sh()W11 ing- machine operators, and hand decorators, in Forms A and It of each test art' nut sufficiently orrelation of .81 was obtained be- which a identical \ith respect tc their ra\v score ineans and Motor Coordination Eye-Ifand and standard deviations to use the same con- tween Speed (Honig-et., 1952). The test-retest reliabil- \esion table for Volans A and It of each test ity of Part K (.91) justifies using it as a single to convert raw test scores to aptitude scores. measure. Part K of B-1001 has been desig- Therefore. ;1 si_paritte conversion table \vas de- nated as Part S in II-1002. The aptitude meas- \-eloped to convert the raw test scures of Form tired by fart S in 002 was named Motor It of each test to aptitude Sr)l'eS. fiea;;unably Coordination. close agreement het \Veen the scores :q tests included Thus. whereas IS B-1001 derived front Forms A and 11 of the separate- aptitudes, B-1002 includes 12 measuring- \":1-; obtained in the altel'- antilVer-Sileet tests 1i) tests measuring 9 aptitudes. The 12 tests in- nate form reliability studies. (See Chapter I. cluded in 11--1002 and the 9 aptitudes that they of this Se(tion.) of this measure :te described in Chapter 11. Consideration \vas given to the possibil- .1 Section. ity of reducing the administration time of the inswer sheet for Iii the late 1950's an battery witInnit any appreciable in Incas_ 12. the GA!: which could be scored by optical omitting ureinent ..veral front It -104)9 by scanning, equipment Was developed in a special tests that had been ;Winded in TWo 1- IOU] research study conducted by William J. Schrader arid Part tests, designated as Par.. II, had of the ".S. Army Ordnance Corps and Dr. Ken- been included Spa- its nleaSttres part Ifoyt of the State University of Iowa ttai Aptitude. ja. had been II.) .ighl 11,1, neth It. fp use Center Measurement so that it contributed an insignitican4 .imount 1Zesearch on GATI1, SI.:4"I'I0N N A I , FOR 'I' I I I 1 version of the Pegboard used to measure equipment. Examinees had difficulty (MI;(') 1r:ic Parts II and 10. This Pe2fioard and the pegs using the first answer sheet (MIZ(' --A); thee- made of durable plastic, with ;ire prePhed it fore used revised Sheet ;Lnti\VCI. a NV;l5 whereas tither versions or the Pegboard and th irereill'eS Were found i) AM.( . Be_ the pegs used with them are made of wood. bet \\.(ten scores made tin the S )5 a 11S\ I ; I I cause of tie possible effect of different types Of sheet :old those Thade on the 1111al- A for Parts itus on 5()1'05 obtained, several compara- 7 of the (1ATII. A similar comparison be- :iplr: I t bility studies Were conducted, A study com- reN-ealeil no signifi- tvecn :111:r A ;ind cant differences. Dire to administrative /woe_ Security Employment by the pleted 1970, indi- Ceinmissien or North Carolina in !ems, a direct comparisen betveen the IP:11 xis:, cated that there are no significant differences ans\er sheets was not made. and the MIN' between sores obtained with the wooden and In 1902, the employment service also be- 1:t. plastic Pegboard. As this study involved a re- gan work to develop an answer sheet for view nr all known previous research on the (;ATit could he scored by optical sc;111- the as independent research, matter as nin,r equipment. Tile DocuTr;in (science E.e- \yell two types of Peg-boards ,dintild he considered search Associates) answer sheet developed \\.as follo\ving a interciningeable. into operational use in put 196:1 comparability study conducted in (*alit:mill:I, 10. In 1909 answer sheets for Forms A and Pennsylvania and West Virginia. The conlp:na- It of the CATI1 were developed which could be bility study showed that adjustrnents for raw I MI 12:10, 12:11 or 12:12 optical scan- scored on (desribed in 1. 5 and 7 on t he -DocuTran scores on Parts 1. :1, This study ning equil)111('Ilt. Chapter 7 of this Section) was conducted by answer sheet \vere needed to make them corn- 805 answer sheets. The parahle to scores on the Vtah Test Development ('enter in coopera- 1)octiTran answer sheet 11;e1 Vert' cramped re- tion with the California, :11ichigan, Texas, and sponse areas, Therefore. w-hen the expanded Ohio agencies. The results showed that slight intio- adjustments were needed in Part. 5 scores to NtS answer sheet described belo\v fell (3uced, use of the 1)octiTran answer sheet make the scores comparable to the scores ob- off quickly. Although the DoetiTran answer tained using the 111M 8O answer sheet. There- is no 101.1-rer available, the development fore, separate aptitude conversion tables were sheet of this answer sheet represented the employ- prepared for that test for the IBM 12:10-1232 ment service's entry into the area of optical answer sheets. scannim., of ans\ver sheets and computerized 17. ln 1970 a Digit el: answer sheet (produced conversion of raw scores to aptitude scores. by the Optical Scanning ('orporation) was de- Computer Systems National In P.01 veloped for Form B. A comparability study 1.I. (N('S) ;i1ISNVer sheets for Forms A and It of conducted by the Ohio State Department of de\-eleped on the the C.ATB, 11- ln2, adjustments were Nk'ere that indicated Education stud (desibed iii basis of a coniIn needed in' scores obtained on Part 1 of the Digi- Chapter 7 of this Section). The results showed tek ;ins w e r sheet to make them emnparable to t 11:it 1(1.ili:4inelitS Were required on Parts 1, 2, score, on the 1 BM 805 allSWere sheet. 5 and 7 on the NCS answer sheet to make them cnnip;trable to scores on the If111 805 an- REFERENCE swer sheets. Therefore. separate aptitude con- -version tables were developed for those tests IL The generalized distance lune- .1. differential aptitude testing. Un- for the :\ CS answer sheets. 1. ion and doctoral dissertation, (Iniver. of puliiished 15. In the mid- 1960's one of the apparatus sum:hers developed a lighter and more innesota, 1952. corn- 2. Item Analysis RESULTS OF STUDIES COLLECTION OF' DATA The results of the item analysis studies dif- The experimental forms of the tests to be in- fered for the power tests (such as the numeri- cluded in the separate-answer-sheet form of cal and verbal tests) and speed tests (such as the CATB (B-1002) were administered un- the name-comparison test). As would be ex- timed to several experimental samples to ob- pected, the power tests yielded a wider range of tain data for item analysis purposes. The tests item difficulty than the speed tests as well as were administered untimed so that each exami- exhibiting a correspondingly wider ronge than nee would have an opportunity to work all of the speed tests for the discriminating power or the items. In all, a total of 10 samples, ranging 'diagnostic values of the items. In the selection in size from 196 to 236 examinees each, were of items for the final forms of the power tests, tested for this purpose in Colorado, Florida, items that were too easy and those that did not Michigan, Ohio, Texas, Utah, and Wisconsin. show sufficiently great diagnostic power were The experimental samples included local Em- eliminated. In general, items that had critical ployment Service office applicants, high school ratios of less than 2 were not included in the juniors and seniors, college sophomores and final forms of the tests. However, several very juniors, commissioned and noncommissioned beginning of easy items were placed at the officers of the United States Air Force, and each test for warm-up purposes. The alternate some groups such as buSiness women's clubs forms of each test were mad( ;Is equivalent as and civic luncheon groups. possible by including in each form an equal number of items of the same difficulty level METHODS OF ITEM ANALYSIS and discriminating power. The items on the final form of each test were arranged in in- The data were analyzed to determine the dif- ficulty level and diagnostic value or discrimi- creasing order of difficulty. nating power of each item. The difficulty level Since differentiation of individuals on speed was determined by counting the number of tests is determined primarily by the rate at correct responses to each item. To determine which each examinee works items that are rel- the discriminating power of each item, each atively homogeneous in difficulty, the criteria., selecting items for the sample was divided into quartiles based on the employed for final distribution of total scores for each test; subse- forms of the power tests were not applied to quently, for each item, the significance of the the same extent to the speed tests. However, difference between the percentage of correct since some of the speed-test items did vary in responses in the highest and lowest quartiles difficulty level and discriminating power from was determined. The significance of the differ- others, these factors were used as guides in se- ence was determined in each instance by ob- lecting items for and arranging them on the taining the critical ratio equal to the difference final forms of the speed tests. The results of the item analysis studies divided by the standard error of the difference. To eliminate the necessity of calculating each made it quite apparent that the tests measur- critical ratio, reference was made to an abac ing Form Perception and Clerical Perception devised by Mosier & McQuitty (1940) from are primarily speed tests and those measuring which the critical ratios could be read directly Intelligence, Numerical Aptitude, Spatial Apti- tude, and Verbal Aptitude are primarily power once the percentage of correct responses in the tests. No item in the tests of Form Perception highest and lowest quartiles was known. 5 mANt'Ai, rolt GATII, SECTION HI fi and (lericLI l'ereeption was failed hy more pliclde to these types of tests, are also speed 10 percent of the experitmtal sample than tests. during the utitned administration; vhereas the range for the percent failing items on the EFEH EN(:E other tests extended up to 91 percent. The tests Methods of 1losier', ('. & A1cQuitty, measuring- :\lotor Coordination, Finger Dexter- item vali(lation and ;times for ite-test cor- ity, and Nlatitial Dexterity, vhicli not i 11- the item studies lio of tipper-lower dif- Chided relation and (Tit iCal in I)ec :ruse standard Item analysis procedures are not ap- Terence. P,,q/choiiiitiiii.rt, 1910, 5, 57-65.
Description: