DTIC ADA245199: Measuring Benefits of Manpower, Personnel, and Training (MPT) Research and Development

60 Pages·2.5 MB·English

by  Defense Technical Information Center

I, J
AL-CR-1 991-0007
A
199
AD-A245
MEASURING BENEFITS OF MANPOWER,
PERSONNEL, AND TRAINING (MPT)
A
RESEARCH AND DEVELOPMENT
R
OTt ,
S
Jonathan C. Fast
-D
Bruce H. Dunson
T S
Rebecca Wortman tL-CT
Brice M. Stone AN29 199
0
Metrica, Incorporated A
8301 Broadway, Suite 215
N
San Antonio, TX 78209
G
Larry T. Looper
Sheree K. Engquist, Capt, USAF
L
HUMAN RESOURCES DIRECTORATE
MANPOWER AND PERSONNEL RESEARCH DIVISION
Brooks Air Force Base, TX 78235-5000
B
0
R January 1992
A
Final Paper for Period July 1988 - November 1991
T
0
R
y Approved for public release; distribution is unlimited.
92-02175
9 2 1 2 7 1 0 0III IlIHIllIII I i
AIR FORCE SYSTEMS COMMAND
BROOKS AIR FORCE BASE, TEXAS 78235-5000
NOTICES
This technical paper is published as received and has not been edited by
the technical editing staff of the Armstrong Laboratory.
When Government drawings, specifications, or other data are used for
any purpose other than in connection with a definitely Government-related
procurement, the United States Government incurs no responsibility or any
obligation whatsoever. The fact that the Government may have formulated or in
any way supplied the said drawings, specifications, or other data, is not to be
regarded by implication, or otherwise in any manner construed, as licensing the
holder, or any other person or corporation; or as conveying any rights or
permission to manufacture, use, or sell any patented invention that may in any
way be related thereto.
The Office of Public Affairs has reviewed this paper, and it is releasable
to the National Technical Information Service, where it will be avlilo -'- le
general public, including foreign nationals.
This paper has been reviewed and is appn
LARRY T. LOOPER WILLIAM E. Al v _ irector
Project Scientist Manpower and Personnel Division
ROGER W. ALFORD, Lt Col, USAF
Chief, Manpower and Personnel Researzh Division
J Form Approved
REPORT DOCUMENTATION PAGE OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, incuding the time for reviewing instructions, searching existing data sources, gathering
aid mintaing the data needed, and completing and rewewig the colec1ion of in m n. Send comrnra rdng this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for information Operations and Reports 1215 Jefferson Davis Highway, Suite
1204, Arlington, VA 22202-4302, and to the Office of Management and Budget Paperwork Reduction Project (0704-0188), Washington, DC 20503.
1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 13. REPORT TYPE AND DATES COVERED
January 1992 Final Report - July 1988 - November 1991
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
Measuring Benefits of Manpower, Personnel, and Training (MPT) Research C - F41689-88-D-0251
and Development PE - 62205F
PR - 7719
TA - 20
6. AUTHOR(S)
Jonathan C. Fast Brice M. Stone WU - 14
Bruce H. Dunson Larry T. Looper
Rebecca Wortman Sheree K. Engquist
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
REPORT NUMBER
Metrica, Incorporated
8301 Broadway, Suite 215
San Antonio, TX 78209
9. SPONSORING/MONITORING AGENCY NAMES(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING AGENCY
Armstrong Laboratory REPORT NUMBER
Human Resources Directorate AL-CR-1 991-0007
Manpower and Personnel Research Division
Brooks Air Force Base, TX 78235-5000
11. SUPPLEMENTARY NOTES
Armstrong Laboratory Technical Monitor: Larry T. Looper, [phone]
12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE
Approved for public release; distribution is unlimited.
13.ABSTRACT (Maximum 200 words)
The Air Force is constantly trying to develop new or improve existing tools to increase the efficiency in the
way personnel life cycle resources are managed. One metric, commonly used, is based on utility. This research
produced a utility assessment technology to aid decision makers. This technology involves the process of
identifying, measuring, and combining attributes to create an explicit value structure to form a basis for evaluating
MPT research projects and selecting the most beneficial and cost effective portfolio of MPT research efforts
Four different techniques were evaluated and compared, those being utility analysis, cost benefit analysis,
production functions, and decision theory. The research identified cost benefit analysis and decision analysis
as being most applicable to MPT research projects. .
/
14. SUBJECT TERMS 15. NUMBER OF PAGES
ost benefit analysis decision theory utility assessment. 62
-
decision assessment , decision analysis 16. PRICE CODE
.
17. SECURITY CLASSIFICATION 118. SECURITY CLASSIFICATION I1O9 FSEACU RITY CLASSIFICATION 20. LIMITATION OF ABSTRACT
OF REUPOnRcTla ssified I1 1.OF THUISn PcAlaGsEs ified Inc~s C ifi UL
NSN 7540-01-280-5500 Standard Form 298 (Fv 2-89)
Prescibed by ANSI Std Z39-18
290-102
II
________________________________
TABLE OF CONTENTS
Page
SUM MA RY ........... ............................... 1
I. INTRODUCTION .................................. 1
Technical Approach ................................ 2
Utility Assessment ................................ 2
Step 1. Identify the Perspective from Which Utility
Is to be Assessed ...........................
3
Step 2. Determine the Scope of the Problem and Identify the
Overall Objective of the Utility Assessment Situation . 3
Step 3. Identify the Set of Alternatives to be Evaluated ...... 3
Step 4. Determine the Attributes on Which the Alternatives
Are to be Assessed .......................... 3
Step 5. Develop Measures for Each Attribute .............. 3
Step 6. Choose an Assessment Model .................... 4
Step 7. Evaluate Each Alternative Using the Assessment
M odel . ............................... 4
Step 8. Select the Best Alternative or Rank Order the
Alternatives ............................ 4
II. DESCRIPTION OF EVALUATIVE TECHNIQUES ............... 4
Utility Analysis... ................................ 4
Production Function ................................ 5
The Effect of R&D in the Production Function ............... 6
Key Issues Identified in Past Research .................... 6
Analysis of MPT R&D at the Aggregate Level ............... 7
Steps to Estimating the Production Function of MPT R&D .... 8
Cost Benefit Analysis ............................... 8
Valuation of Benefits and Costs for MPT Research and
Development ................................ 9
Enumeration of Project Benefits ........................ 10
Enumeration of Project Costs ......................... 11
Decision to Perform or Abandon the Project ............... 11
Decision Analysis . ................................. 12
III. APPLICATION OF TECHNIQUES ......................... 12
Development of the Assessment Model ........................ 12
Utility Analysis ............................... 12
Pre-selection ............................. 12
Post-selection ................................ 13
Data Requirements ............................ 13
Model Specifications ............................ 13
iii
Page
Production Function ................................ 13
Cost Benefit Analysis .................................... 14
Decision Analysis ...... ................................ 19
IV. ANALYSIS RESULTS .............................. 23
Sensitivity Analysis ................................. 23
Utility Analysis ............................... 23
Cost Benefit Analysis ........................... 26
Decision Analysis .............................. 27
Comparison of Output of Each Technique ..................... 29
D iscussion ................................ 31
Utility Analysis ............................... 31
Production Function ............................ 32
Cost Benefit Analysis ........................... 33
Net Benefits Versus the Benefit to Cost Ratio ............. 34
Uncertainty in the Environment .................... .... 35
Evaluating the Flow of Future Returns and Costs ......... ... 35
Valuation of Benefits and Costs ...................... 36
Decision Analysis .............................. 36
Accumulation of Data ......................... 37
V. CONCLUSIONS AND RECOMMENDATIONS .................. 37
REFERENCES .......... .............................. 40
APPENDIX A: SIMULATED DATA FOR FOUR TECHNIQUES ........ 42
APPENDIX B: MODEL SPECIFICATIONS ......................... 47
APPENDIX C: FINAL VALUES FOR UTILITY ...................... 50
Aooession For
-NTIS CPAH GIo
DTIC TAF 0
J . C "' o
By _
D st r - t.jC, orn/
Ava iC0 11 ty Codes
iv D i '11- and/or
iv Dist Spoolal
LIST OF TABLES
Table Page
1 ANOVA Results for Pre-selection Utility Analysis Index ............ 25
2 ANOVA Results for Post-selection Utility Analysis Index ........... 25
3 ANOVA Results for Cost/Benefit Analysis Index .................. 27
4 ANOVA Results for Decision Analysis Index .................... 29
5 Rank Order Correlations .............................. 30
LIST OF FIGURES
Figure Page
1 Cost Benefit Analysis ............................... 15
2 Decision Theory Analysis ................................ 20
V
PREFACE
The work documented in this report is a component of the Quality Force Models unit
of the Manpower and Personnel Division's manpower and force management systems research
and development program. It was accomplished as part of Project 7719, Force Acquisition and
Distribution Systems and Task 771920, Manpower and Personnel Models. The tool will
improve the conduct of manpower, personnel, and training research, and allow personnel
managers and force planners to make more informed resource allocation decisions to achieve the
Air Force's mission.
vi
LIST OF ACRONYMS
AFHRL Air Force Human Resources Laboratory
AFHPSP Armed Forces Health Professions Scholarship Program
AFQT Armed Forces Qualifying Test
AFROTC Air Force Reserve Officer Training Corps
AFS Air Force Specialty
AFSC Air Force Specialty Code
AI Artificial Intelligence
ANOVA Analysis of Variance
ARPTT Air Refueling Part Task Trainer
ASVAB Armed Services Vocational Aptitude Battery
ATC Air Traiiiig Command
BMT Basic Military Trainin-
CAT Computerized Adaptive Testing
CBA Cost Benefit Analysis
DoD Department of Defense
ENJJPT EURO-NATO Joint Jet Pilot Training
FAR Fighter-Attack-Reconnaissance
IDMS Integrated Decision Modeling System
IMIS Integrated Maintenance Information System
JDI Job Difficulty Index
vii
MAC Military Airlift Command
MPT Manpower, Personnel and Training
OJT On-The-Job Training
OTS Officer Training School
PACE Processing and Classification of Enlistees
PJM Person Job Match
PROMIS Procurement Management Information System
R&D Research and Development
RPR Request for Personnel Research
RRC Research Review Council
SMART Simple Multiattribute Rating Technique
SME Subject Matter Expert
TAC Tactical Air Command
TTB Tanker-Transport-Bomber
UDB Unified Data Base
VOICE Vocational Occupational Interest Choice Examination
WAPS Weighted Airman Promotion System
viii