Table Of ContentIAA-00-IAA.I.I.01
IMPROVING THE DISCIPLINE OF
COST ESTIMATION AND ANALYSIS
WILLIAM M. PILAND, DAVID J. PINE,
and DELANO M. WILSON
NASA Langley Research Center
Hampton, Virginia 23681-0001
U.S.A.
51st International Astronautical Congress
2-6 Oct 2000/Rio de Janeiro, Brazil
For permission to copy or republish, contact the International Astronautical Federation
3-5 Rue Mario-Nlkls, 75015 Paris, France
IAA-00-IAA.I.I.01
IMPROVING THE DISCIPLINE OF COST ESTIMATION AND ANALYSIS
William M. Piland*, David J. Pine*, and Delano M. Wilson-*
Independent Program Assessment Office
NASA Langley Research Center
Hampton, Virginia 23681-0001 USA
ABSTRACT in selected, new analysis tools, creating
advanced training opportunities for analysts,
The need to improve the quality and and developing career paths for future
accuracy of cost estimates of proposed new analysts engaged in the discipline are all
aerospace systems has been widely elements of the plan. Plans also include
recognized. The industry has done the best increasing the human resources available to
job of maintaining related capability with conduct independent cost analysis of Agency
improvements in estimation methods and programs during their formulation, to improve
giving appropriate priority to the hiring and near-term capability to conduct economic
training of qualified analysts. Some parts of cost-benefit assessments, to support NASA
Government, and National Aeronautics and management's decision process, and to
Space Administration (NASA) in particular, provide cost analysis results emphasizing
continue to need major improvements in this "full-cost" and "full-life cycle"
area. Recently, NASA recognized that its considerations.
cost estimation and analysis capabilities had
eroded to the point that the ability to provide The Agency cost analysis
timely, reliable estimates was impacting the improvement plan has been approved for
confidence in planning many program implementation starting this calendar 5'ear.
activities. As a result, this year the Agency Adequate financial and human resources are
established a lead role for cost estimation and being made available to accomplish the goals
analysis. The Independent Program of this important effort, and all indications are
Assessment Office located at the Langley that NASA's cost estimation and analysis
Research Center was given this responsibility. core competencies will be substantially
improved within the foreseeable future.
This paper presents the plans for the
newly established role. Described is how the INTRODUCTION
Independent Program Assessment Office,
working with all NASA Centers, NASA Everyone knows that the enemy of approval
Headquarters, other Government agencies, and startup of new programs is the cost
and industry, is focused on creating cost analyst who has accurate estimation tools and
estimation and analysis as a professional a desire to deliver an honest estimate of what
discipline that will be recognized equally with the program will really cost. The estimator is
the technical disciplines needed to design new the person who develops program costs that
space and aeronautics activities. Investments exceeds budget availability, makes the
*Director, Independent Program Assessment Office
*Deputy Director, Independent Program Assessment Office
Senior Cost Analyst, Independent Program Assessment Office
Copyright © 2000 by the American Institute of Aeronautics and Astronautics, Inc. No copyright isasserted inthe
United States under Title 17, U.S. Code. The U.S. Government has a royalty-free license to exercise all rights under the
copyright claimed herein for Governmental purposes. All other rights are reserved by the copyright owner.
program less competitive with other programs other countries. Of particular interest is the
competing for limited resources, and often Intelligent Synthesis Environment Program of
forces the program to focus on objectives NASA Langley (reported at last year's IAF
somewhat less than desirable. Right? No, Congress,) _ with objectives to advance the
that is not exactly the way good management state-of-the-art in near- and far-term
is supposed to work. A responsible program analysis/design tools and promote
or project manager should consider a reliable collaborative engineering among engineering
cost estimate as a resource available for organizations. Providing reliable cost
assuring management success. estimates (the focus of this paper) is a subject
that has received less attention. Reliable cost
Included in the primary reasons why estimation, as a resource to the program
many of today's aerospace program managers manager, has become a scarce commodity, at
eventually find themselves in trouble are: least in NASA. In addition, the image of the
1) Before program approval, they may cost estimator, as the enemy to program
not have adequately defined (systems approval, is wrong. Hopefully, what follows
engineered) the system they planned to will show that NASA recognizes the value
develop and operate. This includes that reliable cost estimation brings to the
developing a full understanding of technical program formulation and approval process.
and programmatic risks that can be barriers to
success, Before turning to how NASA is
2) They may not have developed a correcting deficiencies in its program cost
reliable estimate of what it will cost to estimation capabilities, it may be helpful to
successfully complete the program, and define "cost estimation" as used in the
3) They may accept the job of following discussion. For the purpose of this
managing the program with a budget (and definition, and for other discussions in this
maybe schedule) with little or no relation to paper, "'program" refers to an activity
the expected actual cost of the program. involving the development and operation of a
hardware system, or more specifically
So, what's so difficult? Why can't perhaps, a space system. Cost estimation is
responsible program management correct the process of analyzing each hardware
these deficiencies and more often than not, element, the buildup, integration and test of
deliver the program product within budget these elements, and the operation of the
and on schedule? A primary requirement for system over some specified life-cycle
success is the program manager wanting to be (including disposal of the asset), with respect
"responsible," and a second requirement is the to the cost associated with the total effort.
program manager having the resources to The analysis uses analysis tools, or models,
complete quality, up-front systems that relate hardware elements, complexity,
engineering and to secure a reliable cost and risks of failure to expected costs - a
estimate. parametric analysis. Parametric estimation
involves the development and utilization of
This paper will not attempt to address cost estimation relationships between
the adequacy of systems engineering tools or historical costs and program, physical, and
other resources needed by the program performance characteristics. The basic
manager. Much has been recently written premise is that the cost of a system, such as a
about efforts to improve tools used in systems spacecraft, is related in an approximate, but
engineering, both in the United States and quantifiable way, to a physical characteristic
suchasweight,pointingaccuracy,numberof became obvious that estimators were too
parts,orotherattribute. involved in the advocacy of the programs.
The appearance of objectivity in the CEA
There may be at leasttwo different process was in question.
types of cost estimates, an "'advocacy" 3) Declining workforce led to the
estimateandan"independent"estimate.An establishment of hiring priorities that limited
advocacy estimate may be derived by the replenishment of CEA talent. In addition,
programmanagement,andassuth, maybe limited budgets forced drastic reductions in
skewed in ways beneficial to successful investments that would lead to upgrading the
program advocacy. An independentcost Agency's CEA tools and state-of-the-art
estimateis derivedbyonedisassociatedwith analysis capabilities.
the program,andtherefore,not encumbered
by the pressuresof advocacyandfreeto be In the early 1990s, several outside
objective-"tell it likeit is." Further,thecost advisory groups began to recognize the
estimationandanalysis(CEA)competencyis declined state of the NASA CEA competency.
defined as the total capability of an Of particular importance was the Presidential
organization to provide the costestimates Commission on the Future of Space ", more
required by the organization for budget commonly known as the Augustine
planning and execution, and program Committee. The Commission recommended
planningandapproval. to NASA that "'an exceptionally well-
qualified, independent cost analysis group be
BACKGROUND attached to headquarters with ultimate
responsibility for all top-level cost estimating,
NASA, at one time, maintained a including cost estimates provided outside of
respectable CEA competency. Qualified cost NASA." As a complementary
analysts were employed across the NASA recommendation to foster the independent
Centers with appropriate skills and in assessment of new proposed programs of
numbers to support the Agency's needs. NASA, the Commission also recommended
Several related situations resulted in the the establishment of an independent "'systems
Agency losing much of its recognized concept and analysis group reporting to the
competency: Administrator."
1) The Agency never recognized CEA
as a discipline as important as other It was in the 1993 time period that the
professional disciplines needed for systems Agency formed a cost analysis group in the
engineering and development. There was Headquarters Office of the Chief Financial
never a "career path" available to those Officer. This group was also given the
responsible for CEA, so possibilities for responsibility to organize and conduct
career advancement were always in doubt. Independent Annual Reviews (IAR) of the
As a result, often other career opportunities progress of Agency programs and report
looked more promising and qualified analysts findings to the Agency Program Management
left the work area. Council. This action recognized that
2) Declining budgets, increasing independent cost analysis and the
competition for limited funds, and other development of dependable estimates were of
institutional considerations, tended to lead critical importance to NASA. About 4 years
many parts of the Agency to underestimate later, at the insistence of the NASA Advisory
program development and operations costs. It Committee, the Independent Program
AssessmenOt ffice (IPAO) wasformedatthe thesedeficiencies,budget-driven processes
NASA LangleyResearchCenter. TheIPAO oftenareunsupportedbyCEA.
provides IndependentAssessments(IA) of
new programs, and reports findings and The Federal Office of Management
recommendationsrelatedto the approvalof andBudget(OMB) is pushingfor improved
programsto theOffice of the Administrator. costanalysispriortobudgetsubmission.The
TheresponsibilitytbrtheIndependenAt nnual OMB goals include better schedule and
Reviewswastransferredtothe IPAO atthat
performance goals, more realistic baseline
time. With the formationof the IPAO, the cost, the inclusion of independent cost
cost estimation function previously estimates, and full life-cycle cost, cost benefit,
establishedall but disappeared,exceptfor a and estimate of risk and uncertainty. All
smallnumberof costanalyststhatjoinedthe these cost considerations should be used as a
IPAO atLangleyto continuecostanalysisto basis for selecting future NASA programs.
supportthereviewandassessmenatctivities.
Lastly, in recognition of the above
Duringtheperiodfromthemid-1980s deficiencies, the Administrator recently
to thepresent,for the reasonsstatedearlier, directed improvements in NASA's
therewasagreaterthan50percentattritionin independent cost estimation abilities. His
theNASA CEA competency.At present,the direction resulted in the assignment to the
Agencyof over 18,000peopleemploysless IPAO this year the Agency lead responsibility
than 25 full-time, careercostanalysts. In to correct the deficiencies and restore the
additionto theinability todevelopsufficient Agency's cost estimation and analysis
cost information for managementof its competency. The following discussion details
programs,thereis concerntbr theAgency's the plans to fulfill the new IPAO role.
ability to adhereto Federalstandardsin this
area. Thevariousremainingelementsof the OVERALL STRATEGY
cost community aredecentralizedandhave
ambiguousrolesandresponsibilities.Thereis With the assignment of the lead role
ineffective useandlimited sharingof these for cost estimation and analysis to the IPAO,
resourcesa,ndthereis nocleardefinitionofa the primary responsibility of the Office will
professionacl ostestimationcompetency. be the reconstitution of a comprehensive, core
CEA competency for the Agency. This
The Agency currently conducts responsibility includes the development of a
systemsengineering, budget development, CEA strategic plan for the accomplishment of
and cost estimating separately. There is the role, providing leadership for the
inconsistentuseof costestimatingtechniques establishment of an adequately staffed and
andtools, andthereare outdatedtools and skilled NASA cost estimating community,
cost databases. During program planning, and rallying the community in a coordinated
programrequirementsandrisks to program effort to eliminate the cost estimation
success are not adequately defined, and deficiencies currently plaguing the Agency.
usually, no program life-cycle costs are The IPAO, NASA Headquarters, and the
considered in the approval process. Cost NASA Centers will work together toward
analysistools areinsufficient for estimating implementation of state-of-the-art costing
program cost for programs that are to be capabilities including investing in new and
conductedwith"newwaysofdoingbusiness" upgraded tools, and organizing appropriate
or "faster, better,cheaper." And to addto training. The Office will function as an
independentAgency resourcefor program, 7. Separate development and operational
Center, and Enterprise management by costs estimates will be replaced by integrated,
providing independent cost estimates to full life-cycle estimates.
support program planning and
implementation. The focus of the CEA improvement
initiative will be on _e, tools,
GOALS organization, and processes.
The overall goal of the lead CEA
activity is to prepare NASA for the future by PEOPLE
restoring the ability to develop accurate,
reliable cost estimates of the Agency's The most important resources in ant'
programs and provide confidence for senior activity are the people involved. For success,
leadership that NASA's programs are based the people must be competent, motivated, and
on a solid foundation of understanding cost have the other resources (tools, monet', etc.)
and risk. The new CEA core competency to get the job done. As stated earlier, NASA is
created will involve state-of-the-art business short of properly skilled cost analysis
practices for a full spectrum of cost analysis workforce. Therefore, attention to the
tools and processes for managers and assure "people" factor becomes a first-order priority
integration of cost estimating, systems of the NASA CEA improvement initiative.
engineering and management, and budget
development. Cost estimates will be timely, Adequate compensation is a genuine
and a better understanding of risk and motivator, but it is well known that when one
schedule issues will be developed prior to decides to work in the public sector, there are
program approval. limits to what one can expect in terms of pay.
Therefore, there must be other motivators that
Specific examples of CEA cause an employee to want to make a career
competency improvement goals include in government. With respect to those
making the following changes in the way working in the CEA field, tbr example,
NASA derives its program cost information: expectations of being treated as a valued
employee with definite opportunities for
1. Traditional, non-integrated tools will be professional growth is a true plus.
replaced by state-of-the-art, integrated tools. Unfortunately, in NASA, there has not existed
2. Center oriented cost organizations will much of a professional career path. In fact,
become part of an intra-Agency cost while systems development activities require
community. a wide variety of professional disciplines to
3. Functionally focused cost tasks will implement the program, the activities also
transition to integrated product teams that require CEA, but CEA is not viewed as a
include estimating professionals. discipline in the same way as other areas
4. Non-professional "journeymen" cost (such as the engineering disciplines). A
analysts will be trained and matured into core primary goal of the initiative is help establish
competency cost analysts. CEA as a recognized professional discipline
5. Non-full cost estimates will be substituted with a formal growth ladder and opportunities
with full cost estimates. for senior level positions for individuals
6. Center oriented tools will become shared dedicated to the area. The IPAO will work
Agency tools.
with NASA's humanresourceorganizations TOOLS
tohelpdevelopthispotential.
NASA must be capable of generating
The IPAO will assist the NASA
responsive, reliable, quality cost estimates of
Centersin recruiting newemployeesto the future missions (such as Mars Exploration)
Agency's core competency of CEA. A
that involve the use of new technologies and
databaseof qualified individuals with interest innovative approaches or concepts for
in working for NASA in this area will be satisfying mission objectives. In order to
maintained, and as a Center wishes to achieve this capability, the IPAO will be
supplement or replenish its CEA staff, the responsible for advancing the state-of-the-art
Office will offer appropriate assistance in in cost models and analytical tools. One of
seeking qualified applicants. The Office will the first capabilities identified for immediate
also work with the Agency's career training development is the integration of systems
organizations to help develop CEA-related development cost models with operations cost
training opportunities for the staff. This models. Efforts are underway to integrate the
should include various professional and Space Operations Cost Model (SOCM) 3,
leadership training as well as specific training which is an internally developed model, with
in the CEA discipline to improve the several commercial models such as the
individual's skills. In addition, the IPAO will NASA/Air Force Cost Model (NAFCOM)
help develop Agency-wide on-the-job training and the PRICE Cost Model.
that will improve professional development
while creating opportunities for inter-Center There are on-going discussions within
exposure and cross-fertilization of CEA the CEA community about the requirements
methods among Centers. of the next generation of NASA cost models.
In the near future, NASA management will
Lastly, the IPAO will invest in the require "full cost" estimates, estimates that
development of a college undergraduate include the workforce, general &
course on CEA. It is interesting that in an administrative costs, facility, and
environment of economic pressure, constant program/project costs. Models to estimate
push to reduce cost, and to build systems workforce cost are being developed to meet
'Taster, better, and cheaper," we graduate this requirement along with other
engineers without a notion of how much it methodologies to estimate the full cost of
costs to build the systems they are trained to NASA projects. The NASA Integrated
design. The course will be designed with the Financial Management System, currently
objective of introducing college engineering being developed elsewhere in NASA, will
students to cost analysis, and will be offered assist in providing the CEA discipline with
to any engineering school desiring to improve the full cost accounting data needed to
its engineering curriculum in this regard. If develop full cost estimates. Also, models will
an appropriate one can not be found, there is be required to more accurately determine the
some thought currently being given to the cost impact of new systems development
development of an undergraduate-level text approaches such as "faster, better, cheaper."
on the subject of CEA for the purpose of
aiding in the teaching of the course. The Intelligent Synthesis Environment
(ISE) program is a NASA initiative to
develop a virtual reality design environment.
The goal is an advancement of the simulation
based design environment involving the representing all the cost estimation
integration of designandcostmodelswith
organizations across NASA. This group is
analytical tools using intelligent systems
actively involved in establishing overall goals
technology. As a result of this new of the initiative, in decisions affecting the
environment, the time to develop new system future of the CEA competency, in defining
designs and to estimate the costs will be workforce and analysis tool requirements, and
greatly reduced. IPAO will be collaborating in the implementation of the initiative's
with the ISE program, specifically with the elements. Group members represent the
Cost and Risk Management Technology CEA-related interests of their home Centers,
element of ISE, to develop cost and risk serve to share experiences (or lessons-
models that work within the ISE environment. learned) from cost analysis activities, and
These cost models must be capable of accept complementary responsibilities for
reflecting the revolutionary reduction in the various initiative actions. In addition, the
time and cost of various phases of the design group will facilitate an Agency-oriented CEA
cycle and be state-of the-art tools. Among culture rather than a specific Center-oriented
the analytical tools planned for development culture.
is an improved schedule assessment tool.
The IPAO, in cooperation with the
CEA Steering Group, is organized to serve
IPAO will lead NASA's participation the Agency in several important ways. It is a
in the establishment of joint tool development primary interface with other government
efforts with other government and private agencies to coordinate inter-Agency CEA
industry cost analysis organizations. activities, perform completely independent,
Recently, NASA became a member of the non-advocate cost estimates in support of
newly established Consortium on Space program formulation, and provide other CEA
Technology Estimating Research (COSTER) support to Centers when Center resources are
organization. The COSTER includes most of insufficient. For the CEA initiative, the IPAO
the government agencies with an interest in workforce was increased by eight, and
space. This relationship will likely result in sufficient funds have been provided to secure
joint tool development investments that will contracted CEA analysis to meet support
benefit a broader range of government demands. In addition to the IPAO, it is
organizations. In addition, there will be expected that the Centers' CEA organizations
efforts to assure cost model training will also be appropriately expanded to meet
opportunities are made available to all cost the cost analysis requirements unique to each
analysts. The result of CEA tool investments Center.
will be powerful cost-, schedule-, and risk-
estimating systems that help NASA better The IPAO CEA responsibilities
understand the cost of doing business and include serving as the voice of the Agency's
make the right management decisions. cost estimation and analysis community. The
Office will integrate the fragmented concerns
ORGANIZATION of 10 NASA Centers into focused CEA
community concerns and issues, and this
The IPAO will carry out its CEA consolidation will enable the community's
responsibilities with active participation of all voice to be much stronger than each Center
NASA Centers and Headquarters. The acting separately. It is expected that this will
Agency-wide initiative includes the creation result in more positive results in efforts to
of a CEA Steering Group with members
resolve the overall CEA competency systems development activities of the
deficiencies. Agency. These benefits include the
following:
PROCESSES
1) A new cost estimating culture and
Since one of the responsibilities of the an integrated cost community that
IPAO in the CEA initiative is to oversee the better serves the Agency.
quality control of the Agency's CEA 2) Enhanced costing skill and a
products, the Office will be working to create professional career path for
standards for the discipline and to have all analysts.
Center "buy-in" to the standards. The main 3 )Better definition of systems
locus on creating standards will be to ensure development and operational risks
that CEA processes are consistent and and an estimate of the costs to
conform to the best business practices, mitigate these risks.
provide timely and accurate cost estimates, 4) Reliable, responsive, full life-cycle
and are of maximum value to the Agency,. cost estimates.
The CEA Steering Group will lead in the 5) Cost estimate quality control.
development of the standards, and it is 6) Continual CEA support throughout
expected that all processes will be subject to project life-cycle.
ISO certification. Included in the processes 7 )Databases for improved cost
will be guidelines for cost analysts' continued estimating and cross program
participation through the project's life and analysis.
being involved in the creation of full-cost 8) State-of-the-art, user friendly tools
estimates of the system development, and processes that accommodate
operation, and disposal. A closer relationship NASA's new ways of conducting
between those responsible for developing systems development activities.
systems requirements and those responsible 9) Make the Agency more OMB-
for estimating requirement's cost must be compliant.
developed. The full integration of CEA into 10)Better fiscal support and budget
design activities is necessary. Cost analysts defense
must be active participants in initial system
trade studies that should be conducted to The IPAO, in its lead role for CEA,
derive the system approach that achieves the will function as an independent Agency-level
requirements in the most cost-effective way. resource and will ensure all Centers are
In today's program management world, the involved in the reconstitution of this most
cost analyst must remain an active, but important Agency discipline. The bottom line
objective member of the team throughout the is that after these changes are instituted, both
entire life-cycle of the program. the Agency and those in charge of NASA's
appropriations will have a much greater
confidence that our proposed costs will also
SUMMARY be our actual costs. This will make our
budgets more defensible, leading to better
The IPAO led CEA initiative will Congressional support.
focus on up-front planning and continued
improvement of the NASA CEA competency
and will provide key benefits to the future
REFERENCES
1. Monell, Donald W. and Piland. William
M., "'Aerospace Systems Design in NASA's
Collaborative Engineering Environment,"
IAF Paper 99.U.1.01, October 1999.
2. Augustine, N., "'Report of the Advisory
Committee on the Future of the U.S. Space
Program," 1990.
3. Moore, Arlene, Proceedings of the
International Society of Parametric Analysts,
"Parametric Prospectives in the Information
Age," New Orleans, May 1997.
