NASA Technical Reports Server (NTRS) 20040110898: Formalizing Space Shuttle Software Requirements

9 Pages·0.29 MB·English

by  NASA Technical Reports Server (NTRS)

Formalizing Space Shuttle Software Requirements
To be presentedat theACM SIGSOFT Workshopon Formal Methods inSoftware Practice, San Diego, CA, January 1996
Judith Crow Ben L. Di Vito
Computer Science Laboratory V(cid:19)(cid:16)GYAN, Inc.
SRI International 30 Research Drive
Menlo Park, CA 94025 Hampton, VA 23666-1325
[email protected] [email protected]
Abstract ralSpace InformationSystems (formerlyIBM,Houston)
participated as a subcontractor for JSC.
This paper describes two case studies in which require- This paper focuses exclusively on LaRC project
ments for new (cid:13)ight-software subsystems on NASA’s activity1, which was performed in the context of a
Space Shuttle were analyzed, one using standard for- broader program of formal methods work [1]. The ef-
mal speci(cid:12)cation techniques, the other using state ex- fort consisted of formalizing selected Shuttle software
ploration. These applications serve to illustrate three (sub)system modi(cid:12)cations and analyzing key system
main theses: (1) formal methods can complement con- properties using either SRI’s PVS speci(cid:12)cation lan-
ventional requirements analysis processes e(cid:11)ectively, (2) guage and interactive proof-checker [14], or Stanford’s
formal methods confer bene(cid:12)ts regardless of how exten- Mur(cid:30) (pronounced \Murphy") (cid:12)nite-state veri(cid:12)cation
sivelytheyareadoptedandapplied,and(3)formalmeth- system [4, 8]. The LaRC work had three main goals.
ods are most e(cid:11)ective when they arejudiciously tailored First, to explore and document the feasibility of formal-
to the application. izingcriticalShuttlesoftwarerequirementsrepresentinga
spectrum ofmaturitylevels. Second,todevelopreusable
formal methods strategies for representative classes of
1 Introduction
Shuttle software. Third, to identify and assess key fac-
tors in the transfer of this technology to JSC and Loral.
Although Space Shuttle (cid:13)ight software is generally con-
TheShuttlesubsystemsselectedfortheLaRCprojects
sidered exemplary among NASA software development
directly re(cid:13)ect the (cid:12)rst two goals. JS, 3E/O, and GPS
projects, requirements analysis and quality assurance in
are all part of critical on-board (cid:13)ight software. The JS
early lifecycle phases still use products and tools dating
requirements are mature and stable, the 3E/O require-
from the late 1970s and early 1980s. As a result, these
ments are somewhat newer, having only recently stabi-
analysis and assurance activities remain largely manual
lized after a series of iterations, and the GPS require-
exercises lackingwell-de(cid:12)ned methods or techniques. At
ments are quite new and still in (cid:13)ux. JS and GPS be-
the same time, Shuttle (cid:13)ight software is life-critical and
long to a class of Shuttle softwarethat is readilyformal-
increasingly complex. Software upgrades to accommo-
ized using a functional model of computation, basically
date new missions such as the recent MIR docking, new
a control function augmented with state variables, and
capabilities such as Global Positioning System (GPS)
e(cid:11)ectively veri(cid:12)ed using standard theorem-provingtech-
navigation, and improved algorithms such as the newly
niques. By contrast, 3E/O represents a class of mode-
automated three-engine-out contingency abort maneu-
sequencingsoftwarethat canbe quite naturallymodeled
vers (3E/O) and the recent optimization of Reaction
with (cid:12)nite-state systems and e(cid:11)ectively veri(cid:12)ed using
Control System Jet Selection (JS) are continually intro-
state exploration. We have developed general strategies
duced. Such upgrades underscore the need recognized
for specifying and verifying these two classes of Shut-
in the NASA community and in a recent assessment of
tle software and have demonstrated their utility in the
Shuttle (cid:13)ight softwaredevelopment,for\state-of-the-art
JS [11, Appendix B], 3E/O [2], and GPS [3] projects.
technology" and \leading-edge methodologies" to meet
The technical approach for JS is essentially the same as
the demands of software development for increasingly
thatusedforGPS,althoughthestrategywasre(cid:12)nedand
large and complex systems [12, p. 91].
documented as part of the GPS study. Accordingly, we
Over the last three years, NASA Langley Research
present only GPS to illustrate the approach. Although
Center (LaRC) and its subcontractors, V(cid:19)(cid:16)GYAN and
SRI,haveinvestigatedtheuseofformalmethods(FM)in 1DescriptionsofsomeoftheJPL,JSC,andLoralactivitiesun-
dertaken for this project can be found in [5, 6, 9]. The overall
aerospace applications, as part of a three-center demon-
project is not large, roughly the equivalent of one full-time posi-
strationprojectinvolvingLaRC,theJetPropulsionLab- tion at each of the three NASAcenters per year, including Loral,
oratory(JPL),andtheJohnsonSpaceCenter(JSC).Lo- V(cid:19)(cid:16)GYAN,andSRItime.
1
PVS and Mur(cid:30) were used for this work, the strategies thorough reviews of new CRs, analyzing them with re-
are applicable to other systems with equivalent power specttocorrectness,implementability,andtestabilitybe-
and functionality. fore turning them over to the development team. Their
Thekeytechnical resultsofthe projectinclude aclear objective is to identify and correct problems in the re-
demonstrationoftheutilityofformalmethodsasacom- quirements analysisphase, avoidingfarmorecostly(cid:12)xes
plement to the conventional Shuttle requirements analy- later in the lifecycle.
sisprocess. TheJS,3E/O,andGPSprojectseachuncov-
ered anomalies ranging from minor to substantive, most
2.2 Summary of PVS and Mur(cid:30)
of which were undetected by existing requirements anal-
ysis processes. A further result is a comparativedemon- PVS (Prototype Veri(cid:12)cation System) is an environment
stration of the di(cid:11)erent roles formal methods can play, for speci(cid:12)cation and veri(cid:12)cation developed at SRI Inter-
underscoringthefollowingtwoprecepts: formalmethods national’s Computer Science Laboratory [14]. The dis-
conferbene(cid:12)tsregardlessofhowextensivelytheyareap- tinguishing characteristic of PVS is a highly expressive
plied, and formal methods are most e(cid:11)ective when they speci(cid:12)cation language coupled with a very e(cid:11)ective in-
are judiciously tailored to the application. teractive theorem prover that uses decision procedures
to automate most of the low-level proof steps. Mur(cid:30)
is a fully automatic state exploration tool developed by
2 Formalization Strategies
David Dill and his students at Stanford University that
uses e(cid:14)cient encodings, including symmetry-basedtech-
In this section we discuss two general strategies and il-
niques, and e(cid:11)ective hash-table strategies to do \reach-
lustrate their application to the GPS and 3E/Orequire-
ability analysis," i.e., to check that all reachable states
ments,beginningwithasketchofShuttlesoftwaredevel-
satisfy speci(cid:12)ed properties [4, 8]. Mur(cid:30) can explore mil-
opment and a brief description of PVS and Mur(cid:30).
lions of states in a matter of minutes.
2.1 Shuttle Software Background
2.3 Functional Speci(cid:12)cation
NASA’s prime contractor for the Space Shuttle is the
Space SystemsDivision of Rockwell International. Loral As one of the larger ongoing Shuttle CRs, the GPS CR
Space Information Systems (formerly IBM Federal Sys- providesasigni(cid:12)cantupgradetotheShuttle’snavigation
tems, Houston) is their software subcontractor. Draper capability. A portion of this CR has been formalized
LaboratoryalsoservesRockwell,providingrequirements usingthefunctionalspeci(cid:12)cationtechnique. Afterabrief
expertise in guidance, navigation and control. overviewof GPS, we develop the abstract state machine
Much of the Shuttle software is organized into ma- model used and describe the results of this application.
jor units called principal functions, each of which may
be subdivided into subfunctions. Since the late 1970s, 2.3.1 Overview of GPS
software requirements have been written using venera-
ble conventions known as Functional Subsystem Soft- The GPS retro(cid:12)t was planned in anticipation of the
ware Requirements (FSSRs) | low-level software re- DoD’sphaseoutoftheTACANnavigationsystem. Shut-
quirements speci(cid:12)cations written in English prose, pre- tle vehicles will be out(cid:12)tted with GPS receivers and ad-
sented with strong implementation biases, and accom- ditionalnavigationsoftwarewillbeincorporatedintothe
panied by pseudo-code, and diagrams and (cid:13)owcharts in Shuttle to process the position and velocity vectors gen-
arcane notations. Interfaces between software units are erated by these receivers. Currently, the Shuttle naviga-
speci(cid:12)ed in input-output tables. Inputs can be variables tionsystemcanacceptstatevectorupdatesderivedfrom
or one of three types of constant data: I-loads ((cid:12)xed for ground-basedradarobservations. The Shuttle GPSsoft-
the current mission), K-loads ((cid:12)xed for a series of mis- wareCRwilladaptthisfeature,providingthecapability
sions), and physical constants (never changed). to update the Shuttle navigation (cid:12)lter states with se-
Shuttlesoftwaremodi(cid:12)cationsarepackagedasChange lectedGPSstatevectorestimatessimilartothewaystate
Requests (CRs), that are typically modest in scope, lo- vector updates are currently accepted from the ground.
calizedinfunction, andintendedtosatisfyspeci(cid:12)cneeds TheGPStrialformalizationfocusedonafewkeyareas
for upcoming missions. Roughly once a year, software becausetheCRisverylargeandcomplex. Afterprelimi-
releases called Operational Increments (OIs) are issued narystudyoftheCRanddiscussionswiththeGPSRAs,
incorporatingoneormoreCRs. Shuttle CRsarewritten we decided to concentrate on two new principal func-
as modi(cid:12)cations, replacements, or additions to existing tions, emphasizingtheir interfacesto existingnavigation
FSSRs.2 Loral Requirements Analysts (RAs) conduct software and excluding crew display functions. The two
principal functions, known as GPS Receiver State Pro-
2The JS requirements are roughly 70 pages of tables and low-
cessing and GPS Reference State Processing, select and
level diagrams, 3E/O is roughly 70 pages of prose, pseudo-code,
(cid:13)owcharts,andtables,andtheGPSsubsetundertakenhereisap- modify GPS state vectors for consumption by the exist-
proximately110pagesofprose,pseudo-code,andtables. ing entry navigation software.
2
2.3.2 Technical Approach
pf_result: TYPE = [# output: pf_outputs,
We have devised a strategy to model Shuttle princi- state: pf_state #]
pal functions based on the use of a conventional ab-
stract state machine model. Each principal function is principal_function
modeled as a state machine that takes inputs and local (pf_inputs, pf_state,
state values, and produces outputs and new state val- pf_I_loads, pf_K_loads,
ues. Thismethodprovidesasimplecomputationalmodel pf_constants) : pf_result =
similar to popular state-based methods such as the A-7
model [7, 17]. (# output := <output expression>,
Onetransitionofthestatemachinemodelcorresponds state := <next-state expression>
toonescheduledexecutionoftheprincipalfunction,e.g., #)
one cycle at rate 6.25 Hz or other applicable rate. All of
theinputstotheprincipalfunctionarebundledtogether
Figure 1: PVS model of a Shuttle principal function.
and a similar bundling of the outputs is arranged. The
state variable holds values that are (usually) not deliv-
ered to other units, but instead are held for use on the contains an output component and a next-state compo-
next cycle. nent. Each of these objects is, in turn, a structure con-
The state machine transition function is a mathemat- taining (possibly many) subcomponents.
ically well-de(cid:12)ned function that takes a vector of input The output and next-state expressions in the general
values and a vector of previous-state values, and maps form above describe the e(cid:11)ects of invoking the subfunc-
them into a vector of outputs and a vector of next-state tions belonging to the principal function. In practice,
values. this can be very complicated so a stylized method of or-
ganizingthisinformationhasbeen devised,basedon the
M :I (cid:2)S ![O(cid:2)S]
useofaLETexpressiontointroducevariablenamescor-
This function M is expressedin PVSand formsthe cen- responding to the intermediate inputs and outputs ex-
tralpartoftheformalspeci(cid:12)cation. Weconstructatuple changed among subfunctions.
composedof the output andstate valuessoonly a single In the CR, GPS Receiver State Processing is decom-
top-level function is needed in the formalization. Some posed into six subfunctions, and GPS Reference State
values may appear in both the output list and the next- Processing into four. In several cases the subfunction
state vector. requirements were su(cid:14)ciently complex that it became
WhilethefunctionM capturesthefunctionalityofthe necessarytointroduceintermediatePVSfunctionstode-
softwaresubsysteminquestion,thestatemachineframe- compose the formalization further. While this is a rea-
workcanalsoservetoformalizeabstractpropertiesabout sonablestrategy,itdoescausesomelossoftraceabilityto
the behavior of the subsystem. The common approach the original requirements. Clarity and readability were
of writing assertions about traces or sequences of input judged more important, however, and such decomposi-
and output vectors is easily accommodated. For exam- tions wereintroduced as needed. A consistent decompo-
ple, we can introduce sequences I(n) = < i1;:::;in > sition scheme helped make the use of such intermediates
and O(n) = < o1;:::;on > to denote the (cid:13)ow of inputs as transparent as possible.
and outputs that would have occurred if the state ma- The two GPS principal functions were formalized in
chine were run for n transitions. A property about the about 3300 lines of PVS speci(cid:12)cations (including com-
behaviorof M can be expressed as a relation P between mentsandblanklines),packagedaselevenPVStheories.
I(n) and O(n) and formally established, i.e., we prove Writing the original version and performing three revi-
that the property P does indeed follow from the formal sions to track requirements changes took an estimated
speci(cid:12)cation M using the PVS proof-checker. twosta(cid:11)monthsofe(cid:11)ortoverafour-monthperiod. This
Figure1showstheabstractstructureofaShuttleprin- period followed an earlier round of familiarization with
cipalfunctionrenderedinPVSnotation. Keyfeaturesof the CR.
this structure are the two kinds of variable data (input
values,previous-statevalues)andthreekindsofconstant
2.3.3 GPS Results
data(I-loads,K-loads,constants)usedasarguments,and
the result returned containing both output values (pro- Experience with the GPS e(cid:11)ort showedthat the outlook
ducedforotherprincipalfunctions)andnext-statevalues forformalmethodsinrequirementsanalysisispromising,
(used by this principal function on the next cycle). but it is still too early to declare victory. The GPS re-
The PVSde(cid:12)nition assumes all input and statevalues quirementswerestillconvergingatthetimeofthisstudy,
have been collected into the structures pf inputs and andastartduringalaterrevisioncyclewouldhavemade
pf state. Additionally, all I-load, K-load, and constant the FM results more meaningful. Stable requirements
inputs used by the principal function are collected into were not expected until late in 1995.
similar structures. The pf result type is a record that PVScanandhasbeenusede(cid:11)ectivelytoformalizethis
3
application; thecustomspeci(cid:12)cationapproachshouldbe Issue Severity With FM Existing
easytoduplicateforotherareas,yieldinggoodprospects High Major 2 0
for continuation of these e(cid:11)orts. Although the speci(cid:12)ca- Low Major 5 1
tion activity was assisted by tools, manual speci(cid:12)cation High Minor 17 3
is also feasible here, albeit with reduced bene(cid:12)ts. Low Minor 6 0
Some Shuttle RAs are optimistic about the potential Totals 30 4
impact of FM. Their feedback indicated our approach
was helpful in detecting three classes of errors normally While most of these problems were of the minor Type 9
tracked by the Shuttle program: variety, a few were more serious. Since errors escaping
detectionduringthisphasewillbemorecostlytocorrect
1. Type 4 | requirements do not meet CR author’s during development or system test, our results suggest
intent. that the added precision of formalization used early in
the lifecycle can yield tangible bene(cid:12)ts.
2. Type6|requirementsnottechnicallyclear,under-
Although many of these issues could have been found
standable and maintainable.
with lighter-weight techniques, the use of formal speci-
3. Type 9 | interfaces inconsistent. (cid:12)cations can detect errors and leave open the option of
deductive analysis later on. When we reach the point of
AnexampleofType4errorsencounteredintheCRis
modelinghigherlevelpropertiesandcarryingoutproofs,
omission due to conditionally updating variables. Sup-
weexpect toseefewererrors. Lightweightformsofanal-
pose, for example, one branch of a conditional assigns
ysisappliedearlydetect moreproblemsanddetect them
several variables, leaving them unassigned on the other
quickly, but the errors tend to be super(cid:12)cial. As more
branch. The requirements author intends for the values
powerful analysis methods are introduced, we expect to
tobe\don’tcares"intheotherbranch,but occasionally
(cid:12)nd fewer, but more subtle problems.
this is faulty because some variables such as (cid:13)ags need
The next stepin theapplicationofFM toGPS,which
tobe assignedinbothcases. Similarproblemscanoccur
hadbeendeferredasofthiswriting,istoidentifyandfor-
with overlappingconditions which cause ambiguity with
malize importantbehavioralpropertiesofthe processing
respect to correct variable assignments.
ofGPSpositionandvelocityvectors. Provingtheseprop-
Examples of Type 9 errors, the most prevalent type
erties hold will o(cid:11)er a powerful means of further shak-
encountered, include numerous, minor cases of incom-
ing out the requirements before (and even after) passing
pleteandinconsistentinterfaces. Missinginputsandout-
them on to development.
putsfromtables,mismatchesacrosstables,inappropriate
types, and incorrect names are all typical errors seen in
the subfunction and principal function interfaces. Most 2.4 Mode-Sequencing Speci(cid:12)cation
are problems that could be avoided through greater use
of automation in the requirements capture process. The primary purpose of the 3E/O CR is to automate
It is worth noting that most errors detected in the and upgrade the 3E/O contingency guidance function.
CR during the formalization were not exposed by type- Followingabriefoverviewof3E/O,wedescribethe(cid:12)nite-
checking or other automated analysis activity, but were statemachinemodelusedtospecifythemode-sequencing
found duringthe actofwritingthe speci(cid:12)cationsor dur- component of 3E/O and summarize the results of the
ingthe reviewand preparationleadingup tothewriting analysis.
step. Attempting to write a PVS construct and look-
ing around for the declared objects it needs e(cid:11)ectively
2.4.1 Overview of 3E/O
(cid:13)ushed out many problems. On the other hand, there
were also cases where additional errors were found dur- 3E/O is responsible for monitoring ascent parameters
ingthetypecheckingphase. Forexample,interfaceprob- and, if three Shuttle main engines fail sequentially or
lems (Type 9) usually surfaced during the act of speci- simultaneously, calculating and commanding the appro-
(cid:12)cation writing, while more serious problems stemming priateabortmaneuverifanabortisnecessary. Incertain
from logic errors or inappropriate operations (Types 4 situations,3E/Oisalsoresponsibleforautomaticcontin-
and 6) often persisted to the typechecking stage, where gency maneuvers resulting from the failure of two Shut-
type mismatches wouldrevealanerrorsuchasamissing tlemainengines(2E/O).3E/Oisexecutedrepeatedlyat
subscript. speci(cid:12)ed intervals that range from 1.92 seconds to 0.16
A preliminary comparisonwith the conventionalanal- seconds; each execution is part of a guidance cycle that
ysis process is revealing. By considering errors detected remains active during powered (cid:13)ight until either a con-
inReferenceStateProcessingduringthe(cid:12)rstpassofthe tingencyabortis required orprogressalongthe powered
FM-assistedanalysisversusthesubsetofthoseerrorsalso (cid:13)ight trajectoryis su(cid:14)cient to preclude an abort even if
foundbythe currentprocess,a bitofanecdotalevidence three main engines fail.
emergesfor whatcanbemissedbyamanualreviewpro- 3E/O consists of two main functions: a region-
cess: selectfunctionthatselectsacontingency-maneuvermode
4
based on the values of ascent parameters, and a contin- to verify a particular sequence that (partially) de-
gency guidance function that is used strictly for display termines assignment of abort maneuver regions, it
if the ascent is normal, but is responsible for calculating is su(cid:14)cient to check whether the current altitude
and commanding initial abort maneuversdetermined by and altitude rate predict an apogee altitude greater
the selected region if a contingency arises. The maneu- or less than the calculated altitude-velocity curve.
vers calculated and commanded by the two 3E/O func- Since the exact values or physical laws involved are
tions may di(cid:11)er from one guidance cycle to the next in irrelevant for verifying the crucial sequencing prop-
responsetochangesintheexternalenvironmentorinthe erties, a simple boolean-valued check su(cid:14)ces. Simi-
Shuttle’s internal state. larly,toverifyanothersequencethatalso(partially)
determinesassignmentofabortmaneuverregions,it
2.4.2 Technical Approach is necessary to check that the inertial velocity falls
within certain I-loaded thresholds. Since the exact
3E/Oistypicalofmostfault-handlinglogicinthefollow- inertial velocity is not a factor (in this case or any
ing respects: it consists largely of mode switching and other), there is no need to represent a continuous
exception handling, it exhibits virtually no algorithmic range of values; the sequencing property can be es-
complexity, and its input and state spaces lack a reg- tablished with respect to a qualitative range that
ular and easily characterizable structure. Due largely re(cid:13)ects the set of possibilities de(cid:12)ned by the (cid:12)xed
to these three characteristics, conventional speci(cid:12)cation thresholds.
and veri(cid:12)cation strategies that use type checking and
theorem proving to establish correctness of functional 2. Constraints on the simultaneous values assumed by
requirements are not e(cid:11)ective validation methods for variables representing physical parameters can be ig-
fault-handling applications. The simplest way to vali- nored. Forexample,wemakenoattempttocapture
date fault-handling systems is to enumerate the entire therelationbetweenvelocityandaltitude. Although
inputandstatespacesbybruteforce. Whilethisisrarely this is clearly naive, it is also overly general; while
practical|the state space of most applications of inter- wemayconsidertoomanycases,wedonotoverlook
5
estis fartoogreattopermitexhaustiveenumeration|it any.
is often possible to \downscale" the state space of an
3. The implicit notion of time inherent in an ordered
application to a reasonably small, (cid:12)nite size, while re-
sequence of events is su(cid:14)cient. No further or more
taining essential behaviors of the original system. The
explicit representation of time is necessary for ana-
downscaledoraggressivelyabstractedsystemcanthenbe
lyzing 3E/O sequencing properties.
analyzed e(cid:11)ectively using state exploration, as we have
donewith 3E/O.Inbroadterms,the technicalapproach
We further reduce the large number of inputs by ex-
used for 3E/O has been to develop an abstract model
ploiting the fact that a boolean-valuedoperation on two
thatwouldenableustoencodethebasicsequencingcon-
inputs is equivalent, as a sequencing constraint, to a
straints as transitions in a (cid:12)nite-state machine and to
simple boolean variable. For example, in 3E/O, the
specify keypropertiesofthe 3E/Osequencingalgorithm
boolean expression used to determine if the Shuttle has
as invariants that must hold in all reachable states. We
su(cid:14)cient range for a particular type of abort maneu-
elaborate key elements of this approach below.
ver checks two conditions: is the down-range horizon-
The challenge in modeling Shuttle (cid:13)ight softwaretyp-
3 tal earth-relative velocity strictly less than 0, and is the
ically derives from the number of input variables and
di(cid:11)erence between the predicted and actual range ca-
their inherent complexity. The strategy we developed
pability strictly greater than an I-loaded minimum ac-
de(cid:12)nes astatetransition system operatingwithin atwo-
ceptable range di(cid:11)erence, i.e., v horiz dnrng < 0 AND
level context: a global environment consisting of vari-
delta r > del r usp. The value of the operation in
ables representing sampled sensor values for external
eachconjunctiseithertrueorfalse,andindistinguishable
physical parameters (e.g., current dynamic pressure),
from a single boolean-valued variable; as a sequencing
andalocalenvironmentconsistingofvariablesrepresent-
4 constraint, this conjunction is equivalent to the expres-
ing the Shuttle’s internal status. To provide a reason-
sion: v horiz dnrng LT 0 AND delta r GTR del r usp,
ablytractableandaccuratemodeloftheinputspace,we
whereeachconjunct isreducedtoasimple booleanvari-
make the following simplifying assumptions.
able. By universallyquantifying overthese variables,we
1. The largely continuous values representing the Shut- e(cid:11)ectively show that for all possible values of the two
tle’s physical environment can be modeled using ei- (original) expressions, certain properties hold. We use
ther qualitative ranges or booleans. For example, this strategy for all inputs that represent the Shuttle’s
physical environment.
3The 3E/O requirements document contains six full, double-
spacedpagesofinputs,mostofwhichrepresentI-loadedthresholds 5In the context of (cid:12)nite-state veri(cid:12)cation, a technique which
usedtocalculatetheorderandtimingofthemaneuversequences. prides itself on being able to handle very large (but (cid:12)nite) state
4This strategy is reminiscentof the standard A-7 approach [7, spaces,itisfarbettertoconsidertoomanypossibilities,thantoo
17],but di(cid:11)ersinthewaytheenvironmentismodeled. few.
5
rata)CR.Interestingly,manyoftheissueswefoundmost
Ruleset <external physical parameter 1>: TYPE
compelling werenot of interest to the RA. For example,
Do
. we discovered a sub-optimal sequence in which an entry
.
. maneuverispotentiallycalculatedtwicebecauseatestis
executed after rather than before the calculation. How-
Ruleset <external physical parameter n>: TYPE
ever,sincetheShuttlecurrentlyusesonlyaround50%of
Do
the available compute cycles, the anomaly was not con-
sidered important. The logical error listed below repre-
Rule "<rulename>"
sentsasigni(cid:12)canterrorintherequirementsthatwasalso
. discoveredbythe existingrequirementsanalysisprocess.
.
. To our knowledge, the other issues listed below had not
been previously discovered.
Rule "<rulename>"
ErrorType Number
Endruleset;
undocumented assumption 3
. inconsistent or anomalous terminology 10
.
. redundant calculation 2
missing initialization 1
Endruleset;
interface anomaly 2
logical error 1
Figure2: Mur(cid:30)AbstractSyntaxUsedtoModelShuttle’s
3 Issues and Conclusions
Physical Environment.
In these concluding remarks, we touch on the issue of
Figure 2 shows a template for the Mur(cid:30) rules used to
technology transfer and return to an earlier theme: the
generate input values for external physical parameters.
versatility of formal methods, using the 3E/O, JS, and
The Ruleset construct is syntactic sugar that generates
GPS studies to illustrate the various roles and concomi-
a copy of the rules within its scope for every value of
tant bene(cid:12)ts available through the judicious application
the bound variable. Inputs that represent the Shuttle’s
of formal techniques.
internal state, e.g., the (cid:13)ag that indicates whether con-
tingency 3E/O has been activated or the variable that
encodes whether main engine cuto(cid:11) has occurred, are 3.1 Technology Transfer
modeled as independent inputs using simple nondeter-
One aspect of technology transfer, demonstrating feasi-
ministic rules that generate all possible (combinations
bility on important applications, has been addressed by
of) values in the input space.
the results of the case studies. Other issues pertain to
Finally,wemodelbasicmodesequencingpropertiesas
the eventual uptake of formal methods by the aerospace
state transition constraints and then show that if these
industry. Workingdirectlywithindustryonthesestudies
constraintsaresatis(cid:12)ed,keypropertiesofthealgorithms
hasbeenhelpful forallsidesandhasledtothefollowing
also hold. The key properties arespeci(cid:12)ed as invariants,
observations.
i.e.,expressionsthatmustbetrueinallreachablestates.
Forexample,weuseaninvarianttocheckthatalloutputs
persisting from the previous cycle remain correct with (cid:15) Perceived bene(cid:12)ts: Shuttle RAs are generally in-
respect to conditions in the current cycle. clined to believe in the bene(cid:12)ts of formalism and
Employingthe general strategy outlined here, the two need only modest demonstrations such as GPS and
main 3E/O functions were speci(cid:12)ed in approximately 3E/O to convince them. RAs have long felt they
1200 lines of Mur(cid:30) code including comments, using ap- lack adequately precise, mechanizable ways to ex-
proximately a dozen invariants. press and analyze complex requirements.
(cid:15) Cost e(cid:11)ectiveness: Our case studies, largely carried
2.4.3 3E/O Results
out by experienced practitioners, showed that lim-
Our experience has been that the process of formalizing ited, tailoreduseofFMappearstobecoste(cid:11)ective.
and analyzing requirements invariably exposes undocu- Largeprojects,however,havearealandunderstand-
mented assumptions, inconsistent and imprecise termi- able need for quanti(cid:12)able cost predictions. This re-
nology,redundantcalculations,missinginitialization,in- mains an unmet need and could be an obstacle to
terfaceanomalies, andlogicalerrors. Of the issuestabu- convincing managers to adopt FM. The cost e(cid:11)ec-
lated below and reported to the 3E/ORA, roughly one- tiveness of FM in the hands of novice users is still
third will appear in an upcoming Documentation (Er- an open question.
6
(cid:15) Reading speci(cid:12)cations: PVS providesa formalspec- the multi-center team over several months and re-
i(cid:12)cation language of considerable power while still liedextensivelyonresidentexpertiseatLoral. How-
preserving the syntactic (cid:13)avor of modern program- ever, when LaRC formalized the high-level JS re-
ming languages. As a result, PVS speci(cid:12)cations are quirements in approximately 500 lines of PVS, it
quite readable by those with little prior exposure became clear that the JS function is basically very
to PVS. The GPS RAs were able to read and un- simpleand,withthehelpoftheformalspeci(cid:12)cation,
derstand the PVS speci(cid:12)cations without becoming could probably be communicated in less than a day
PVS practitioners, corroborating a similar experi- to those with no prior JS exposure.
ence in the AAMP5 project jointly undertaken by
SRI and Collins Commercial Avionics [10], where (cid:15) Articulate Implicit Assumptions: Formalisms can
the Collins engineers quickly became adept at read- help identify and supplement limitations in require-
ing PVS speci(cid:12)cations of the AAMP5. ments methodology. The concept of state variables,
for example, is not explicitly mentioned in FSSR-
(cid:15) Writing speci(cid:12)cations: RAs have had less practice style requirements. Their existence must be in-
writing formal speci(cid:12)cations, and still feel uncom- ferred from context by noting when local variables
fortable with it. Developing the skill set and expe- appear to be persistent. Explicitly modeling these
rience necessary to write formal speci(cid:12)cations will state variables has made the augmented GPS re-
requiremoreattentioninasecondroundoftechnol- quirements clearer and more precise. Similarly, us-
ogytransfer. Nevertheless,wearehighlyencouraged ingMur(cid:30)toexplorethe cyclicbehaviorof3E/Oex-
byourexperienceatarecenttrainingcourseo(cid:11)ered posed undocumented assumptions about the input
byNASA LaRC, wherethe participants(LoralRAs space well before we introduced invariants.
and JSC personnel) all speci(cid:12)ed and proved several
problems drawn from aerospaceapplications. (cid:15) Expose Flaws: Although the GPS project has not
yet undertaken any proofs, it has exposed a signi(cid:12)-
(cid:15) Veri(cid:12)cation: Not enough formal veri(cid:12)cation was cantnumberof (cid:13)awsin the requirements,especially
performed by RAs in the case studies to draw valid among the subsystem interfaces. Applications like
conclusions. We anticipate a signi(cid:12)cant learning GPSthatinvolvelarge,complexsystemsandimma-
curveinthisarea,althoughthetrainingcoursecited ture requirements, can expect to realize substantial
above was encouraging. bene(cid:12)t from formal speci(cid:12)cation alone. Similarly,
state exploration of the 3E/Ospeci(cid:12)cation revealed
Overall,theoutlookforindustrialadoptionofthetype
several potential issues even before introducing in-
of formal methods we explored is promising. Contin-
variants.
uedinsertionprojectsareneededtodevelopin-houseex-
pertise in selected companies. The best strategy is to Given a tool like PVS that o(cid:11)ers a highly expres-
team experienced practitioners with motivated applica- sive languageand strong typecheckingthat is inherently
tionareaexpertsandtotackleafeasibleportionofareal undecidable, the distinction between speci(cid:12)cation only
application. and speci(cid:12)cation with proof of simple properties is some-
whatblurredbecausespeci(cid:12)cationscannotbeconsidered
type-correct until all type-correctness obligations have
3.2 Formal Methods: Roles and Bene(cid:12)ts
been discharged in the prover. Nevertheless, challenging
The role of formal methods on a project can vary along aspeci(cid:12)cationbyprovingpropertieswithaproof-checker
several dimensions. We consider (cid:12)rst the variation con- and checking invariants through state explorationrepre-
ferred by the di(cid:11)erent levels of formal methods analysis sent a di(cid:11)erent level of activity that providesfar greater
available within a single technique or tool, the (cid:12)rst level assurances than speci(cid:12)cation alone, as noted in [16]. At
being speci(cid:12)cation only. Although a speci(cid:12)cation that this level, the bene(cid:12)ts include:
has not been validated through proof can be aptly com-
pared to a program that has not been debugged, there (cid:15) Con(cid:12)rm/Discon(cid:12)rm Key Properties: System prop-
are nevertheless real bene(cid:12)ts to be gained from model- erties or constraints can be precisely stated and de-
ing and formally specifying requirements, including the ductively veri(cid:12)ed. The JS speci(cid:12)cation was vali-
following. dated by proving a dozen lemmas derived from a
listofexpectedJSproperties. ThepropertythatJS
(cid:15) Clarify Requirements: A formal speci(cid:12)cation pro- shallneverchooseafailedjetprovedtobefalse(i.e.,
vides a concise and unambiguous statement of the thecorrespondinglemmawasunprovable),exposing
underlying requirements, thereby exposing funda- a genuine problem in mature requirements.
mental issues that tend to be obscured by lengthy
informal statements. Our experience with JS illus- (cid:15) Debug Speci(cid:12)cations: Less signi(cid:12)cant properties
trates this point nicely. Decoding the JS require- that ought to be true about a speci(cid:12)cation can be
ments documents required the combined e(cid:11)orts of statedaschallengesandproved. When suchaproof
7
fails, it can be due to a real problem in the require- alsoliketothankJohnKelly(JPL),JohnRushby(SRI),
mentsormerelytoamistakeinexpressingthespec- NatarajanShankar(SRI),RickButler(LaRC),andthree
i(cid:12)cations. anonymous reviewers. This work was supported in part
by the National Aeronautics and Space Administration
A second source of versatility derives from the variety
under Contract No. NAS1-19341(V(cid:19)(cid:16)GYAN) and NAS1-
offormalmethodstechniques. As suggestedinthe GPS,
20334(SRI).
JS,and3E/Ostudies,therearemanywaystomodelsys-
temsandcalculatetheirproperties,eachofwhichconfers
di(cid:11)erent bene(cid:12)ts. References
(cid:15) Finite-State Veri(cid:12)cation: These techniques, includ-
[1] Ricky W. Butler, James L. Caldwell, Victor A.
ing state exploration and model checking, are good
Carren~o, C. Michael Holloway, Paul S. Miner, and
for speci(cid:12)cation debugging; counter example gener-
BenL.DiVito.NASALangley’sResearchandTech-
ation; automatic, rapid, and reliable exploration of
nology Transfer Program in Formal Methods. In
variations;andexhaustiveenumeration. Conversely,
Tenth Annual Conference on Computer Assurance
(cid:12)nite-stateveri(cid:12)cationoftendemandsmoreconcrete
(COMPASS95),pages135{149,Gaithersburg,MD,
speci(cid:12)cations, which are typically poor vehicles for
June 1995.
communicating and documenting systems and their
properties, as we found in the case of 3E/O. [2] Judy Crow. Finite-State Analysis of Space Shuttle
ContingencyGuidanceRequirements. TechnicalRe-
(cid:15) Proof Checking: Deductivetechniquessupportmore
portSRI-CSL-95-17,ComputerScienceLaboratory,
varied and more abstract models, more expres-
SRIInternational,MenloPark,CA,December1995.
sive speci(cid:12)cation, more varied properties, and more
AlsoforthcomingasaNASA ContractorReport for
reusable veri(cid:12)cations. Conversely, theorem proving
Task NAS1-20334.
techniques are typically less automatic and invari-
ably require more e(cid:11)ort on the part of the analyst. [3] Ben L. Di Vito and Larry Roberts. Using Formal
Methods to Assist in the Requirements Analysis of
Furthermore, although JS, GPS, and 3E/O have each
theSpaceShuttleGPSChangeRequest. Contractor
used a single formal methods technique, a further vari-
report, NASA Langley Research Center, Hampton,
ation o(cid:11)ering potentially greater productivity involves
VA, 1996. To appear.
applying a combination of methods to a given problem
byintegratingmodel-checking,proof-checking,andtech- [4] David L. Dill, Andreas J. Drexler, Alan J. Hu, and
6
niques such as simulation. C. Han Yang. Protocol Veri(cid:12)cation as a Hardware
A formal speci(cid:12)cation, particularly one that has been Design Aid. In 1992 IEEE International Confer-
veri(cid:12)ed,isbestviewedasapointofdeparture,providing ence on Computer Design: VLSI in Computers and
an e(cid:11)ective basis for documenting, calculating, and pre- Processors,pages522{525.IEEEComputerSociety,
dictingcurrentsystembehaviorandforanalyzingfuture 1992. Cambridge, MA, October 11-14.
modi(cid:12)cations and extensions. Reuse of formal methods
products and strategies provides the best return on in- [5] David Hamilton, Rick Covington, and John Kelly.
vestment in formal speci(cid:12)cation and analysis. The most Experiences in Applying Formal Methods to the
lasting contribution of the workdescribed here has been Analysis of Software and System Requirements. In
the development of two reusable strategies and a clear WIFT’95: WorkshoponIndustrial-StrengthFormal
demonstration of the tradeo(cid:11)s o(cid:11)ered by the versatility Speci(cid:12)cation Techniques, pages 30{43, Boca Raton,
of formal methods techniques. FL, 1995. IEEE Computer Society.
[6] DavidHamilton,RickCovington,andAliceLee.Ex-
Acknowledgments perience Report on Requirements Reliability Engi-
neering Using Formal Methods. In ISSRE ’95: In-
Theauthorsaregratefulfortheassistanceoftherequire- ternational Conference on Software Reliability En-
mentsanalystsandothersta(cid:11)atLoralSpaceInformation gineering, Toulouse, France, 1995. IEEE Computer
Systems: Larry Roberts, Mike Beims, Ron Avery, and, Society.
in earlier phases of this work, David Hamilton and Dan
[7] K. L. Heninger. Specifying Software Requirements
Bowman. All patiently answered questions and served
for Complex Systems: New Techniques and Their
up voluminous amounts of documentation. We would
Application. IEEE Transactions on Software Engi-
6Severalformalmethodssystemsintegratemodel-checkingand
neering, SE-6(1):2{13,January 1980.
proof-checking, some more intimately than others. See [15] for a
generaldiscussionofthePVSapproachandasummaryofrelated
[8] C. Norris Ip and David L. Dill. Better Veri(cid:12)cation
e(cid:11)orts, and [13] for a detailed discussion of tabular and state-
transitionspeci(cid:12)cationsandtheirveri(cid:12)cationusingtheintegrated through Symmetry. In CHDL ’93: 11th Conference
facilitiesofPVS. on Computer Hardware Description Languages and
8
their Applications, pages 87{100. IFIP, 1993. Ot- Computer Science Laboratory, SRI International,
tawa, Canada. Menlo Park, CA, December 1995.
[9] RobynR.LutzandYokoAmpo.ExperienceReport: [14] Sam Owre, John Rushby, Natarajan Shankar, and
UsingFormalMethodsforRequirementsAnalysisof Friedrich von Henke. Formal Veri(cid:12)cation for Fault-
Critical Spacecraft Software. In 19th Annual Soft- Tolerant Architectures: Prolegomena to the Design
ware Engineering Workshop, pages 231{248.NASA of PVS. IEEE Transactions on Software Engineer-
GSFC, 1994. Greenbelt, MD. ing, 21(2):107{125,February 1995.
[10] Steven P. Miller and Mandayam Srivas. Formal [15] S. Rajan, N. Shankar, and M.K. Srivas. An Inte-
Veri(cid:12)cation of the AAMP5 Microprocessor: A Case gration of Model-Checking with Automated Proof
Study in the Industrial Use of Formal Methods. In Checking.InPierreWolper,editor,Computer-Aided
WIFT’95: WorkshoponIndustrial-StrengthFormal Veri(cid:12)cation, CAV ’95, volume 939 of Lecture Notes
Speci(cid:12)cation Techniques, pages 2{16, Boca Raton, in Computer Science, pages 84{97, Liege, Belgium,
FL, 1995. IEEE Computer Society. June 1995. Springer-Verlag.
[11] Multi-CenterNASATeamfromJetPropulsionLab- [16] JohnRushby. FormalMethodsandtheCerti(cid:12)cation
oratory, Johnson Space Center, and Langley Re- of Critical Systems. Technical Report SRI-CSL-93-
search Center. Formal Methods Demonstration 7,ComputerScienceLaboratory,SRIInternational,
ProjectforSpaceApplications{PhaseICaseStudy: MenloPark,CA,December1993. Alsoissuedunder
SpaceShuttleOrbitDAPJetSelect,December1993. the title Formal Methods and Digital Systems Val-
NASA Code Q Final Report (Unnumbered). idation for Airborne Systems as NASA Contractor
Report 4551, December 1993.
[12] National Research Council Committee for Review
of Oversight Mechanisms for Space Shuttle Flight [17] A. John van Schouwen. The A-7 Requirements
SoftwareProcesses,NationalAcademyPress,Wash- Model: Re-ExaminationforReal-TimeSystemsand
ington, DC. An Assessment of Space Shuttle Flight an Application to Monitoring Systems. Technical
Software Development Practices, 1993. Report90-276,DepartmentofComputingandInfor-
mation Science, Queen’s University, Kingston, On-
[13] Sam Owre, John Rushby, and Natarajan Shankar.
tario, Canada, May 1990.
Analyzing Tabular and State-Transition Speci(cid:12)ca-
tionsinPVS. Technical Report SRI-CSL-95-12,
9