System Verification. Proving the Design Solution Satisfies the Requirements

322 Pages·2007·14.405 MB·English

by  Jeffrey O. Grady (Auth.)

LIST OF ILLUSTRATIONS
1.1-1 Item verification schedule. 5
1.2-1 Matrix organizational structure. 16
1.2-2 We are all specialists. 22
1.2-3 Generic system development process. 25
1.2-4 Grand system pattern. 28
1.2-5 A common enterprise life cycle model. 29
1.2-6 Correlation of a process model and program phasing. 29
1.2-7 The w^aterfall model. 30
1.2-8 The spiral model. 31
1.2-9 The V model. 33
1.2-10 The N development model. 33
1.2-11 Development environment space. 34
1.2-12 Preferred spaces example. 35
1.3-1 Requirements taxonomy. 42
1.3-2 Space launch system use system function. 46
1.3-3 Sample decomposition tool approval matrix. 49
1.3-4 Structured decomposition for grand systems and hardw^are. 50
1.3-5 UML overview^. 62
1.3-6 Life cycle functional flov^ diagram. 64
2.1-1 Verification strings. 74
2.1-2 Specification types versus verification classes. 78
2.1-3 Method selection criteria. 83
2.1-4 Verification planning documentation suite. 86
2.1-5 Specification verification section structure. 87
2.1-6 Class-differentiated organization. 89
2.1-7 Method-differentiated verification section structure. 90
2.1-8 Verification data structures. 91
2.2-1 System verification context. 95
XV
XVI LIST OF ILLUSTRATIONS
2.2-2 Qualification process flow. 96
2.2-3 Verification planning and implementation management
documentation. 99
I.IA Documentation overview. 103
2.2-5 Accomplish qualification process. 105
2.2-6 Item L verification process. 107
2.2-7 Generic item test task flow. 108
2.2-8 Generic item analysis task flow. 109
2.2-9 Generic item examination task flow. 109
2.2-10 Generic item demonstration task flow. 110
2.3-1 The relationship between product entities and specifications. 116
2.3-2 Item VTN map. 118
2.3-3 Verification documentation structure. 118
2.3-4 Integrated verification plan outline. 119
2.3-5 Item verification string fusion. 122
2.3-6 Verification strings to VTN transform. 122
2.3-7 Promoting and demoting verification strings. 123
2.3-8 Generic test task. 125
2.3-9 Item L test task plan fragment. 127
2.3-10 One verification planning string. 128
2.3-11 Life warranty utility curve. 135
2.3-12 Use of test drivers and stubs: (a) bottom-up development
and (b) top-down development. 144
2.4-1 Organizational structure for qualification. 150
2.4-2 Universal task identification numbering system. 151
2.4-3 Individual test implementation model. 154
2.4-4 VTN N-square diagram. 162
2.4-5 VTN interface tabular listing. 162
2.4-6 Test sequencing. 163
2.4-7 Corrective action loop. 164
2.5-1 Analysis task grouping. 175
2.6-1 Integrated verification data report structure. 181
2.6-2 Task decision-making process. 183
2.7-1 Verification task groups. 190
2.7-2 The verification task tracking matrix. 191
2.7-3 Multiple FCA schedule. 198
2.7-4 FCA agenda. 200
2.7-5 Specifications and verification support database structure. 204
3.1-1 Tolerance funneling. 223
3.2-1 Nontest acceptance verification method responsibilities
coordination. 226
3.3-1 Integrated verification data report structure. 235
3.4-1 The path to acceptance. 239
3.4-2 Development to production organizational transform:
(a) development phase and (b) production phase. 242
3.4-3 FRACAS flow. 244
3.4-4 Supporting techniques: (a) Ishikawa (fishbone) diagram
and (b) Pareto diagram. 244
4.1-1 The V model encourages the right timing. 252
4.2-1 Overall system test and evaluation situation. 268
4.2-2 Integrated verification data report structure. 271
LIST OF ILLUSTRATIONS XVII
4.3-1 Reverification exposure periods. 279
5.1-1 Life cycle functional flow diagram. 287
5.1-2 Program and functional planning documentation tree. 290
5.2-1 Matrix structure optimizing V&V functions. 300
6.1-1 Product-only WBS enables good planning. 312
6.1-2 The product-only WBS. 314
6.1-3 The grand planning environment using product-only WBS. 316
6.2-1 Generic V tasks. 327
LIST OF TABLES
2.1-1 Sample Verification Traceability Matrix 72
2.2-1 Sample Verification Compliance Matrix Fragment 97
2.2-2 PID Accommodation 98
2.2-3 Verification Task Matrix Example 101
2.2-4 Verification Item Matrix 102
2.4-1 Test Task Responsibilities and Exit Criteria 154
3.1-1 Requirements Comparison 217
4.1-1 Test and Evaluation Master Plan Outline 262
5.2-1 Metrics List Representative Candidates 296
5.2-2 Verification Responsibilities 298
6.1-1 Nonproduct WBS Transform 315
6.2-1 Major Review^s and Audits Dictionary 328
6.2-2 V Task Dictionary 329
6.2-3 V Word Vocabularies 330
6.2-4 Detailed Source References 331
XIX
PREFACE
The original work for this book began under the title System Validation and
Verification^ published by CRC Press in 1997. It was rewritten in 1998 to
some extent and coordinated with a rewrite of several other books that I had
written to form Volume 4 of a four-volume manuscript series, titled Grand
Systems Development^ for use as text material in a system engineering and
development instruction program offered by my consulting and training com
pany, JOG System Engineering, Inc. The original book covered both validation
and verification, but this book is focused only on verification. The validation
material was moved to the companion book in the same series, titled Systems
Requirement Analysis^ recently published by Academic Press.
This book extends the same process applied to product verification to pro
cess verification, following the pattern of (1) define the problem, (2) solve the
problem, (3) prove it. Fairly recently, sound standards have evolved for system
development, system engineering, and software engineering. These standards
act as process specifications for the corresponding processes we must design.
Sound standards are also maturing for capability maturity models for use in
assessing the quality of the process design, providing a process verification
mechanism.
XXI
ACKNOWLEDGMENTS
This book is dedicated to the memory of Mr. Max Wike, a retired naval officer
and a system engineer in the department I managed at General Dynamics Space
Systems Division in the mid-1980s. Before he passed av^ay long before his
time, he and his wife to be, Ms. Debbie Matzek, had succeeded in beginning
my education in requirements verification, and I very much appreciate that to
this day.
It never fails that during every class on system engineering that I have
taught for the University of California San Diego, UC Irvine, UC Berkeley,
through short course companies, and independently through my consulting
firm at companies, students have offered me great insights and ideas about
how^ this process can be done better. Many of these inputs find their way into
future courses as well as new books like this one and, hopefully, revisions of
others. Unfortunately, I have not shown due diligence in keeping a list of these
people by name and location so that I can properly thank them. The Ust would
be very long. By the time this book is published it would likely include on the
order of a thousand engineers.
A lady from Raytheon in Tucson, Arizona, Christine Rusch, attending the
International Council on Systems Engineering (INCOSE) Symposium V&V
tutorial I did in Brighton, England, in 1999, suggested coverage of the rever-
ification situation, adding content to the course and a chapter to the text.
A gentleman attending a similar tutorial at the 2000 INCOSE Symposium
in Minneapolis, Minnesota, discovered that I had not covered the reporting
process adequately, causing the addition of chapters on that subject. Also, at
the same time, while considering how that material should be added, I noticed
that system test and evaluation was covered in a most cursory fashion and a
whole part was added to the text. It is true that a lecturer learns a great deal
through teaching because you have the opportunity of talking to some very
bright people. I appreciate all of those suggestions.
XXIII
XXIV ACKNOWLEDGMENTS
I benefited a great deal from a UC San Diego System Engineering Cer
tificate Program course I attended while the ideas for this book were initially
swarming in my mind. That course, titled "System Verification," was taught
by a great guy and fine system engineer, Mr. James R. (JB) Hill, at the time
a vice president at Scientific Applications International Corporation (SAIC) in
San Diego, California. I very much appreciate the new knowledge I derived
from his course.
Several people in industry provided much appreciated and valuable
insights into modern techniques for validation and verification on items as
small as integrated circuits and as large as a transport aircraft. Mr. David
Holmes, a system engineer at Interstate Electronics Corporation in Anaheim,
California, very generously arranged a demonstration of how they develop
application-specific integrated circuits (ASIC) during a certificate program I
presented for UC Irvine at their facility in 1995. Mr. David B. Leib, a test
engineer on the McDonnell Douglas C-17 Flight Control System Simulator
(Iron Bird), gave me a tour of the simulator at their facility in Long Beach,
California, during a certificate program I presented at their facility for UC
Irvine in 1996 that was very useful in describing a typical large-scale validation
instrument.
While I have benefited from the advice of those identified and many others
in encounters that I cannot recall precisely, they should not be held account
able for any errors of commission, omission, or wrong-headed thinking. They
belong wholly to the author.
LIST OF ABBREVIATIONS
AFB air force base
ASIC application specific integrated circuit
ASROC anti-submarine rocket (rocket thrown torpedo or depth charge)
ASW anti-submarine warfare
ATP acceptance test plan
CBD commerce business daily
CDRL contract data requirements list
CMM capability maturity model
CMMI capability maturity model integrated
CONOPS concept of operations
CDD capabilities development document
CDR critical design review
CD ROM compact disk read only memory
CPM critical path method
C/SCS cost/schedule control system
DET design evaluation testing
DFD data flow diagram
DOD Department of Defense
DTC design to cost
DT&E Development test and evaluation
ECP engineering change proposal
E^PROM electrically erasable PROM
EDI electronic data interchange
EIA Electronic Industry Association
EIT enterprise integration team
EMI electromagnetic interference
ERD entity relationship diagram
FAA Federal Aeronautics Administration
XXV
XXVI LIST OF ABBREVIATIONS
FDA Food and Drug Administration
FRACAS failure reporting and corrective action system
GD General Dynamics
GDSS General Dynamics Space Systems
HOL higher order language
lAT integration, assembly, and test
ICAM integrated computer aided manufacturing
ICBM intercontinental ballistic missile
ICD initial capabilities document
interface control document
IDEF integrated definition [language]
IEEE International Electrical and Electronic Engineering
INCOSE International Council On Systems Engineering
IOC initial operating capability
lOT&E initial operational test and evaluation
IPO input process output
IPPT integrated product and process team
IRAD independent research and development
ITP integrated test plan
ITEP integrated test and evaluation plan
IVDR integrated verification data report
IVMR integrated verification management report
IVP integrated verification plan
IVR integrated verification report
IV&V independent validation and verification
JCD joint capabilities document
JFX joint strike fighter
FCA functional configuration audit
FMECA failure modes effects and criticality analysis
FOT&E follow-on test and evaluation
NASA National Aeronautics and Space Administration
OOA object oriented analysis
ORD operational requirements document
OT&E operational test and evaluation
PAM process-architecture matrix
PERT program evaluation review technique
PBT program business team
PCA physical configuration audit
PDR preliminary design review
PID product identification
PIT program integration team
PROM programmable read only memory
QA quality assurance
RAM reliability, availability, maintainability
RATO rocket assisted take off
RFP request for proposal
RID requirement identification
RPM revolutions per minute
SADT structured analysis design technique
SDR system design review
SDRL supplier data requirements list
SOW statement of work
LIST OF ABBREVIATIONS XXVII
SQA software quality assurance
SRD system requirements document
SRR system requirements review
STE special test equipment
STEP system test and evaluation plan
TEMP test and evaluation master plan
TPM technical performance measurement
TQM total quality management
UHF ultra high frequency
UML unified modeling language
USAF United States Air Force
USN United States Navy
USS United States ship
USSR Union of Soviet Socialist Russia
VHDL VHSIC hardware description language
VHSIC very high-speed integrated circuit
VID verification identification
VSN verification string number
VTN verification task number
WBS work breakdown structure