Safety of Computer Control Systems 1986 (Safecomp '86). Trends in Safe Real-Time Computer Systems

183 Pages·1986·16.111 MB·English

by  William J. Quirk

NOTICE TO READERS
If your library is not already a standing/continuation order customer or subscriber to this series, may we recommend that you place a
standing/continuation or subscription order to receive immediately upon publication all new volumes. Should you find that these volumes no longer serve
your needs your order can be cancelled at any time without notice.
Copies of all previously published volumes are available. A fully descriptive catalogue will be gladly sent on request.
ROBERT MAXWELL
Publisher
IF AC Related Titles
BROADBENT 8c MASUBUCHI: Multilingual Glossary of Automatic Control Technology
EYKHOFF: Trends and Progress in System Identification
ISERMANN: System Identification Tutorials (Automatica Special Issue)
SAFETY OF COMPUTER
CONTROL SYSTEMS 1986
(SAFECOMP '86)
Trends in Safe Real Time
Computer Systems
Proceedings of the Fifth IF AC Worfahop
Sarlat, France, 14—17 October 1986
Edited by
W. J. QUIRK
Computer Science & Systems Division,
Atomic Energy Research Establishment, Harwell, U.K.
Published for the
INTERNATIONAL FEDERATION OF AUTOMATIC CONTROL
by
PERGAMON PRESS
OXFORD · NEW YORK · BEIJING · FRANKFURT
SÄO PAULO · SYDNEY · TOKYO · TORONTO
U.K. Pergamon Press, Headington Hill Hall, Oxford OX3 OBW, England
U.S.A. Pergamon Press, Maxwell House, Fairview Park, Elmsford, New York 10523, U.S.A.
PEOPLE'S REPUBLIC Pergamon Press, Qianmen Hotel, Beijing, People's Republic of China
OF CHINA
FEDERAL REPUBLIC Pergamon Press, Hammerweg 6, D-6242 Kronberg, Federal Republic of Germany
OF GERMANY
BRAZIL Pergamon Editora, Rua Ega de Queiros, 346, CEP 04011, Säo Paulo, Brazil
AUSTRALIA Pergamon Press Australia, P.O. Box 544, Potts Point, N.S.W. 2011, Australia
JAPAN Pergamon Press, 8th Floor, Matsuoka Central Building, 1-7-1 Nishishinjuku, Shinjuku-ku,
Tokyo 160, Japan
CANADA Pergamon Press Canada, Suite 104, 150 Consumers Road, Willowdale, Ontario M2J 1P9, Canada
Copyright © 1986 IFAC
All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in
any form or by any means: electronic, electrostatic, magnetic tape, mechanical, photocopying, recording or other-
wise, without permission in writing from the copyright holders.
First edition 1986
British Library Cataloguing in Publication Data
SAFECOMP '86 (Conference : Sarlat)
Safety of computer control systems 1986
(SAFECOMP '86) : trends in safe real time
computer systems : proceedings of the Fifth
IFAC Workshop, Sarlat, France, 14-17
October 1986.
1. Automatic control—Data processing
I. Title II. Quirk, W. J. III. International
Federation of Automatic Control
629.8'312 QA402.3
ISBN 0-08-034801-7
These proceedings were reproduced by means of the photo-offset process using the manuscripts supplied by the
authors of the different papers. The manuscripts have been typed using different typewriters and typefaces. The
lay-out, figures and tables of some papers did not agree completely with the standard requirements: consequently
the reproduction does not display complete uniformity. To ensure rapid publication this discrepancy could not be
changed: nor could the English be checked completely. Therefore, the readers are asked to excuse any deficiencies
of this publication which may be due to the above mentioned reasons.
The Editor
Printed in Great Britain by A. Wheaton & Co. Ltd., Exeter
FIFTH IFAC WORKSHOP ON SAFETY OF COMPUTER
CONTROL SYSTEMS (SAFECOMP '86)
Trends in Safe Real Time Computer Systems
Organized by
Association pour le Developpement de PEnseignement, de
l'Economie et des Recherches de Midi-Pyrenees (ADERMIP)
Sponsored by
The International Federation of Automatic Control (IFAC) through
Association Frangaise pour la Cybernetique, Economique et Technique (AFCET)
European Workshop on Industrial Computer Systems (EWICS)
International Federation of Information Processing (IFIP)
Electricite de France (EDF)
International Program Committee
J. M. A. Rata, France (Chairman) R. Genser, Austria
W.J. Quirk, UK (Editor) E. Johnson, UK
E de Agostino, Italy S. Keresztely, Hungary
T. Anderson, UK Th. Lalive d'Epinay, Switzerland
A. Avizienis, USA J. C. Laprie, France
J. Bernussou, France R. Lauber, FRG
R. Bloomfield, UK N. Leveson, USA
S. Bologna, Italy F. Redmill, UK
P. Ciompi, Italy B. Runge, Denmark
G. Dahll, Norway I. C. Smith, UK
B. K. Daniels, UK B. Sterner, Sweden
J. Debelle, Belgium J. P. Vautrin, France
J. A. Dobbins, USA U. Voges, FRG
W. Ehrenberger, FRG R. W. Yunker, USA
H. Frey, Switzerland
National Organizing Committee
A. Costes (Chairman)
H. Krotoff
A. Poujol
PREFACE
This fifth SAFECOMP workshop is looking to the trends The papers in this workshop cover a wide range of
in computer safety which have emerged since the topics. As well as underlining the three trends
first SAFECOMP, held in Stuttgart in 1979. The noted above, papers address the problems of quality
micro-computer has evolved to the stage that much assurance, fault tolerance, safe architectures and
conventional instrumentation is in fact computer-based. safe design, operator interface and, lastly,
Indeed, such basic building blocks for safety systems assessment and qualification.
as relays are fast being replaced by programmable
logic contrc'lers. The potential benefits of increased The initiative and impetus for these events continues
safety to be gained from using computers are well to be EWICS TC 7, the 'Safety, Security and
appreciated. But with these benefits come corresponding Reliability1 technical committee of the European
challenges; the software industry is not renowned Workshop on Industrial Computer Systems. TC 7 is a
for the freedom from error of its products. These body of experts concerned with all aspects of safety
challenges have lead to a number of trends, three and security arising from the use of computers in
of which are directly relevent to this SAFECOMP. potentially hazardous situations. It addresses the
problems of protecting human wellbeing, the
The first is that safety is not a local, private environment and the plant itself against hazards
matter. On the contrary, accidents take no notice arising from failures in computer control or safety
of plant boundaries, town boundaries or even national systems however these may occur. The objectives of
boundaries. The need for widely accepted international TC 7 include the determination and dissemination of
standards has never been so strong. But to be of procedures to construct, document, test and verify
value, such standards must not be volumes full of the safe performance of such systems. It is currently
pious intents. Practical guidance on the successful involved in the production of guidelines for System
application of available techniques in real situations Integrity, Design for System Safety, Software Quality
is of paramount importance. Assurance and Measures, and Safety and Reliability
Assessment.
The second trend is fast recognition of the potential
of knowledge-based systems. With the traditional The programme committee wish to record their thanks
conservatism of the safety industry, it is at first to the sponsoring organisations: IFAC, AFCET, IFIP
sight somewhat surprising that their application to & ADERMIP; also to the National Organising Committee
safety systems is so advanced. Yet this is precisely and EDF for their administrative efforts, to TC 7,
the reality of the situation. One reason for this particularly to its chairman J.-M.A. Rata whose
may be that it is well known that most humans do not tirelessness has urged the committee to work so hard
function optimally in crisis situations, so the themselves, and to the Safety and Reliability Society
availability of reliable expert knowledge in such of Great Britain for their support in administering
situations is a great advantage. the contract with the Commission of the European
Communities on behalf of TC 7 and so enabling it to
The third trend is the proposed use of diversity, continue its work. The editor is once again grateful
particularly in software. There has, until recent for the assistance and forebearance of the staff of
years, been a pervading view that more and more the IFAC Publisher Pergamon Books in the preparation
care, time and effort should be taken over producing of these proceedings. It is hoped that all will find
a single, ultimate quality software product. Diversity these new trends both stimulating and reassuring.
techniques bring this view into question. The
arguments over the precise cost and effectiveness
of such techniques have been quite widely rehearsed, W.J. Quirk
and will no doubt continue beyond this workshop. But AERE Harwell
more practical experience and real project data are
becomming available.
Copyright © IFAC SAFECOMP '86 SOFTWARE QUALITY ASSURANCE
Sarlat, France, 1986
SOME THOUGHTS ON SOFTWARE QUALITY
ASSURANCE
K. Frühauf
Brown Boveri &f Cie, Baden, Switzerland
Abstract. The paper tries to review the problems a software quality
assurance engineer faces in implementing a software quality assurance
organisation. The delimitation of the software quality assurance is
summarised in five statements. The assignment of responsibilities and
tasks to the software quality assurance organisation in spirit of
these statements is illustrated using system test activity as an
examp I e .
Keywords. Computer software; software quality assurance; software
engineering; software management.
INTRODUCTION
Software quality assurance is a topic of Though we are not able to validate the
great interest nowadays. The reasons quality of a large software product, we
are well-known. Less known is the maybe able to check and judge the
actual content of the term. If a quality of the procedures by which it is
software engineer or a quality assurance produced. Therefore more and more
engineer receives the responsibility to purchasers will require a documented
introduce software quality assurance in quality assurance program with special
an organisation he or she will have emphasis on software. There they will
difficulties to obtain a clear find what the corporate quality standard
definition of responsibilities and tasks i s suppos ed to be.
because the perception what software
quality assurance should do differs First definitions of some terms are
largely among the software community. given and their implications are
Most likely no two companies will have discussed. Based on the established
the same assignment of responsibilities terminology the delimitations of a
and tasks. This is good as long as software quality assurance organisation
every company has a defined policy on are summarised in five statements. The
which the assignment is based. Such assignment of responsibilities is
policy is a prerequisite for avoiding illustrated on the example of the system
conflicts and preventing frustrations. test activity.
Our view is one from within a company
involved in conventional manufacturing
as well as in software development. The
quality assurance organisation spans the
whole company and the problem of DEFINITIONS
integrating actions assuring software
quality in the traditional quality
assurance program has been tackled. It In this section those terms are defined
is feasible and necessary to integrate and discussed which are necessary to
the software quality assurance in understand the paper.
organisations producing large embedded
software systems. Unfortunately the Software Quality Assurance. The
standardisation went the opposite way. definitions in the available standards
The CSA Q396.1 (1982) standard for are similar but not identical. In IEEE
software quality assurance program is 730 (1984) quality assurance is defined
written in spirit of the CSA Z299.1 as follows :
(1979) standard but still, the
implementation requires a merge of the "A planned and systematic pattern of
two. Merge of the IEEE 730 (1984) all actions necessary to provide
standard with CSA Z299.1 (1979) is even adequate confidence that the item or
more difficult. product conforms to established
technical requirements."
1
2 K. Friihauf
The software manager is avai labl e t here fore th ey a re
entirely responsible for the not su i tab le f o r co m p a r i n g
project costs, schedules, and prod uc t s pro duce d i n di f fe rent
quality as well as for the env i ronm ents N ever the I ess
quality of the resulting they can be usef ul i f applied
product. on pro duct s f rom th e same
env i ronm ent. We exp erience
The craf t sman ι n his work shop that m e t r ic s eva luat ion in
take s t he f uL L r espo n s i b iI i ty prod uc t aud i ts i s of great
f o r the f i n a nc i a I and techn i ca I va I ue i n o rde r to ass ess the
succ ess . He has no d ocume nt ed adeq ua cy o f the project
cone ept i on of qua I i ty but he prog ress an d o f t he produc t
will t ry ha rd t o co m m u n i ca t e evo I ut i on . Appl ying th e same
h i s f e e I ln g fo r and sma I I set of m e t r ic s to
unde rsta n d i n g of q ua L it y to h i s di f fe ren t pr oduc ts a nd to the
appr en t ic e . A prod uc t not same P rodu ct at di fferent
conf o rm in g to h i s cone e p t i on of stag es o f ev olut i on prov ides a
qua L ity will ce r t a in I y not re I ev ant c ol Le ctio n o f data
L ea ve the work shop E very wh i ch c an i nd i cate prob I em
even i ng when the c raft sman does area s .
the bo okkee ping he gets an
i mmed i at e f eedba ck on the The metrics for quantification
adeq ua cy of his cone e p t i on o f of software quality need to
qua L ity. have the following properties.
The metric shall be
The sam e pr l nc i pl e i s valid for
a com pany pr oduc i n g large measurab I e
so ftwa r e sy stems The scales i.e. an algorithm or
are d if fe rent how ever. The method exists
so ftwa r e ma nage r can not do the
work a lone H e o r she will reproduceab I e
emp toy an ac coun tant for i.e. it can be measured
support i n bookk eep i n g and will exactly thus its value can
expect h i m or he r t o know a t be monitored continuously
any t im e e.g. how much money
has bee n sp ent o nth e project. expressive
If he or she i s a clever i.e. permits the
so ftwa r e ma nage r he or she will distinction of "good" and
emp Loy a sof twa r e quality "bad"
assuran ce e n g i ne e r and will
expect h i m or her to know a t meaningful
any t im e e.g. ho portab I e i.e. results corresponds
is the pro duct ( i f portability to other useful measures
is a re qu i re men t ) . The analogy
i s : The a c co u n t a n t is efficient
respons ible for pro ducing the i.e. the potential benefit
balance she et in t e rm s of money of the findings outweights
and t he soft ware quality the cost of the measurement
assuran ce e n g i ne e r in terms of
quality a 11 r ib u t e s. The cost indicating
softwa r e ma nage r nee d s , similar i.e. can be related to the
to th e craft sman in the development and maintenance
worksho Ρ# both as basis for costs
d e c i s i no ma king.
The choi ce of a metric must
have an ob j e c t i ve . A simple
2. The primary goal of the exam p I e 1 S ope ra ting system
software quality assurance depe nden ceo f the produc t The
organisation is to know the ob j ec t i ve f or the metric is to
quality of the software have a co r re I a t i no be twe en its
products and projects. va I ue an d th e cost of p o r t i n g
the sof twa r e to a new r e I ease
A prerequisite for achieving of t he o pe ra ting s ys t em o r to a
this goal is the definition of comp lete ly d i f f e re n t on e. It
criteria for the evaluation of i s th e task of the so ftwa re
quality. For the project qua I ity ass u ran c e engin ee r to
quality the documentation of find the met r i c w it h the best
the quality assurance program CO Γ Γ e lat ion. The spec t rum o f
is the basis. A checklist can pot e n t i aI m etric s is I i m i t e d
be derived and a project audit on I y by hi s or her f antasy
will reveal the extent to which (num ber of modu I es con t aining
the required quality assurance ope r atin 9 sys tern refer ences,
actions are implemented by the tota I numb er o f ope rating
project team. sys t em re f er ences , numb er of
re f e renc ed opera ting sys tern
Metrics are the means to f ac i I i t ie s, etc . ) .
evaluate quality of software
products. Currently no
commonly accepted metrics are
Some Thoughts on Software Quality Assurance
The CSA Q396.1 (1982) standard defines documents so concisely that they are
software quality assurance in following practical, an aid to the project team
terms : and not to the papermill.
"A planned and systematic pattern of Project. Project is the planned pattern
all actions necessary to provide of all actions necessary to build a
adequate confidence that software software product. For the purpose of
components conforms to established this paper we do not distinguish between
requirements and specifications." development projects (product sold many
times) and customisation projects
From these definitions immediately (single shot product). For this
follows that actions contributing to the distinction see Frtfhauf, Sandmayr
software production are not quality (1983). The people involved in the
assurance actions, i.e. to build project form the project team, and the
software is not software quality leader is called here software manager.
assurance. The key issue for software
quality assurance is confidence gaining. Quality Assurance Organisation.
Some actions like test are on the edge - Responsible for the provision and
nobody would claim that software can be effectiveness of the quality assurance
built without any test, and on the other program. Independent from the projects.
hand it is clear that tests provide The members of the quality assurance
confidence. System testing will be used organisation are called software quality
later to illustrate the delimitation of assurance engineers. The important
responsibility areas. matter is that all actions in the
project are identified and the
Quality Assurance Program. The responsibility for the particular action
documented set of quality assurance is uniquely assigned by the management
actions. In IEEE 73 0 (1984) the term either to the project team or to the
plan is used instead of program. quality assurance organisation. Such
Corresponding to CSA Q396.1 (1982) the project activities, characterised by
elements of the documentation are : their output, are for instance :
Quality assurance manual system specification (output =
system specification document)
This is the constitution of the
quality assurance program. It system specification review
contains the software quality (output = review report)
assurance policy of the company
and certifies the commitment of subsystem design (output =
the management to it. subsystem design document)
Quality assurance procedures component coding and test
(output = release of a
The qua I ity assurance component),
procedures are the laws of the
program. They specify the system test design (output =
implementation of the software system test procedures),
quality assurance actions. The
procedures must be concise so system integration and test
that they can be easily (output = release of the
implemented and the conformance system),
to them can be checked
unambiguously. It is a must document standardisation
that the responsibilities for (output = standard for document
carrying out the actions are I ayout and content)
uniquely assigned.
product auditing (output =
Regulations (standards, rules,
product audit report)
conventions)
The example should illustrate the range
of activities we have in mind. The
For specific topics (e.g.
software quality assurance is performing
coding rules in a programming
per definition - only a subset of
language) detailed descriptions
these activities. It is a difficult and
are required. These are not
ambiguous but necessary task to define
written in the form of quality
the complete set of these actions.
assurance procedures. We
recommend to write such
regulations in the form of a
requirement specification for a
tool (e.g. tailored editor,
prettyprinter). This enforces
preciseness and stimulates the DELIMITATION OF SOFTWARE
provision of such a tool - the QUALITY ASSURANCE
best way to ensure conformance
to the regulation.
This section provides some guidelines
The danger is big that a pile of paper for delimitation of the responsibilities
will be produced and nobody will read and tasks of a software quality
it. The real challenge is to write all assurance organisation.
4 K. Friihauf
3. The secondary goal of the issues up to the adequate
software quality assurance management level.
organisation is to provide and
maintain the documentation of
the quality assurance program. 5. Software quality assurance is a
discipline within software
Of course the first thing to do engineering and not the other
is to document the quality way around.
assurance program. By this
statement we would like to This is at least our view and
point out that the mission of our way to delimitate software
the quality assurance quality assurance. Other
organisation is quality software engineering
accounting and not paper disciplines are e.g.
production. The provision and specification methods and
maintenance of the quality tools, design methods and
assurance manual, procedures, tools, software project
and regulations is a major task management, software cost
of the quality assurance estimation, and configuration
organisation. The actual management. The reason for
content of the documents shall this delimitation is the common
be worked out in cooperation practice we experience : Under
with the project team in order the heading of software quality
to increase the acceptance of assurance a lot is said or
the specified actions. Forcing written e.g. about software
regulations upon a project team life cycle and structured
from an independent software programming. We are convinced
quality organisation will that a wide room for research,
seldom work. The project team teaching, and practice remains
and first of all the software for software quality assurance
manager must have a deep without excursions into other
commitment to the quality disciplines.
assurance program. They gain
it easiest by participation in
setting the quality objectives,
i.e. providing the meat
(content) on the bones
(documents) of software quality
assurance. SYSTEM TESTING :
AN EXAMPLE OF DELIMITATION
4. The software quality assurance
is a service organisation
concerning all matters of On the activity type system testing we
software qua I i ty. want to illustrate the assignment of
responsibilities to the software quality
The attitude of the software organisation in detail. Let us first
quality assurance engineers is try to identify the particular actions
crucial. They must be aware of involved in system testing (first of all
the fact that they have to a project should of course recognise
perform for software managers that system testing must be carried
and project teams and not the out) .
other way around. The quality
assurance engineers must not 1. A metric for the definition of
try to replace software the system test quality level
managers, but also, the must be chosen (e.g. every
software managers must not specified function and quality
wriggle out of responsibility attribute shall be tested at
by delegating it to the least by one "normal" and one
software quality assurance "error" test case, every
engineer. Especially in the specified output shall be
latter case the outcome will be produced at least once, etc.).
a disaster.
2. A metric for the end of the
Software quality assurance is system test activity must be
very similar to with the work specified (e.g. less than five
of consultants. The software deficiencies found, estimated
quality assurance engineer effort for the repair of the
stands between the supplier deficiencies less than x
(project team) and the mandays, etc.).
purchaser (software manager).
He or she has the right and 3. A method for test
duty to make recommendations selection must be chosen.
but has no control over the
flow of money and consequently 4. A method for test case
should not have the veto right. specification must be chosen.
The independency of the quality
assurance organisation, 5. The test cases must be selected
however, shall enable the and the test procedures must be
raising of software quality w r i 11 en .
Some Thoughts on Software Quality Assurance 5
6. A method for reviewing test not only increase the level of
procedures must be established. confidence but also have the effect of
know-how transfer within the project
7. A method for documenting team. This side-effect has a prevailing
reviews must be chosen. value in large projects. Therefore
these activities are carried out by
8. The test procedures must be project team members. The review and
reviewed. test team assignment is the sole
responsibility of the software manager
9. The review report must be and his or her own interest to obtain an
prepa red . outcome he or she can rely on. The
appointment of an independent software
10. The test procedures must be quality assurance engineer in tests and
accepted and released for reviews is a good practice.
carrying out the test.
1 1 , The review report mus t be
evaluated.
12. A method for documenting the CONCLUSIONS
test must be chosen.
13. The test must be carried out. The responsibilities of the software
quality assurance organisation are from
14. The test report must be our point of views as follows :
prepa red .
Provide and maintain the
15. The test must be accepted and documentation of the software
finished (see criteria above). quality assurance program, i.e.
manual, procedures, guidelines.
16. The test report must be
evaluated. Define metrics for measurement
of the software product and
An impressive list. Although not project quality.
comprehensive (e.g. the test
installation must be planned and made Obtain the value of the metrics
available, all chosen methods must be by means of product and project
documented) sufficient for our purpose. audits.
The steps of test and review report Actively participate in reviews
evaluation need some explanation. We and tests.
mean by that the collection of these
reports and their evaluation aiming at Evaluate test, review, and
identification of frequent error types software problem reports.
and of software items with high error
rate. This serves as a basis to Provide recommendations for
initiate corrective actions in the corrective actions (in project
project (in case of frequent error o r produc t) .
types) or concerning the software
product (in case of items with high We are aware that quality must be built
error rate). in, that the project team must be
quality conscious, and that the software
The evaluation of the review and test manager is responsible for the quality.
reports is - in the light of the Therefore our conclusion is that the
delimitation 2 - undoubtedly the software quality assurance organisation
responsibility of the software quality is to be made responsible for the
assurance organisation. The analytical and the project team for the
identification of the activities for constructive quality assurance actions.
which a method must be chosen and However, the communication between the
documented (test case selection and software quality assurance engineer and
specification, review of test the specialists for software development
procedures, review and test reporting in methods and tools is vital for providing
our example) shall be the responsibility the adequate software development
of the software quality assurance environment.
organisation. The responsibility for
the actual selection of the methods is - A practising software quality assurance
in the light of the delimitation 1 - the engineer will have to do a lot of
responsibility of the software manager. research type work and use extensively
The software quality assurance his or her fantasy in the quality
organisation shall play the role of a analysis work. This job requires a deep
consultant office and provide understanding of the purpose of the
recommendations based on the findings produced software and of the software
from the evaluations. development process in order to devise
the few figures which pointed express
We consider the preparation of the test the state of affairs.
procedures (i.e. test design), their
review as well as the test itself an The other main part of the job is to
integral part of the software communicate with the project team to
development process. Reviews and tests draw off its conception of quality and