Software Fault Tolerance: Achievement and Assessment Strategies

249 Pages·1992·6.164 MB·English

by  Manfred Kersken (auth.), Manfred Kersken, Francesca Saglietti (eds.)

Research Reports ESPRIT
Project 300 . REQUEST· Vol. 1
Edited in cooperation with
the Commission of the European Communities
M. Kersken F. Saglietti (Eds.)
Software Fault Tolerance
Achievement and Assessment Strategies
Springer-Verlag
Berlin Heidelberg New York London Paris
Tokyo Hong Kong Barcelona Budapest
Editors
Manfred Kersken
Francesca Saglietti
Gesellschaft fOr Reaktorsicherheit (GRS) mbH
ForschungsgelAnde, W-8046 Garching, FRG
ESPRIT Project 300 "Reliability and Quality of European Software Technology
(REQUEST)" belongs to the Subprogramme "Software Technology" of ESPRIT,
the European Strategic Programme for Research and Development in Infor
mation Technology supported by the Commission of the European Communities.
Project 300 aims at progress in quantification of software quality and reliability,
thus enabling their specification, prediction, measurement, and assurance. The
areas of work include: identification and validation of metrics for the "quality"
concept, and construction of a quantitative model for its prediction; development
of metrics and models for reliability prediction, both for software systems in
general and for domains requiring ultra-high reliability; and investigation of the
impact of using formal methods on reliability prediction and demonstration. The
commitmentto work in metric and model validation has led to an emphasis on the
topic of collection of software project data.
CR Subject Classification (1991): D.1, D.2.0 -1, D.4.5, C.4, B.4.5, K.6.4, J.7
ISBN-13:978-3-540-55212-3 e-ISBN-13:978-3-642-84725-7
001: 10.1007/978-3-642-84725-7
This work is subject to copyright.AII rights are reserved. whether the whole or part olthe material is concerned,
specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction
on microfilms or in other ways, and storage in data banks. Ouplication of this publication or parts thereof is
only permitted under the provisions of the German Copyright Law of September 9, 1965, in its current version
and a copyright fee must always be paid. Violations fall under the prosecution act of the German Copyright
Law.
Publication No. EUR 13538 EN of the
Commission of the European Communities,
Scientific and Technical Communication Unit,
Directorate-General Telecommunications, Information Industries and Innovation,
Luxembourg
LEGAL NOTICE
Neither the Commission of the European Communities nor any person acting on behalf of the
Commission is responsible for the use which might be made of the following information.
© ECSC - EEC - EAEC, Brussels - Luxembourg, 1992
Typesetting: Camera ready by author
45/3140-543210 - Printed on acid-free paper
Foreword
The first ESPRIT programme contained several ambitious projects. of which
REQUEST. with its wide brief covering all issues of assessment of quality
and reliability of software process and product. was one. Within REQUEST.
the research described in this volume. concerning those special problems of
software that is required to have extremely high reliability. was particularly
difficult and ambitious.
The problems of software reliability are essentially twofold. On the one
hand there is a concern with methods for achieving adequate reliability. on
the other hand there is a need to evaluate what has actually been achieved
in a particular case. Naturally. far more effort has been spent over the years
on the former problem; indeed. there is a sense in which all of conventional
software engineering can be seen as a response to this problem. However. it
is becoming clearer than ever that we can only claim to have a truly
sCientific approach. and so justify the description software engineering.
when we are able to measure the attributes of process and product.
It is still common to find software development methods recommended to
users on purely anecdotal grounds. This is not good enough. Rational
choices between rival approaches can only be made on the basis of
quantified costs and benefits.
Even more worrying is the tendency to argue that a software product can
be depended upon merely because it has been developed by honest men
using such anecdotal 'good practice'.
These concerns become extremely serious when we are dealing with
software that will play a safety-critic~ role - that. in the worst case. can kill
many people if it fails. Here it seems clear that special techniques are
required. over and above those used in ordinary best practice. and we need
objective evidence oj their e.fficacy. Perhaps even more important in cases
like this. we need to know that the actual software product is suffiCiently
dependable by evaluating its reliability. It is to these crucial issues
concerning the evaluation of process and product for very high reliability
that the work in this volume is addressed.
The difficulties here are immense. Consider. as an example. the analogy
that is sometimes drawn between hardware redundancy and software
diversity. In the hardware case it is often claimed that is possible to build a
system of arbitrarily high reliability from components of arbitrary
unreliability. Of course. such an assertion rests upon the assumption of
independence of random component failures. but this may sometimes be
quite plausible. In the case of failures arising from software faults (or indeed
from hardware design faults) in a design-diverse system. such an
assumption is simply false. and the theoretical modelling problems become
very hard.
vi
Issues of diversity. of version dependence. of adjudication between the
outputs of different versions are the subject matter of the volume. This is
new work at the frontiers of our current understanding. As Manfred
Kersken says in the Introduction. this is not intended for students; rather it
is a description of some significant new work which has advanced our
understanding in this difficult. but vitally important area of computer
science.
I was involved with the REQUEST project as a reviewer for almost five
years. This was a very enjoyable time. involving interesting discussions and
not a little argument. Now that I have had a chance to see all the results
collected together. I can only say that it confirms my original view that this
should be read by anyone with a professional interest in safety-critical and
fault-tolerant computing.
Bev Littlewood
Centre for Software Reliability
City University
London
Acknowledgement
The software fault-tolerance group of the REQUEST protect would like to
thank the Commission of the European Communities for supporting its
work. These thanks go especially to the CEC Project Officers of DG XIII.
Pierre-Yves Cunin. Jack Metthey and Jean-Jacques Lauture who have
accompanied the project with their most helpful administrative and pro
fessional advice.
We also would like to thank our project reviewers
Professor Bev Littlewood. The City University. London
Harry Sneed. SES GmbH. Neubiberg
Robert Troy. Verilog. Toulouse
Sinclair Stockman. British Telecom. Martlesham Heath
who have always taken a great interest in our work and contributed actively
to the success of the project. Their ideas and proposals as well as their
constructive criticism were always highly appreciated by our group.
The good quality of research and development work in projects like
REQUEST is always dependent on discussions and exchange of experience
among colleagues. We cannot name here the numerous colleagues within
the REQUEST project who contributed with fruitful discussions. ideas and
proposals to our work. but we would like to express here many thanks to all
of them. We are also grateful to our man at the wheel G. Hugh Browton.
STC. We know that it was not always easy to navigate such a heavy ship
through the reefs. but we always felt safe and fairly treated.
We gratefully acknowledge the publication of parts of this book by the
following publishers:
lEE & British Computer Society:
F. Saglietti: Software Diversity Metrics Quantifying Dissimilarity in the Input
Partition. In: lEE Software Engineering Journal. January 1990. Vo1.5. No.1
IEEE Computer Society Press:
F. Saglietti: Location of Checkpoints in Fault-Tolerant Software. In: Proc. of
the 5th Jerusalem Conf. on Information Technology (JCIT-5). Jerusalem (ILl.
October 1990
Elsevier / North-Holland:
F. Saglietti: A Theoretical Evaluation of the Acceptance Test as a Means to
Achieve Software Fault-Tolerance. In: Proc. of the IFIP/IFAC/EWICS Conf.
on Hardware and Software for Real-Time Process Control. Warsaw (PL).
May/ June 1988
viii
Pergamon Press:
F. Saglietti. W. Ehrenberger: Software Diversity - Some Considerations
about its Benefits and its Umitations. In: Proc. of the 5th IFAC Workshop on
Safety of Computer Control Systems (SAFECOMP 86). Sarlat (F). 1986
F. Saglietti: Strategies for the Achievement and Assessment of Software
Fault-Tolerance. In: Proc. of the 11th IFAC World Congress. Tallinn (USSR).
August 1990
Springer-Verlag:
F. Saglietti. M. Kersken: Quantitative Assessment of Fault-Tolerant Software
Architecture. In: Proc. of the 3rd Int. GIl ITGI GMA Conf. on Fault-Tolerant
Computing Systems. Informatik-Fachbertchte. Band-Nr. 147. F. Belli. W.
Gorke (Hrsg.). Bremerhaven (D). September 1987
F. Saglietti: The Impact of Voter Granularity in Fault-Tolerant Software on
System Reliability and Availability. In: Proc. of the 4th International GIl
ITGI GMA Conf. on Fault-Tolerant Computing Systems. Informatik
Fachbertchte. Band-Nr. 214. W. Gorke. H. Sorensen (Hrsg.). Baden-Baden
(D). September 1989
F. Saglietti: The Impact of Forced Diversity on the Failure Behaviour of
Multi-Version Software. In: Proc. of the GI & VDI/VDE-GMA Conf.
"Prozej3rechensysteme '91". Informatik-Fachbertchte. Band-Nr. 269. G.
Hommel (Hrsg.). Berlin (D). February 1991
Finally. our thanks go to Helga Moosmang for the careful preparation of
the manuscrtpt.
Garching. November 1991 M. Kersken. F. Saglietti
Table of Contents
Chapter 1
Introduction
Manfred Kersken
Chapter 2
Overview
Marta Teresa Malnini. Francesca Saglietti. David Nicholas Wall
2.1 The Concept of Software Fault-tolerance ________________ 5
2.2 Failure Dependence _______________________ 7
2.2.1 The Problem of Failure Dependence ________________ 7
2.2.2 Reduction of Failure Dependence _________________ 9
2.2.2.1 Forced Diversity _____________________ 9
2.2.2.2 Functional Diversity ___________________ 10
2.2.3 Measurement of Failure Dependence _--:,--________- -------- 10
2.2.3.1 Measurement by Statistical Inference from Past Failure Data _____ 10
2.2.3.2 Measurement by Static Analysis ________________ 11
2.2.3.3 Measurement by Dynamic Analysis _______________ 11
2.2.3.4 A Pattern Matching Approach _________________ 12
2.2.3.5 An Expert System Approach ________________ 13
2.2.3.6 Measurement of Functional Diversity ______________ 14
2.3 Evaluation of Reliability of Fault Tolerant Software ___________ 15
2.3.1 General Considerations _______- --,,--____________ 15
2.3.2 Model Application to Functionally Diverse Software __________ 16
2.4 AdJudication Mechanisms _____________________ 16
2.4.1 Voting Systems ______________________ 16
2.4.2 Acceptance Tests _______________________ 17
2.4.3 Location of CheckpOints ____________________ 17
2.5 Conclusion ___________________________ 17
Refurences ____________________________
18
Chapter 3
Considerations on Software Diversity on the Basis of
Experimental and Theoretical Work
Francesca Saglietti. Wolfgang Ehrenberger
3.1 The Different Failure Sets of a Two-fold Diverse System __________ 21
3.2 Experimental Approach 24
3.3 Theoretical Approach 27
3.4 Additional Requirements 29
3.5 Comparison Between Single and Diverse Use of Programs 29
3.6 ConclUSion 31
References 31
x
Chapter 4
The Impact of Forced Diversity on the Failure Behaviour
of Multiversion Software
Francesca Saglietti
4.1 Introduction ___________________________ 33
4.2 Common Failure Behaviour of Forced and Unforced Diverse Systems
w. r. t. the Voter Majority _____________________ 34
4.2.1 Theoretical Results of Littl~wood and Miller 34
4.2.2 Experimental Results of Kelly and Avizienis 35
4.3 Common Failure Behaviour of Forced and Unforced Diverse Systems
w. r. t. the Voter Granularity _____________________ 39
4.3.1 Theoretical Results 39
4.3.2 Experimental Results of PODS and STEM 42
4.4 Conclusion 45
References 45
Appendix 46
Chapter 5
Functional Diversity
Paola Burlando, Laura Gianetto, Marta Teresa Mainini
5.1 Introduction 49
5.2 Limitations of Normal Diversity 49
5.3 Description of Functional Diversity Methodology 50
5.4 Advantages of Functional with respect to Normal Diversity 51
5.5 Disadvantages of Functional Diversity 51
5.6 Application Fields 52
5.7 Choice of the Modelling Approach for Functional Diversity 53
5.8 Classical Semantic Approach 55
5.8.1 Operational Semantics 56
5.8.2 Denotational Semantics 59
5.9 Functional Semantics 63
5.10 Semantic Modelling of Functional Diversity 65
5.11 Functional Diversity Metrication 68
5.12 Definition of Functional Diversity Metrics 68
5.12.1 The EFF_WOR Metric 70
5.12.2 The IND_WOR and IND_AVE Metrics 71
5.12.3 The VER_WOR and VER_AVE Metrics 72
5.12.4 The GLO_REL Metric 73
5.13 Classification of the Metrics 74
5.14 Reliability AnalysIs for Functionally Diverse Systems 76
5.15 Static Specification Analysis 77
5.16 Reliability Evaluation 77
5.16.1 One Version Reliability Evaluation 77
5.16.2 System ReliabilIty Evaluation 78
5.17 Semantic Specification Language 89
5.17.1 Specification Language Characteristics for Functionally Diverse Systems __ 89
5.17.2 Guidelines for a Semantic Specification Language Definition 90
5.17.2.1 Declaration Block 90
5.17.2.2 Specification Body 94
5.17.3 Specification Structure 94
5.18 Semantic Specification AnalysIs Methodology 96
5.18.1 Static Specification AnalysIs 96
5.18.1.1 Diversity Degree Assessment 96
5.18.1.2 Reliability Evaluation 97
References 100
Appendix 102
xi
Chapter 6
Estimation of Failure Correlation in Diverse Software Systems
with Dependent Components
Francesca Saglietti
6.1 Introduction 115
6.2 Evaluation of the Inaccuracy Resulting from the Independence Assumption __ 115
6.3 The Case of Available Failure Observations 118
6.4 The Case of No Available Failure Observations 119
6.5 Conclusion 121
References 122
Appendix 123
Chapter 7
Measurement of Diversity Degree by Quantification of Dissimilarity
In the Input Partition
Francesca Saglietti
7.1 Input Partition and Coverage Diversity _________________ 125
7.2 Partition Diversity during the Testing Phase 128
7.3 Conclusion 132
References 132
Chapter 8
Comparison of Mnemonics for Software Diversity Assessment
Michael Martin Burke, David Nicholas Wall
8.1 The Initial Prototype Investigation __________________ 135
8.1.1 Initial Tests and Results 137
8.l.2 ShortCOmingS of the Prototype Technique 141
8.l.2.1 Length of Programs 141
8.l.2.2 Suitability of Trial Data 141
8.l.2.3 MatchtngAlgOrithm 141
8.l.2.4 Programming Style 142
8.1.2.5 Lack of Automation 142
8.l.2.6 Assessment of Results 142
8.2 Enhancement of the Prototype 142
8.2.1 Improvements to Overcome Identified Shortcomings 142
8.2.l.1 Automation of Mnemonic Code File Generation 142
8.2.l.2 Selection of Trial Data 142
8.2.l.3 Reducing the Effect of Noise 143
8.2.2 Tests with Improved Technique 143
8.3 Further Improvements to Technique 143
8.3.1 Selection of a Better Set of Test Data 143
8.3.2 Mathematical Comparison of Results and Presentation 144
8.3.3 Testing of Further Improvements 144
8.3.4 Results 145
8.4 Conclusions 145
References 146