IEEE Software (July/August)

83 Pages·2001·2.515 MB·English

requirements
Editor: Suzanne Robertson (cid:2) The Atlantic Systems Guild (cid:2) [email protected]
Are We Afraid of the Dark?
Welcome to the first IEEE Software column on Requirements. I am the column’s editor, Suzanne Robertson. My goal is to
make requirements and their importance more widely understood by developers, business people, and management. I am look-
ing for columnists who come from a variety of disciplines (not necessarily software development) and encourage them to com-
municate what requirements mean to them along with examples from their work. The aim is to make practical ideas about re-
quirements engineering accessible to a wide range of people. I would value your input on proposals for columns and ideas for
subjects that you would like to see the column cover—send your suggestions to [email protected].
R
equirements are hot. Conferences, acting as a translator, catalyst, imagineer,
books, courses, and now this column explorer, and trout tickler. We are still mak-
devoted to the subject all point to a ing the mistake of expecting people to know
growing interest in the discipline. In- precisely and exactly what they want and of
creasingly, my clients tell me that they being able to communicate those wants. So
recognize the importance of require- let’s look at our requirements skills and con-
ments and that they are ready to sider what we are doing well and where we
invest effort in making improve- can improve.
ments. They plan to formalize
their processes, standardize their Where are the brightest lights?
specifications, clarify require- Over the past 25 years we have developed
ments for users, and so forth. skills that help us define some types of re-
Mostly, they tell me that they are quirements. Our learning curve has led us
in the process of buying a tool. through structured, real-time, and object rev-
Of course these things are im- olutions. Techniques for modeling process,
portant, but it’s all too easy to re- data, and state have focused us on func-
treat into tools and technical tional requirements (the requirements that
software engineering details at focus on what a product must do). I see a lot
the expense of other less familiar aspects. of requirements specifications, and the func-
We are comfortable with tools, we believe tional requirements that are best specified
that methods and modeling formalisms will are those that reflect some kind of current
help us, and we feel secure with the amount reality. In this well-lit place, we can look at
of attention given to these aspects of the re- the way things are done now (by whatever
quirements discipline. Practically each combination of manual or automated
month a new vendor or author turns up the processes we can see), and we can build
lights and directs our attention to one of the models to specify a new system’s functional-
formal or mechanical (and thus easily auto- ity. We ask people what they want and they
mated) aspects of requirements. However, tell us what is uppermost in their minds,
that’s not all there is to requirements—there usually influenced by the status quo. How-
are other aspects of the field still illuminated ever, this approach is limited to finding con-
by little more than candlepower. scious requirements—things that people
I’m not seeing the effort put into the hard know they want. To find the unconscious
work of finding the requirements by listen- and undreamed-ofrequirements, we need to
ing, observing, and stimulating people by look into some dark corners.
12 IEEE SOFTWARE July/August 2001 0740-7459/01/$10.00 © 2001 IEEE
REQUIREMENTS
Where are the badly lit duce a software product that would mixture of cultures brought together
places? satisfy all the requirements of all the under the project’s umbrella and each
One place we need to look is in the users, no matter what country they other’s differences—huge differences
area of nonfunctional requirements worked in. Someone high in the orga- in stakeholder intentions, under-
such as look and feel, usability, oper- nization had the sense to get the key standing, meaning, and just about
ational environment, performance, stakeholders together before embark- every other damn thing. After four
maintainability, security, legality, and ing on the project. My main role was days, we came up with some consen-
culture. These requirements are usu- to help the different people talk to sus, but it was a different project
ally vaguely specified, if they are men- each other—they (mostly) wanted to, from the one that they had walked
tioned at all. Of all these require- but it was difficult. We spent a lot of into the room with on Monday.
ments, specifications for performance time trying to understand the diverse A group of stakeholders is like a
requirements are the least ambiguous.
There’s a good reason for this.
To make performance require-
ments unambiguous, we quantify
them using time. We have grown up
with time as an everyday scale of
measurement. If someone asks for
fast, our natural response is how
many days, seconds, years, cen-
turies...whatever we mean by fast.
The possession of this familiar and
well-understood scale of measure-
ment gives us the basis for explo-
ration because it makes us feel safe
and confident and helps us ask the
right questions.1,2
On the other hand, cultural re-
quirements are found in the darkest
places. We are not sure what we are
looking for or how we will know it
when we find it or even whether we
want to look for it in the first place,
but our ambitions make cultural re-
quirements increasingly critical. We
build products that involve multiple
technologies operating in multiple en-
vironments, and this inevitably in-
volves different people operating in a
multiplicity of what we refer to as cul-
tures. We talk about the culture of or-
ganizations, projects, countries, and
individuals and the need to build cul-
turally acceptable products, but what
do we really mean? When we try to
understand a culture, we are trying to
understand a particular civilization
well enough to help the inhabitants (in
our case, the project’s stakeholders)
discover their requirements.
The project sociology
A few months ago, I worked with
a group of 20 people including busi-
ness users, designers, managers, and
software engineers from seven coun-
tries. The project’s aim was to pro-
July/August 2001 IEEE SOFTWARE 13
REQUIREMENTS
family. In a family, everyone has dif- women do not shake hands with with and would not think to mention
ferent opinions and levels of impor- men? Did you know that when an to someone else. So many of our dif-
tance, yet, in a healthy family, the African-American wears a white rose ferences are reflected in what we eat,
members (without losing their indi- on Mother’s Day, it means that his or what we listen to, and in the words
viduality) collaborate to contribute her mother is dead? Why is it not ap- we use. Try having a meal with key
to the family’s overall happiness and propriate to send carnations to a stakeholders and ask questions about
success. Requirements engineers can French house? What does the color favorite foods, singers, or literature.
learn a lot about how to work with a purple mean in Spain? If you do not When you get an answer that sounds
stakeholder family from the work of know the answers and you are about unfamiliar, out of place, different, or
family therapists.3 to deploy software outside your own just plain wrong (to your ears), that’s
We gave the project a healthy start immediate geographical environ- an indication that you might have a
by devoting some time to under- ment, then you might be making cultural difference. Consider whether
standing the project’s sociology, ac- some cultural faux pas. The icons this difference might affect your pro-
tively searching for cultural differ- you have used, or the words, pic- ject’s success. If the answer is yes or
ences, and facilitating collaboration tures, or spelling, might have a dif- maybe, chances are you have identi-
between the project’s stakehold- ferent meaning from the ones you in- fied a relevant cultural requirement.
ers.4,5 This is difficult; listening to tended. The problem is that there are
people and helping them listen to so many questions to ask that it is Carry a flashlight
each other is hard work. It leads us hard to know where to start. Where else can we look to get the
into all sorts of dark and uncertain My father was a world traveler best advances in discovering the right
places where awareness of other peo- who was at home in many different requirements? Well, almost pre-
ple’s unpredictable behavior is our cultures. His way of getting to know dictably, the best places to look are the
only clue to what is going on. a culture was to explore the food, darkest. I have discussed the advisabil-
music, and language. This provides ity of investigating cultural require-
Finding cultural requirements an insight into people’s everyday ments, but there’s more. Look and feel
Does it surprise you that Muslim lives—things that they are familiar is another area worth illuminating.
cluster computing
collaborative computing
dependable systems
distributed agents
distributed databases
distributed multimedia
grid computing
middleware
mobile & wireless systems
operating systems
real-time systems
security
IEEE
D S O
istributed ystems nline
c o m p u t e r . o r g / d s o n l i n e
14 IEEE SOFTWARE July/August 2001
REQUIREMENTS
What is the most appropriate ap- Another dark area that we need to enth incarnation of the payroll sys-
pearance that your product should explore is creative invention.7People tem that your company first imple-
have? Will your product look like all can tell us only so much of what they mented in 1976 and has changed
the rest, or is there a special appear- want; we (the requirements engi- very little in the intervening years,
ance that will distance it from its neers) must invent and inspire the then look no further. If you want
competitors? This might seem like a rest. Customers can tell us how they your next product to anticipate the
trivial exercise, but remember that would like the existing product to be future, to be one that its users like, to
the world’s best-known brands got improved, but they cannot tell us offer significant advantages to your
that way by paying attention to how what they would like us to invent for organization, then “requirements”
their products appear.6To appreciate them. Did customers ask for their has to take on a different meaning. It
the importance of look and feel, go telephones to be portable? I think has to mean looking into areas that
to the Web sites of well-known that most of us asked for speed dial, are vague or unexplored, poking into
brands such as Coca-Cola, Federal caller identification, and so on, but the unlit corners to discover dark se-
Express, McDonald’s, and so forth not for the mobile phone—it was in- crets, and inventing what your users
and find their corporate branding vented for us. How many of the truly never dreamed of.
pages. The good news is that we successful products—including soft-
don’t need to invent ways of specify- ware products—were invented? How
ing look and feel requirements; we much of successful software is inno-
already have experts available. Once vation instead of re-implementation?
you have defined the image that the This is an area that deserves signifi-
product is aiming for (the business cant illumination.
requirement), then call for help from
graphic artists. Graphic artists are
specialists in color, type, material—
all the things that give a product the References
desired image. 1. T. Gilb and S. Finzi, Principles of Software
Engineering Management, Addison-Wesley,
The idea of calling on expert help
Wokingham, England, 1988.
for specifying details works well for W 2. J. Robertson and S. Robertson, Mastering the
many nonfunctional requirements. hy don’t we look into the dark? Requirements Process, Addison-Wesley, New
York, 1999.
Legal, security, usability, environmen- Are we afraid that it represents
3. V. Satir et al., The Satir Model: Family Ther-
tal, and many other areas have recog- an unknown quantity and is apy and Beyond, Behavior Books, Palo Alto,
nized experts. We can ask these peo- therefore dangerous? If we have little Calif., 1991.
4. H. Beyer and K. Holtzblatt, Contextual De-
ple to act as consultant stakeholders, time before our deadline, do we want
sign: Defining Customer-Centered Systems,
provided we give them a clear specifi- to spend it looking into areas that Morgan Kaufmann, San Francisco, 1998.
cation of the business requirement. our tools and methods don’t men- 5. J.A. Highsmith III, Adaptive Software Devel-
opment: A Collaborative Approach to Manag-
How much effort do you spend tion? I guess the answer depends on
ing Complex Systems, Dorset House, New
observing your clients, customers, the kind of product you intend to York, 2000.
and users at work? How much time build. If your product is to be the sev- 6. D. Ogilvy, Ogilvy on Advertising, Pan Books,
London, 1984.
do you spend sitting with them as
7. R. von Oech, A Whack on the Side of the
they do their daily tasks? How much Head: How You Can Be More Creative, Cre-
learning have you put in to discover- ative Think, Menlo Park, Calif., 1998.
ing what kind of people they are or
imagining what kinds of products
would do them the most good?
When you take your people away
It is this mysterious,
from their tasks to interview them,
sometimes seemingly
you are destroying the very link that
you most need—the link between unfathomable feeling that
that person and his or her job. It is
people have for their
this mysterious, sometimes seemingly
unfathomable feeling that people work that helps you
have for their work that helps you
learn what kind of
learn what kind of product they
Suzanne Robertsonis a principal and founder of The
need. These are the sorts of require- product they need. Atlantic Systems Guild, a London- and New York-based think
ments that people do not mention— tank. She specializes in the field of requirements and is coauthor
of Volere, a widely used approach to requirements engineering.
until you implement a product that
Contact her at [email protected]; www.systemsguild.
lacks them. com.
July/August 2001 IEEE SOFTWARE 15
quality time
Editor: Jeffrey Voas (cid:2) Cigital (cid:2) [email protected]
Composing Software
Component “ilities”
Jeffrey Voas, Cigital
I
recently had the opportunity to attend system results with the desired functionality,
part of the Component-Based Software given a system created solely by joining A
Engineering (CBSE) Workshop at this and B.
year’s International Conference on Increasingly, our community is discover-
Software Engineering in Toronto. One ing that even if FC were a solved problem
of the more interesting discussions dealt (using formal methods, architectural design
with the issue of certification. The speakers approaches, model checking, and so on), it’s
and participants discussed how to calculate not mature enough to experience other seri-
the “ilities” of two software ous concerns that arise in CBSE and CBD.
components (hooked in a simple These concerns stem from the problem of
series) for a composition before composing “ilities.”
they’re joined and executed. (“il- Composing “ilities” is complicated be-
ities” is my term for the nonfunc- cause we can’t, for example, know a priori
tional properties of software about the security of a system composed of
components that define charac- two components, Aand B, based on knowl-
teristics such as security, reliabil- edge of the components’ security. This is be-
ity, fault tolerance, performance, cause we base the composite’s security on
availability, and safety.) The work- more than just the individual components’
shop impressed on me a sense that security. Numerous reasons for this exist;
a fruitful research area for the here, we’ll just discuss component perfor-
software engineering community exists here, mance and calendar time.
so I have opted to share some of the interest- Suppose that A is an operating system
ing issues related to “ility” composability here and B is an intrusion detection system. Op-
in this column. erating systems have some authentication
security built into them, and intrusion de-
A few examples tection systems have some definition for the
For the past 10 years, much of the work types of event patterns that often warn of an
on CBSE and component-based develop- attack. Thus, the composition’s security
ment (CBD) dealt with functional compos- clearly depends on the individual compo-
ability. FC deals with whether F(A) ξF(B) = nents’ security models. But even if A has a
F(AξB) is true (where ξis some mathemat- weak security policy or flawed implementa-
ical operator)—that is, whether a composite tion, the composite can still be secure. If you
16 IEEE SOFTWARE July/August 2001 0740-7459/01/$10.00 © 2001 IEEE
QUALITY TIME
make A’s performance so poor that no one worrisome in commercial off-the-shelf
can log on—that is, if the operating system (COTS) software products. Nonfunctional
authenticates inefficiently—security is actu- behaviors can include malicious code (such
ally increased. And, if the implementation as Trojan horses and logic bombs) and any
of A’s security mechanism is so unreliable other undocumented behavior or side effect.
that it disallows all users—even legitimate Another worrisome problem facing CBSE
ones—access, security is again increased. and CBD is hidden interfaces. Hidden inter-
Although these examples are clearly not a faces are typically channels through which
desirable way to attain higher system secu- application or component software can
rity, both do decrease the likelihood that a convince an operating system to execute un-
system will be successfully attacked. desirable tasks or processes. An example
If we reuse our example of Aas an oper- would be an application requesting higher
ating system and B as an intrusion detec- permission levels than the application
tion system and this time assume that A should have. Interestingly, fault injection
provides excellent security and B provides can partially address this issue by detecting
excellent security, we must accept that B’s hidden interfaces and nonfunctional behav-
security is a function of calendar time. This iors by forcing software systems to reveal
is simply because we are always discovering those behaviors and events after a COTS
new threats and ways to “break in.” Even component’s input stream receives cor-
if you could create a scheme such as Secu- rupted inputs.
rity(A) ξ Security(B) = Security(A ξ B), Se- In addition to reliability and security,
curity(B) is clearly a function of the version performance appears—at least on the sur-
of Bbeing composed and what new threats face—to be the “ility” with the best possi-
have arisen. bility for successful composability. This
possibility is problematic, however, from a
Easy “ilities”? practical sense. Even if you performed a
Which “ilities,” if any, are easy to com- Big O algorithmic analysis on a compo-
pose? Unfortunately, the answer to this nent, the component’s performance after
question is that no “ilities” are easy to com- composition depends heavily on the hard-
pose, and some are much harder than oth- ware and other physical resources. What
ers. Furthermore, we lack widely accepted we most likely need, then, is to drag many
algorithms for the composition. I just different hardware variables along with a
demonstrated this problem for security, but certificate that makes even minimal, worst-
the same holds true for other areas such as case claims about the component’s perfor-
reliability. For reliability, consider a two- mance. Clearly, this can introduce serious
component system in which component A difficulties.
feeds information in B, and B produces the
composite’s output. Assume that both com-
ponents are reliable. What can we assume
about the composite’s reliability? Although
it certainly suggests that the composite sys-
tem will be reliable, the components (which
were tested in isolation for their individual
reliabilities) can suddenly behave unreliably
when connected to other components, par-
ticularly if the isolated test distributions did
F
not reflect the distribution of transferred in- or CBSE and CBD to flourish, technolo- No “ilities”
formation after composition. Moreover, we gies must exist that let you successfully
are easy to
could have nonfunctional component be- predict software component interoper-
haviors, which we can’t observe and don’t ability before composition occurs. Without compose, and
manifest until after composition occurs. predictability, interoperability cannot be some are much
Such behaviors can undermine the composi- known a priori until after a system is built.
harder than
tion’s reliability. Finally, if one of the com- At that point, it might be too late in the life
ponents is simply the wrong component— cycle to recover financially if you discover others.
although highly reliable—the resulting that certain components are incompatible.
system will be useless. Therefore, predictive technologies that ad-
Nonfunctional behaviors are particularly dress the “ilities” are truly needed.
July/August 2001 IEEE SOFTWARE 17
focus
guest editor’s introduction
Software Fault Tolerance:
Making Software Behave
Jeffrey Voas, Cigital
etween the late 1960s and early 1990s, the software engineer-
B
ing community strove to formalize schemes that would lead to
perfectly correct software. Although a noble undertaking at
first, it soon became apparent that correct software was, in gen-
eral, unobtainable. And furthermore, the costs, even if achievable, would
be overwhelming.
18 IEEE SOFTWARE July/August 2001 0740-7459/01/$10.00 © 2001 IEEE
About the Author
Jeffrey Voasis a cofounder and chief scientist of Cigital. His research interests include
composition strategies for COTS software, software product certification and warranties, and
software quality measurement. He coauthored Software Fault Injection: Inoculating Programs
Modern software systems, even if cor-
Against Errors(Wiley, 1998) and is working on Software Certificates and Warranties: Ensuring
rect, can still exhibit undesirable behaviors Quality, Reliability, and Interoperability. He received his BS in computer engineering from Tu-
as they execute. How? Well, the simplest ex- lane University and his PhD in computer science from the College of William & Mary. He was
the program chair for the Eighth IEEE International Conference on Engineering of Computer-
ample would be if a software system were
Based Systems. He was named the 1999 Young Engineer of the Year by the District of Colum-
forced to experience an input it should not bia Council of Engineering and Architectural Societies, was corecipient of the 2000 IEEE Relia-
have. In this situation, the software could bility Engineer of the Year award, and received an IEEE Third Millennium Medal and an IEEE Computer Society Meritorious
Service award. He is a senior member of the IEEE, a vice president of the IEEE Reliability Society, and an associate editor
in chief on IEEE Software. Contact him at [email protected].
1. handle it gracefully by ignoring it,
2. execute on it and experience no ill ef- The results concluded that the cost over-
fects, or head varies from 25 to 134 percent accord-
3. execute on it and experience catastrophic ing to the development phase.
effects. Les Hatton’s article, “Exploring the Role
of Diagnosis in Software Failure,” builds on
Note that 1 and 2 are desirable, but 3 is not, the premise that software systems have,
yet in all cases, the software is still correct. among engineering systems, the unique char-
This issue’s focus is dedicated to the re- acteristic of repetitive failure. His article ex-
search results and ideas from a group of ex- plores various reasons for this situation, par-
perts who discuss their views on how to cre- ticularly poor diagnosability. Hatton argues
ate fault-tolerant software—that is, software that this cause exists largely because of edu-
that is designed to deliberately resist exhibit- cational problems. Through examples,
ing undesirable behaviors as a result of a thearticle highlights the need for an attitude
failure of some subsystem. That subsystem change toward software failure and for im-
could be part of the software, an external proved diagnostics. Finally, he introduces
software or hardware failure, or even a hu- the concepts of diagnostic distance and diag-
man operator failure. Generally speaking, nostic quality to help with categorization.
fault-tolerant software differs from other Michel Raynal and Mukesh Singhal’s ar-
software in that it can gracefully handle ticle deals with ways to overcome agree-
anomalous internal and external events that ment problems in distributed systems. The
could lead to catastrophic system conse- authors focus on practical solutions for a
quences. Because correct software is an oxy- well-known agreement problem—the non-
moron in most cases and, as I just men- blocking atomic commitment.
tioned, correct software can still hurt you, Finally, William Yurcik and David Doss’s
software fault tolerance is one of the most article addresses software aging. The article
important areas in software engineering. discusses two approaches to this problem:
The first article, “Using Simplicity to Con-
trol Complexity,” by Lui Sha begins by dis- (cid:2) provide a system with the proactive capa-
cussing the widely held belief that diversity in bility of reinitializing to a known reliable
software constructions entails robustness. state before a failure occurs or
The article then questions whether this is re- (cid:2) provide a system with the reactive capa-
ally true. It goes on to investigate the rela- bility of reconfiguring after a failure oc-
tionship between software complexity, relia- curs such that the service provided by
bility, and the resources available for the software remains operational.
software development. The article also pres-
ents a forward recovery approach based on The authors discuss the complementary na-
the idea of “using simplicity to control com- ture of these two methods for developing
plexity” as a way to improve the robustness fault-tolerant software and give the reader a
of complex software systems. good overview of the field in general.
Karama Kanoun’s article analyzes data So in conclusion, I hope that after you
collected during the seven-year development read these articles you will have a better un-
of a real-life software system. The software derstanding of the underlying principles of
under consideration comprised two diverse software fault tolerance. All systems need de-
variants. For each development phase, fensive mechanisms at some point. These ar-
Kanoun evaluated the cost overhead in- ticles, along with the references in the
duced by the second variant’s development Roundtable (see p. 54), provide information
with respect to the principal variant’s cost. on how to get started.
July/August 2001 IEEE SOFTWARE 19
focus
fault tolerance
Using Simplicity to Control
Complexity
Lui Sha, University of Illinois at Urbana-Champaign
ccording to a US government IT initiative, “As our economy and
A
society become increasingly dependent on information technology,
we must be able to design information systems that are more se-
cure, reliable, and dependable.”1 There are two basic software reli-
Does diversity
ability approaches. One is fault avoidance, using formal specification and ver-
in construction
ification methods2 and a rigorous software development process. An example
improve
of a high-assurance software development process is the DO 178B standard
robustness?
The author
adopted by the US Federal Aviation Adminis- Rather, it is the existence of a simple and reli-
investigates
tration. Fault avoidance methods allow able core component that ensures the system’s
the relationship
computer-controlled safety-critical systems critical functions despite the failure of non-
between such as flight control, but they can only core software components. I call this ap-
complexity, handle modestly complex software. The proach using simplicity to control complexity.
trend toward using a large network of sys- I will show how to use the approach system-
reliability, and
tems based on commercial-off-the-shelf atically in the automatic-control-applications
development
components (COTS) also makes applying domain, creating systems that can manage up-
resources, fault avoidance methods more difficult. grades and fix themselves when complex soft-
and presents Another approach is software fault toler- ware components fail.
ance through diversity (for example, using the
an approach to
N-version programming method3). Many be- The power of simplicity
building a system
lieve that diversity in software construction re- Software projects have finite budgets.
that can manage sults in improved robustness, but is that true? How can we allocate resources in a way that
upgrades and Would the system be more reliable if we de- improves system reliability? Let’s develop a
voted all our effort to developing a single ver- simple model to analyze the relationship be-
repair itself
sion? In this article, I show that dividing re- tween reliability, development effort, and soft-
when complex
sources for diversity can lead to either ware’s logical complexity. Computational
software improved or reduced reliability, depending on complexity is modeled as the number of steps
components fail. the architecture. The key to improving relia- to complete the computation. Likewise, we
bility is not the degree of diversity, per se. can view logical complexity as the number of
20 IEEE SOFTWARE July/August 2001 0740-7459/01/$10.00 © 2001 IEEE
1
0.8
C = 1
steps to verify correctness. Logical complexity
y 0.6
itsh aat ftuhnec tvioernif iocfa ttihoen nourm tbesetri nogf pcraosecse s(ss tmatuesst) abilit 0.4 C = 2
handle. A program can have different logical Reli
0.2
and computational complexities. For exam-
ple, compared to quicksort, bubble sort has
lower logical complexity but higher computa-
0 2 4 6 8 10
tional complexity. Degrees of effort
Another important distinction is the one
between logical complexity and residual log-
ical complexity. For a new module, logical Figure 1. Reliability
complexity and residual logical complexity rate with respect to development effort. P3 and complexity.
are the same. A program could have high log- implies that diversity is not free (diversity ne- C is the software
ical complexity initially, but if users verified cessitates dividing the available effort). complexity.
the program before and can reuse it as is, the
residual complexity is zero. It is important to A simple reliability model
point out that we cannot reuse a known reli- The following model satisfies the three pos-
able component in a different environment, tulates. We adopt the commonly used expo-
unless the component’s assumptions are sat- nential reliability function R(t) = e–λt and as-
isfied. Residual complexity measures the ef- sume that the failure rate, λ, is proportional to
fort needed to ensure the reliability of a sys- the software complexity, C, and inversely pro-
tem comprising both new and reused portional to the development effort, E. That
software components. I focus on residual log- is, R(t) = e–kCt/E. To focus on the interplay be-
ical complexity (just “complexity” for the re- tween complexity and development effort, we
mainder of the article) because it is a domi- normalize the mission duration t to 1 and let
nant factor in software reliability. From a the scaling constant k = 1. As a result, we can
development perspective, the higher the com- rewrite the reliability function with a normal-
plexity, the harder to specify, design, develop, ized mission duration in the form R(E, C) =
and verify. From a management perspective, e–C/E. Figure 1 plots the reliability function
the higher the complexity, the harder to un- R(E, C) = e–C/Ewith C= 1 and C= 2, respec-
derstand the users’ needs and communicate tively. As Figure 1 shows, the higher the com-
them to developers, find effective tools, get plexity, the more effort needed to achieve a
qualified personnel, and keep the develop- given degree of reliability. R(E, C) also has a
ment process smooth without many require- monotonically decreasing rate of reliability
ment changes. Based on observations of soft- improvement, demonstrating that the remain-
ware development, I make three postulates: ing errors are subtler and, therefore, detecting
and correcting them requires more effort. Fi-
(cid:2) P1: Complexity breeds bugs. All else be- nally, the available budget E should be the
ing equal, the more complex the soft- same for whatever fault-tolerant method you
ware project, the harder it is to make it use.
reliable. We now have a simple model that lets us
(cid:2) P2: All bugs are not equal. Developers analyze the relationship between development
spot and correct the obvious errors early effort, complexity, diversity, and reliability.
during development. The remaining er- The two well-known software fault toler-
rors are subtler and therefore harder to ance methods that use diversity are N-
detect and correct. version programming and recovery block.3–5
(cid:2) P3: All budgets are finite. We can only I’ll use them as examples to illustrate the
spend a certain amount of effort model’s application. For fairness, I’ll compare
(budget) on any project. each method under its own ideal condition.
That is, I assume faults are independent under
P1 implies that for a given mission duration N-version programming and acceptance test is
t, the software reliability decreases as com- perfect under recovery block. However, nei-
plexity increases. P2 implies that for a given ther assumption is easy to realize in practice
degree of complexity, the reliability function (leading to the forward-recovery approach,6
has a monotonically decreasing improvement which I’ll discuss later).
July/August 2001 IEEE SOFTWARE 21