Mastering Software Quality Assurance: Best Practices, Tools and Technique for Software Developers

377 Pages·2010·1.04 MB·English

by  Murali Chemuturi

MASTERING
SofTwARE QuAlITy
ASSuRANcE

Best Practices, Tools and Techniques
for Software Developers

MuRAlI chEMuTuRI

J. Ross Publishing; All Rights Reserved

Copyright © 2011 by Murali Chemuturi

ISBN 978-1-60427-032-7

Printed and bound in the U.S.A. Printed on acid-free paper
10
9
8
7
6
5
4
3
2
1

Library of Congress Cataloging-in-Publication Data

Chemuturi, Murali, 1950-
Mastering software quality assurance : best practices, tools and techniques for
software developers / by Murali Chemuturi.
p.
cm.
Includes index.
ISBN 978-1-60427-032-7 (hardcover : alk. paper)
1. Computer software—Development. I. Title.
QA76.76.Q35C45 2010
005.1′4—dc22
2010019184

This publication contains information obtained from authentic and highly regarded
sources. Reprinted material is used with permission, and sources are indicated. Reasonable
effort has been made to publish reliable data and information, but the author and the pub-
lisher cannot assume responsibility for the validity of all materials or for the consequences
of their use.
All rights reserved. Neither this publication nor any part thereof may be reproduced,
stored in a retrieval system or transmitted in any form or by any means, electronic, me-
chanical, photocopying, recording or otherwise, without the prior written permission of the
publisher.
The copyright owner’s consent does not extend to copying for general distribution for
promotion, for creating new works, or for resale. Specific permission must be obtained from
J. Ross Publishing for such purposes.
Direct all inquiries to J. Ross Publishing, Inc., 5765 N. Andrews Way, Fort Lauderdale,
Florida 33309.

Phone: [phone]
Fax: [phone]
Web: www.jrosspub.com

J. Ross Publishing; All Rights Reserved

TABLE OF CONTENTS

Foreword ............................................................................................................. vii
Preface .................................................................................................................. ix
About the Author ............................................................................................. xiii
Acknowledgments ............................................................................................... xv
Web Added Value™ ........................................................................................ xvii

Chapter 1. Quality Assurance Basics ............................................................... 1
Connotations of the Word Quality ................................................................... 1
What Is Quality? .................................................................................................. 2
Specifications ........................................................................................................ 3
Definition of Quality from the Standpoint of the Provider ........................... 4
Quality and Reliability ........................................................................................ 5
Evolution of the Concepts of Quality ............................................................... 8
Quality Gurus ..................................................................................................... 11
Total Quality Management ............................................................................... 16
Are We Giving Adequate Importance to Quality in Organizations? ........... 17
Organizational Goals and Quality Goals ......................................................... 20
Is a Quality Department in Software Development Organizations
Really Needed? ................................................................................................... 22
The Present Scenario in Software Development Organizations ................... 23

Chapter 2. Four Dimensions of Quality ........................................................ 25
Background ......................................................................................................... 25
Specification Quality .......................................................................................... 26
Design Quality .................................................................................................... 27
Development (Software Construction) Quality .............................................. 29
Conformance Quality ........................................................................................ 30

iii
J. Ross Publishing; All Rights Reserved

iv
Mastering Software Quality Assurance

Ensuring Quality in Specifications ................................................................... 31
Ensuring Quality in Design .............................................................................. 32
Ensuring Quality in Development (Software Construction) ........................ 33
Ensuring Conformance Quality ....................................................................... 33

Chapter 3. Software Product Quality ............................................................ 35
Functionality Standpoint ................................................................................... 35
White Box (Glass Box) Standpoint ................................................................. 38
Presence of Defects in the Product ................................................................. 41
Program Quality ................................................................................................ 44
Measurement of Product Quality .................................................................... 48

Chapter 4. Organizational Environment that Fosters a Quality
Culture ................................................................................................................ 61
Quality and Organizational Environment ....................................................... 61
Need for an Independent Quality Assurance Department ........................... 62
The Role of the Quality Assurance Department ............................................ 64
The Position of the Quality Assurance Department in an Organization ... 66
Organization of the Quality Assurance Department ..................................... 68
Organization and Staffing of the Quality Assurance Department ............... 74
A Well-Defined and Institutionalized Software Development Process ....... 76
Explicit System of Rewards and Recognition for Achieving Excellence
in Quality ............................................................................................................ 80
Commitment and Involvement of Senior Management in Fostering a
Culture of Quality in the Organization .......................................................... 82
Final Words ........................................................................................................ 83

Chapter 5. Software Verification .................................................................... 85
Verification ......................................................................................................... 85
Walkthroughs (Peer Reviews) .......................................................................... 89
Inspections ........................................................................................................ 102
Audits ................................................................................................................ 110
Verification Process ......................................................................................... 124
Implementation of Verification Activities in Projects ................................. 126

Chapter 6. Validation ..................................................................................... 129
Definition of Validation .................................................................................. 129
Validation of Software Designs ...................................................................... 132
Validating the Product Specifications ............................................................ 133
Validating the Software Product .................................................................... 133

J. Ross Publishing; All Rights Reserved

Table of Contents
v

Testing Different Types of Software Products ............................................. 135
Testing Basics ................................................................................................... 139
Approaches to Testing ..................................................................................... 143
Test Case Design .............................................................................................. 146
Test Environment ............................................................................................ 161
Testing Scenarios ............................................................................................. 163
Project Testing or Embedded Testing ........................................................... 163
Product Testing ................................................................................................ 169
Best Practices in Testing ................................................................................. 178
Automation of Testing and Use of Testing Tools ....................................... 183
Final Words about Software Testing ............................................................. 186

Chapter 7. Software Product Quality: Reliability ...................................... 187
Software Disasters ............................................................................................ 187
Software Reliability .......................................................................................... 189
Cause of Software Failures ............................................................................. 192
Prediction of Software Reliability .................................................................. 194
Software Reliability Improvement .................................................................. 195

Chapter 8. Process Quality ............................................................................ 197
Process Quality Evolution ............................................................................... 197
Process ............................................................................................................... 199
Process Quality ................................................................................................. 200
Process Definition ............................................................................................ 201
Aligning the Process with a Process Model .................................................. 205
Process Improvement ...................................................................................... 206
Process Stabilization ........................................................................................ 209
Software Development Process ...................................................................... 211
Components of a Process ............................................................................... 211
Process Certification ........................................................................................ 213

Chapter 9. New Paradigm for Software Quality ........................................ 217
Current Certification Paradigms .................................................................... 217
The Fallacy of Certifications ........................................................................... 219
Criticisms of Maturity Models ....................................................................... 221
A New Paradigm for Software Quality Assurance ....................................... 228
Final Words ...................................................................................................... 233

Appendix A. Audit Process ............................................................................ 235

Appendix B. Defect Resolution Methodology ............................................ 247

J. Ross Publishing; All Rights Reserved

Appendix C. Guidelines for Error Guessing ............................................... 257

Appendix D. Guidelines for Graphical User Interface Quality
Conformance ................................................................................................... 263

Appendix E. Guidelines for Stress Testing ................................................. 273

Appendix F. Guidelines for Negative Testing ............................................. 279

Appendix G. Measurement of Quality ......................................................... 287

Appendix H. Quality Assurance of Databases ............................................ 305

Appendix I. Coding Guidelines .................................................................... 309

Appendix J. Sample Review Process ............................................................ 323

Appendix K. Software Quality Assurance Plan ........................................... 337

Appendix L. Abbreviations ............................................................................ 345

Index ................................................................................................................. 347

vi
Mastering Software Quality Assurance

J. Ross Publishing; All Rights Reserved

FOREWORD

As I sit surrounded by the majestic forests and
glacial lakes in the North American state of
Maine, I am reminded of how the laws of nature,
planet Earth’s natural processes, have carefully
constructed, nurtured, and sustained this stun-
ning example of natural engineering and process
implementation.
Strict compliance with this natural process,
rather than process experimentation and impro-
visation, has allowed these forests to grow and thrive unencumbered by the
special cause of variation from destructive forces. Indeed, the natural world has
suffered from attempts to circumvent this process with disastrous results, and
when left to recover does so with alarming alacrity and efficiency. There is much
to be learned from this natural process that can be applied to the software
engineering discipline.
Granted, software engineering does not have the luxury of unlimited time
and resources, and business will not wait eons for change to occur. But the
application of and compliance with a basic process architecture—which in-
cludes at its foundation software quality assurance—is the key to wringing value
from engineering process improvement.
A thoughtfully constructed architecture with software quality assurance serv-
ing the foundational role of mentor, messenger, and accelerator will lay a foun-
dation for flexibility in a process designed to serve and support a wide variety
of projects with different objectives. In this sense, we require a “set of standard
processes,” not a single standard process, that comply with our process archi-
tecture and are supported by a robust and flexible software quality assurance
organization.

vii
J. Ross Publishing; All Rights Reserved

Like a great concert violinist who spends years embracing and developing the
discipline and mechanics of his or her craft before learning his or her first
concerto and venturing onto the concert stage, software engineers cannot achieve
creative success without embracing an appropriate process architecture and
learning to master their craft by first mastering the natural laws that guide them.
Too long has software engineering been hampered by cowboy-style coding—
a behavior that appears heroic at the time, but that proves to be damaging in
so many ways in the long run. It’s high time someone wrote at length about the
role of software quality assurance and process compliance in the software en-
gineering field. Murali’s book breaks new ground and gives us a glimpse into
the promise of disciplined, productive, and efficient software engineering. It
paints a picture of a brighter future for an industry that has long been suffering
from cost and quality issues and will allow software engineers to reach new levels
of performance and creativity.

Jeff Dalton
President and CEO, Broadsword Solutions Corporation
www.broadswordsolutions.com
SCAMPI Lead Appraiser
www.askTheCMMIAppraiser.com

viii
Mastering Software Quality Assurance

J. Ross Publishing; All Rights Reserved

PREFACE

Gerald M. Weinberg, author of the book The Psychology of Computer Program-
ming, is attributed with the quote “If builders built houses the way programmers
built programs, the first woodpecker to come along would destroy civilization.”
According to an anonymous quote, “Software and cathedrals are much the
same; first we build them, then we pray.” And the confidence in software de-
velopers continues to grow.
If manufacturers controlled and assured quality the way software developers
do, I think many accidents and injuries, even fatalities, would be the sad result.
Many, many improvements have been made in the way programs are written,
but even after nearly 50 years of software development history, the approach to
software quality still leaves much to be desired. While manufacturing learned
from construction and construction in turn learned from manufacturing, soft-
ware development stubbornly refuses to learn from either industry, especially in
the matter of quality assurance of deliverables.
At the beginning of my career in the 1970s, I was employed at a manufac-
turing organization that produced control instrumentation for atomic reactors.
In that organization, quality was sacrosanct and very rigorous. The result of such
high standards of quality is that in a third-world country (as it was in the 1970s,
when the reactors were built), the atomic reactors have been working without
accident or mishap for the last 40 years! I attribute that record to the importance
that was given to quality assurance. I narrate but one incident from that expe-
rience: On the last day of the fiscal year, the quality department held up a
shipment that would have added significantly to the financial results, as it found
a “tiny scratch” on the painted surface of the back side of one piece of the
equipment. Even though the scratched surface would have been facing a wall,
rectification was insisted upon, and the shipment was released late in the evening
only after the “tiny scratch” was fixed. The paint shop guys worked furiously

ix
J. Ross Publishing; All Rights Reserved

to repair that insignificant defect. The boss did not berate the quality depart-
ment for pointing out an insignificant defect, but rather educated the handlers
on the necessity of careful handling.
Contrast this with the software industry, where important software such as
an operating system is shipped with so many bugs that it needs a service pack
within three months of its release to make it function and another service pack
a year later to fix the remaining bugs. Windows 2000 had four service packs,
XP had three service packs, and a service pack for Vista has already been re-
leased. I have come across a few situations where software is delivered with
known defects to be corrected as part of warranty services or product support.
When I switched over to the software development industry in the early
1980s, my first surprise was that there was no inspection or testing of the pro-
grams developed. The programmer certified the program, and if there were any
issues with it, they were due to improper data or usage. The programmer would
make the program better, if and when necessary. Never was it said that perhaps
the program lacked quality, nor did anybody accept that inferior quality was a
possibility.
Things did change later on. Peer reviews and testing emerged as standard
procedure, followed soon after by independent verification and validation. But
even now, I do not find the rigor of software quality assurance to be anywhere
near the rigor of quality assurance that I was used to in the manufacturing
industry where I was employed in the early 1970s. Most people in the software
development industry abhor the terms “inspection” and “testing” and use veri-
fication and validation instead. Whereas inspection and testing imply detail and
have a ring of authority to them, verification and validation somehow connote
cursory actions that lack any true authority. In manufacturing, inspection is
carried out by persons specialized in inspection, and testing is carried out by
persons specialized in testing. In the software industry, however, the “indepen-
dent” in independent verification and validation means someone other than the
person who programmed the software; it does not mean a specialist.
In manufacturing, there are no agencies that certify the maturity of capabil-
ity, such as the Software Engineering Institute (SEI) of Carnegie Mellon Uni-
versity, except for the International Organization for Standardization (ISO),
which certifies for process quality. While automobiles have been manufactured
for over 100 years, an automobile manufacturing capability maturity model does
not exist! A quality department is sine qua non in the manufacturing industry
even though no certification model requires it. In the software development
industry, however, no model suggests that a quality department is necessary.
I know of many software development organizations that do not have a
quality assurance department. Most of these organizations are ISO certified and

x
Mastering Software Quality Assurance

J. Ross Publishing; All Rights Reserved

rated at a Capability Maturity Model or Capability Maturity Model Integration
maturity level 3 or higher by SEI. The software development industry seems to
understand that quality assurance is nothing but testing. You can find many
advertisements seeking applications for the position of quality assurance engi-
neer—to do software testing. How this misunderstanding came about, I do not
know, but nothing could be more wrong than this misconception.
There is a lot of room for improvement in quality assurance in the software
development industry, and a comprehensive reference on quality assurance is
needed that ties together all aspects of quality assurance, not as the software
development industry does but in its true spirit. Hence this book.
Feel free to e-mail me at