Foundations of Software Testing, 2nd Edition

728 Pages·4.072 MB·English

by  Mathur, Aditya P

Foundations of
Software Testing
SECOND EDITION
Fundamental Alogorithms and Techniques
AnUndergraduateandGraduateText
AReferenceforthePractioningSoftwareEngineer
ADITYA P. MATHUR
PurdueUniversity
Copyright © 2013 Dorling Kindersley (India) Pvt. Ltd.
Licensees of Pearson Education in South Asia

No part of this eBook may be used or reproduced in any manner whatsoever without the publisher’s
prior written consent.

This eBook may or may not include all assets that were part of the print version. The publisher
reserves the right to remove any material in this eBook at any time.

ISBN 9788131794760
eISBN 9789332517653

Head Office: A-8(A), Sector 62, Knowledge Boulevard, 7th Floor, NOIDA 201 309, India
Registered Office: 11 Local Shopping Centre, Panchsheel Park, New Delhi 110 017, India
In Memory of
My Loving Mother and Father
My Friend (Late) Dr. Ranjit Singh Chauhan
This page is intentionally left blank.
Contents
PrefacetotheSecond
1.5.4 Assessingprogram
Edition xiii correctness 25
1.5.5 Constructinganoracle 26
PrefacetotheFirst
1.6. TESTMETRICS 28
Edition xix
1.6.1 Organizationalmetrics 28
Acknowledgements xxvii 1.6.2 Projectmetrics 29
1.6.3 Processmetrics 30
PART I: PRELIMINARIES 1.6.4 Productmetrics:generic 30
1.6.5 Productmetrics:OO
software 32
1. PRELIMINARIES: 1.6.6 Progressmonitoringand
SOFTWARETESTING 3 trends 33
1.6.7 Staticanddynamicmetrics 33
1.1. HUMANS,ERRORS,AND
1.6.8 Testability 33
TESTING 4
1.1.1 Errors,faults,andfailures 5 1.7. SOFTWAREANDHARDWARE
TESTING 35
1.1.2 Testautomation 7
1.1.3 Developerandtesteras 1.8. TESTINGANDVERIFICATION 37
tworoles 7 1.9. DEFECTMANAGEMENT 38
1.2. SOFTWAREQUALITY 8 1.10. TESTGENERATION
1.2.1 Qualityattributes 8 STRATEGIES 39
1.2.2 Reliability 10 1.11. STATICTESTING 42
1.3. REQUIREMENTS,BEHAVIOR, 1.11.1 Walkthroughs 42
ANDCORRECTNESS 11 1.11.2 Inspections 43
1.3.1 Inputdomain 12
1.11.3 Softwarecomplexityand
1.3.2 Specifyingprogrambehavior 13 statictesting 43
1.3.3 Validandinvalidinputs 17 1.12. MODEL-BASEDTESTINGAND
1.4. CORRECTNESSVERSUS MODELCHECKING 44
RELIABILITY 18 1.13. TYPESOFTESTING 45
1.4.1 Correctness 18 1.13.1 Classifier:C1:Sourceof
1.4.2 Reliability 19 testgeneration 47
1.4.3 Operationalprofiles 20 1.13.2 Classifier:C2:Lifecycle
phase 48
1.5. TESTINGANDDEBUGGING 21
1.13.3 Classifier:C3:
1.5.1 Preparingatestplan 21
Goal-directedtesting 50
1.5.2 Constructingtestdata 21
1.13.4 Classifier:C4:Artifact
1.5.3 Executingtheprogram 23 undertest 53
v
Contents (cid:2) vi
1.13.5 Classifier:C5:Test PART II: TEST
processmodels 54 GENERATION
1.14. THESATURATIONEFFECT 59
1.14.1 Confidenceandtrue
3. DOMAIN
reliability 60
PARTITIONING 105
1.14.2 Saturationregion 60
1.14.3 Falsesenseofconfidence 61 3.1. INTRODUCTION 106
1.14.4 Reducing(cid:2) 61 3.2. THETESTSELECTIONPROBLEM 107
1.14.5 Impactontestprocess 63
3.3. EQUIVALENCEPARTITIONING 109
1.15. PRINCIPLESOFTESTING 63 3.3.1 Faultstargeted 110
1.16. Tools 66 3.3.2 Relations 111
Summary 67 3.3.3 Equivalenceclassesfor
Exercises 68 variables 116
3.3.4 Unidimensional
partitioningversusmulti-
2. PRELIMINARIES:
dimensionalpartitioning 121
MATHEMATICAL 71
3.3.5 Asystematicprocedure 123
2.1. PREDICATESANDBOOLEAN 3.3.6 Testselection 128
EXPRESSIONS 72 3.3.7 ImpactofGUIdesign 131
2.2. CONTROLFLOWGRAPH 74 3.4. BOUNDARYVALUEANALYSIS 133
2.2.1 Basicblocks 74
3.5. CATEGORY-PARTITIONMETHOD 140
2.2.2 Flowgraphs 76
Summary 147
2.2.3 Paths 77
Exercises 147
2.2.4 Basispaths 83
2.2.5 Pathconditionsand
4. PREDICATE
domains 85
ANALYSIS 152
2.2.6 Domainandcomputation
errors 90
4.1. INTRODUCTION 153
2.2.7 Staticcodeanalysistools
andstatictesting 91 4.2. DOMAINTESTING 153
4.2.1 Domainerrors 153
2.3. EXECUTIONHISTORY 92
4.2.2 Bordershifts 154
2.4. DOMINATORSAND
4.2.3 ON-OFFpoints 155
POST-DOMINATORS 93
4.2.4 Undetectederrors 157
2.5. PROGRAMDEPENDENCE
4.2.5 Coincidentalcorrectness 159
GRAPH 95
4.2.6 Pathstobetested 159
2.5.1 Datadependence 95
2.5.2 Controldependence 96 4.3. CAUSE-EFFECTGRAPHING 160
2.5.3 Callgraph 97 4.3.1 Notationusedin
cause-effectgraphing 161
2.6. STRINGS,LANGUAGES,AND
4.3.2 Creatingcause-effectgraphs 163
REGULAREXPRESSIONS 98
4.3.3 Decisiontablefrom
2.7. TOOLS 99 cause-effectgraph 167
Summary 101
4.3.4 Heuristicstoavoid
Exercises 101 combinatorialexplosion 172
vii (cid:3) Contents
4.3.5 Testgenerationfroma 5.6. THEW-METHOD 242
decisiontable 175
5.6.1 Assumptions 242
4.4. TESTSUSINGPREDICATE 5.6.2 Maximumnumberofstates 243
SYNTAX 176 5.6.3 Computationofthe
4.4.1 Afaultmodel 176 transitioncoverset 243
4.4.2 Predicateconstraints 179 5.6.4 ConstructingZ 245
4.4.3 Predicatetestingcriteria 180 5.6.5 Derivingatestset 245
4.4.4 BOR,BRO,andBRE 5.6.6 TestingusingtheW-method 246
adequatetests 182
5.6.7 Theerrordetectionprocess 249
4.4.5 BORconstraintsfor
non-singularexpressions 191 5.7. THEPARTIALW-METHOD 250
4.4.6 Cause-effectgraphsand 5.7.1 Testingusingthe
predicatetesting 197 Wp-methodform=n 252
4.4.7 Faultpropagation 198 5.7.2 Testingusingthe
Wp-methodform>n 254
4.4.8 Predicatetestinginpractice 200
5.8. THEUIO-SEQUENCEMETHOD 257
4.5. TESTSUSINGBASISPATHS 204
5.8.1 Assumptions 257
4.6. SCENARIOSANDTESTS 205
5.8.2 UIOsequences 257
Summary 207
5.8.3 Coreandnon-corebehavior 259
Exercises 207
5.8.4 GenerationofUIOsequences 261
5.8.5 Distinguishingsignatures 273
5. TESTGENERATION 5.8.6 Testgeneration 275
FROMFINITESTATE 5.8.7 Testoptimization 277
MODELS 213 5.8.8 Faultdetection 279
5.1. SOFTWAREDESIGNAND 5.9. AUTOMATATHEORETIC
TESTING 214 VERSUSCONTROL-FLOW
BASEDTECHNIQUES 282
5.2. FINITESTATEMACHINES 216
5.9.1 n-switch-cover 285
5.2.1 Excitationusinganinput
sequence 220 5.9.2 Comparingautomata
theoreticmethods 286
5.2.2 Tabularrepresentation 221
5.2.3 PropertiesofFSM 222 5.10. TOOLS 287
Summary 288
5.3. CONFORMANCETESTING 224
Exercises 289
5.3.1 Resetinputs 226
5.3.2 Thetestingproblem 228
5.4. AFAULTMODEL 228 6. TESTGENERATION
5.4.1 MutantsofFSMs 231 FROM
COMBINATORIAL
5.4.2 Faultcoverage 233
DESIGNS 295
5.5. CHARACTERIZATIONSET 234
5.5.1 Constructionofthe 6.1. COMBINATORIALDESIGNS 296
k-equivalencepartitions 235 6.1.1 Testconfigurationandtest
5.5.2 Derivingthe set 297
characterizationset 239
6.1.2 Modelingtheinputand
5.5.3 Identificationsets 241 configurationspaces 297
Contents (cid:2) viii
6.2. ACOMBINATORIALTEST 7.2. ADEQUACYCRITERIABASED
DESIGNPROCESS 303 ONCONTROLFLOW 364
6.3. FAULTMODEL 305 7.2.1 Statementandblock
coverage 364
6.4. LATINSQUARES 308
7.2.2 Conditionsanddecisions 367
6.5. MUTUALLYORTHOGONAL
7.2.3 Decisioncoverage 369
LATINSQUARES 310
7.2.4 Conditioncoverage 371
6.6. PAIRWISEDESIGN:BINARY
7.2.5 Condition/decisioncoverage 373
FACTORS 312
7.2.6 Multipleconditioncoverage 375
6.7. PAIRWISEDESIGN:
MULTI-VALUEDFACTORS 318 7.2.7 Linearcodesequenceand
jump(LCSAJ)coverage 378
6.8. ORTHOGONALARRAYS 326
7.2.8 Modified
6.9. COVERINGANDMIXED-LEVEL condition/decisioncoverage 382
COVERINGARRAYS 330
7.2.9 MC/DCadequatetests
6.10. ARRAYSOFSTRENGTH>2 332 forcompoundconditions 383
6.11. GENERATINGCOVERING 7.2.10 DefinitionofMC/DC
ARRAYS 333 coverage 387
7.2.11 MinimalMC/DCtests 396
6.12. TOOLS 342
7.2.12 Errordetectionand
Summary 342
MC/DCadequacy 396
Exercises 343
7.2.13 Short-circuitevaluation
andinfeasibility 398
PART III: TEST
7.2.14 Basispathcoverage 400
ADEQUACY
7.2.15 Tracingtestcasesto
ASSESSMENTAND requirements 400
ENHANCEMENT
7.3. CONCEPTSFROMDATAFLOW 402
7.3.1 Definitionsanduses 404
7. TESTADEQUACY 7.3.2 C-useandp-use 405
ASSESSMENTUSING 7.3.3 Globalandlocal
CONTROLFLOW AND definitionsanduses 405
DATAFLOW 349 7.3.4 Dataflowgraph 406
7.3.5 Def-clearpaths 408
7.1. TESTADEQUACY:BASICS 350
7.3.6 Def-usepairs 409
7.1.1 Whatistestadequacy? 350
7.3.7 Def-usechains 410
7.1.2 Measurementoftest
adequacy 351 7.3.8 Alittleoptimization 411
7.1.3 Testenhancement 7.3.9 Datacontextsand
usingmeasurements ordereddatacontexts 412
ofadequacy 353
7.4. ADEQUACYCRITERIABASED
7.1.4 Infeasibilityandtest ONDATAFLOW 415
adequacy 358
7.4.1 c-usecoverage 416
7.1.5 Errordetectionandtest
7.4.2 p-usecoverage 417
enhancement 360
7.4.3 All-usescoverage 419
7.1.6 Singleandmultiple
executions 363 7.4.4 k-drchaincoverage 419
ix (cid:3) Contents
7.4.5 Usingthek-drchain 8.5. DESIGNOFMUTATION
coverage 421 OPERATORS 470
7.4.6 Infeasiblec-andp-uses 422 8.5.1 Goodnesscriteriafor
mutationoperators 470
7.4.7 Contextcoverage 423
8.5.2 Guidelines 471
7.5. CONTROLFLOWVERSUS
DATAFLOW 425 8.6. FOUNDINGPRINCIPLESOF
MUTATIONTESTING 472
7.6. THE“SUBSUMES”RELATION 427
8.6.1 Thecompetent
7.7. STRUCTURALAND programmerhypothesis 472
FUNCTIONALTESTING 429 8.6.2 Thecouplingeffect 473
7.8. SCALABILITYOFCOVERAGE 8.7. EQUIVALENTMUTANTS 474
MEASUREMENT 431
8.8. FAULTDETECTIONUSING
7.9. TOOLS 433 MUTATION 474
Summary 434
8.9. TYPESOFMUTANTS 477
Exercises 435
8.10. MUTATIONOPERATORSFORC 478
8.10.1. Whatisnotmutated? 479
8. TESTADEQUACY 8.10.2 Linearization 480
ASSESSMENTUSING 8.10.3 Executionsequence 481
PROGRAM 8.10.4 Effectofanexecution
MUTATION 442 sequence 484
8.10.5 Globalandlocal
8.1. INTRODUCTION 443 identifiersets 485
8.2. MUTATIONANDMUTANTS 444 8.10.6 Globalandlocal
8.2.1 First-orderandhigher referencesets 486
ordermutants 445 8.10.7 Mutatingprogram
8.2.2 Syntaxandsemanticsof constants 489
mutants 446 8.10.8 Mutatingoperators 490
8.2.3 Strongandweakmutations 449 8.10.9 Mutatingstatements 497
8.2.4. Whymutate? 450 8.10.10 Mutatingprogram
variables 511
8.3. TESTASSESSMENTUSING
MUTATION 452 8.11. MUTATIONOPERATORS
8.3.1 Aprocedurefortest FORJAVA 516
adequacyassessment 452 8.11.1 Traditionalmutation
operators 517
8.3.2 Alternateprocedures
fortestadequacy 8.11.2 Inheritence 518
assessment 462 8.11.3 Polymorphismand
8.3.3 “Distinguished”versus dynamicbinding 521
“killed”mutants 463 8.11.4 Methodoverloading 523
8.3.4 Conditionsfor 8.11.5 Javaspecificmutation
distinguishingamutant 463 operators 524
8.4. MUTATIONOPERATORS 465 8.12. COMPARISONOFMUTATION
8.4.1 Operatortypes 467 OPERATORS 526
8.4.2 Languagedependenceof 8.13. MUTATIONTESTINGWITHIN
mutationoperators 468 BUDGET 528