Model-based software testing and analysis with C♯

367 Pages·2008·3.036 MB·English

by  Jonathan Jacky, et al

This page intentionally left blank
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
Model-Based Software Testing and
Analysis with C#
Thisbookteachesmodel-basedanalysisandmodel-basedtesting,importantnewways
towriteandanalyzesoftwarespecificationsanddesigns,generatetestcases,andcheck
the results of test runs. These methods increase the automation in each of these steps,
makingthemmoretimely,morethorough,andmoreeffective.
Using a familiar programming language, testers and analysts will learn to write
modelsthatdescribehowaprogramissupposedtobehave.Theauthorsworkthrough
severalrealisticcasestudiesindepthanddetail,usingatoolkitbuiltontheC#language
andthe.NETframework.Readerscanalsoapplythemethodsinanalyzingandtesting
systemsinmanyotherlanguagesandframeworks.
Intended for professional software developers, including testers, and for university
students,thisbookissuitableforcoursesonsoftwareengineering,testing,specification,
orapplicationsofformalmethods.
JonathanJackyisaResearchScientistattheUniversityofWashingtoninSeattle.He
isexperiencedinembeddedcontrolsystems,safety-criticalsystems,signalprocessing,
andscientificcomputing.HehastaughtattheEvergreenStateCollegeandhasbeena
VisitingResearcheratMicrosoftResearch.HeistheauthorofTheWayofZ:Practical
ProgrammingwithFormalMethods.
Margus Veanes is a Researcher in the Foundations of Software Engineering (FSE)
group at Microsoft Research. His research interests include model-based software de-
velopment,validation,andtesting.
Colin Campbell has worked on model-based testing and analysis techniques for a
number of years in industry, for companies including Microsoft Research. He is a
PrincipaloftheconsultingfirmModeledComputationLLCinSeattle(www.modeled-
computation.com).Hiscurrentinterestsincludedesignanalysis,themodelingofreactive
anddistributedsystems,andtheintegrationofcomponentsinlargesystems.
Wolfram Schulte is a Research Area Manager at Microsoft Research, managing the
FSEgroup,theProgrammingLanguagesandMethods(PLM)group,andtheSoftware
DesignandImplementation(SDI)group.
i
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
ii
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
Model-Based Software
Testing and Analysis
with C#
Jonathan Jacky
UniversityofWashington,Seattle
Margus Veanes
MicrosoftResearch,Redmond,Washington
Colin Campbell
ModeledComputationLLC,Seattle,Washington
Wolfram Schulte
MicrosoftResearch,Redmond,Washington
iii
CAMBRIDGEUNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB28RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521886550
© Jonathan Jacky, Margus Veanes, Colin Campbell, and Wolfram Schulte 2008
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2007
ISBN-13 978-0-511-36784-7 eBook (NetLibrary)
ISBN-10 0-511-36784-8 eBook (NetLibrary)
ISBN-13 978-0-521-88655-0 hardback
ISBN-10 0-521-88655-4 hardback
ISBN-13 978-0-521-68761-4 paperback
ISBN-10 0-521-68761-6 paperback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
Contents
Preface xi
Acknowledgments xv
I Overview
1 Describe, Analyze, Test 3
1.1 Modelprograms 4
1.2 Model-basedanalysis 5
1.3 Model-basedtesting 7
1.4 Modelprogramsinthesoftwareprocess 8
1.5 Syllabus 11
2 Why We Need Model-Based Testing 13
2.1 Clientandserver 13
2.2 Protocol 14
2.3 Sockets 15
2.4 Libraries 15
2.5 Applications 20
2.6 Unittesting 23
v
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
vi Contents
2.7 Somesimplescenarios 25
2.8 Amorecomplexscenario 27
2.9 Failuresinthefield 28
2.10 Failuresexplained 29
2.11 Lessonslearned 29
2.12 Model-basedtestingrevealsthedefect 30
2.13 Exercises 31
3 Why We Need Model-Based Analysis 32
3.1 Reactivesystem 32
3.2 Implementation 34
3.3 Unittesting 41
3.4 Failuresinsimulation 44
3.5 Designdefects 46
3.6 Reviewsandinspections,staticanalysis 47
3.7 Model-basedanalysisrevealsthedesignerrors 47
3.8 Exercises 52
4 Further Reading 53
II Systems with Finite Models
5 Model Programs 57
5.1 States,actions,andbehavior 57
5.2 Casestudy:userinterface 59
5.3 Preliminaryanalysis 61
5.4 Codingthemodelprogram 64
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
Contents vii
5.5 Simulation 70
5.6 Casestudy:client/server 72
5.7 Casestudy:reactiveprogram 82
5.8 Otherlanguagesandtools 92
5.9 Exercises 93
6 Exploring and Analyzing Finite Model
Programs 94
6.1 Finitestatemachines 94
6.2 Exploration 99
6.3 Analysis 106
6.4 Exercise 114
7 Structuring Model Programs with Features and
Composition 115
7.1 Scenariocontrol 115
7.2 Features 117
7.3 Composition 121
7.4 Choosingamongoptionsforscenariocontrol 129
7.5 Compositionforanalysis 131
7.6 Exercises 136
8 Testing Closed Systems 137
8.1 Offlinetestgeneration 137
8.2 Tracesandterms 139
8.3 Testharness 142
8.4 Testexecution 146
P1:KNP
cuny1215-book CUNY1215-Jacky 9780521886550 October2,2007 16:18
viii Contents
8.5 Limitationsofofflinetesting 147
8.6 Exercises 148
9 Further Reading 150
III Systems with Complex State
10 Modeling Systems with Structured State 155
10.1 “Infinite”modelprograms 155
10.2 Typesformodelprograms 157
10.3 Compoundvalues 157
10.4 Casestudy:revisioncontrolsystem 169
10.5 Exercises 181
11 Analyzing Systems with Complex State 183
11.1 Explorablemodelprograms 183
11.2 Pruningtechniques 186
11.3 Sampling 190
11.4 Exercises 190
12 Testing Systems with Complex State 191
12.1 On-the-flytesting 192
12.2 Implementation,modelandstepper 194
12.3 Strategies 199
12.4 Coverage-directedstrategies 203
12.5 Advancedon-the-flysettings 210
12.6 Exercises 218
13 Further Reading 219