Case Study Research in Software Engineering: Guidelines and Examples

241 Pages·2012·4.027 MB·English

by  Per Runeson, Martin Host, Austen Rainer, Bjorn Regnell

CASE STUDY RESEARCH
IN SOFTWARE
ENGINEERING
www.it-ebooks.info
CASE STUDY RESEARCH
IN SOFTWARE
ENGINEERING
Guidelines and Examples
PER RUNESON
LundUniversity,Sweden
MARTIN HO¨ST
LundUniversity,Sweden
AUSTEN RAINER
UniversityofHertfordshire,UK
BJO¨RN REGNELL
LundUniversity,Sweden
www.it-ebooks.info
Copyright©2012byJohnWiley&Sons,Inc.Allrightsreserved
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey
PublishedsimultaneouslyinCanada
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformor
byanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptas
permittedunderSection107or108ofthe1976UnitedStatesCopyrightAct,withouteithertheprior
writtenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfeeto
theCopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,[phone],fax
[phone],oronthewebatwww.copyright.com.RequeststothePublisherforpermissionshould
beaddressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ
07030,[phone],fax[phone],oronlineathttp://www.wiley.com/go/permission.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin
preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyor
completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesof
merchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysales
representativesorwrittensalesmaterials.Theadviceandstrategiescontainedhereinmaynotbesuitable
foryoursituation.Youshouldconsultwithaprofessionalwhereappropriate.Neitherthepublishernor
authorshallbeliableforanylossofprofitoranyothercommercialdamages,includingbutnotlimitedto
special,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontactour
CustomerCareDepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat(317)
572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmay
notbeavailableinelectronicformats.FormoreinformationaboutWileyproducts,visitourwebsiteat
www.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Casestudyresearchinsoftwareengineering:guidelinesandexamples/
PerRuneson... [etal.].–1sted.
p.cm.
Includesbibliographicalreferencesandindex.
ISBN978-1-118-10435-4(hardback)
1. Computersoftware–Development–Casestudies. I. PerRuneson.
QA76.76.D47C372012
005.1–dc23
2011031429
PrintedintheUnitedStatesofAmerica
ISBN:9781118104354
10 9 8 7 6 5 4 3 2 1
www.it-ebooks.info
CONTENTS
FOREWORD xiii
PREFACE xv
ACKNOWLEDGMENTS xvii
PART I CASE STUDY METHODOLOGY
1 INTRODUCTION 3
1.1 WhatisaCaseStudy? 3
1.2 ABriefHistoryofCaseStudiesinSoftwareEngineering 5
1.3 WhyaBookonCaseStudiesofSoftwareEngineering? 6
1.4 Conclusion 9
2 BACKGROUNDANDDEFINITIONOFCONCEPTS 11
2.1 Introduction 11
2.2 ResearchStrategies 11
2.3 CharacteristicsofResearchStrategies 13
2.3.1 Purpose 13
2.3.2 ControlandData 14
2.3.3 Triangulation 15
www.it-ebooks.info
vi CONTENTS
2.3.4 Replication 16
2.3.5 InductiveandDeductiveEnquiries 16
2.4 WhatMakesaGoodCaseStudy? 17
2.5 WhenistheCaseStudyStrategyFeasible? 19
2.6 CaseStudyResearchProcess 20
2.7 Conclusion 21
3 DESIGNOFTHECASESTUDY 23
3.1 Introduction 23
3.2 ElementsoftheCaseStudyDesign 24
3.2.1 RationalefortheStudy 24
3.2.2 ObjectiveoftheStudy 24
3.2.3 CasesandUnitsofAnalyses 26
3.2.4 TheoreticalFramework 29
3.2.5 ResearchQuestions 30
3.2.6 PropositionsandHypotheses 31
3.2.7 Concepts 32
3.2.8 MethodsofDataCollection 32
3.2.9 MethodsofDataAnalysis 33
3.2.10 CaseSelection 33
3.2.11 SelectionofData 35
3.2.12 DataDefinitionandDataStorage 36
3.2.13 QualityControlandAssurance 36
3.2.14 MaintainingtheCaseStudyProtocol 37
3.2.15 ReportingandDisseminatingtheCaseStudy 38
3.3 Legal,Ethical,andProfessionalIssues 40
3.4 Conclusion 45
4 DATACOLLECTION 47
4.1 Introduction 47
4.2 DifferentTypesofDataSource 47
4.2.1 ClassificationofDataSources 47
4.2.2 DataSourceSelection 49
4.3 Interviews 50
4.3.1 PlanningInterviews 50
4.3.2 TheInterviewSession 52
4.3.3 PostinterviewActivities 53
4.4 Focusgroups 54
www.it-ebooks.info
CONTENTS vii
4.5 Observations 56
4.6 ArchivalData 57
4.7 Metrics 58
4.8 Conclusion 60
5 DATAANALYSISANDINTERPRETATION 61
5.1 Introduction 61
5.2 AnalysisofDatainFlexibleResearch 62
5.2.1 Introduction 62
5.2.2 LevelofFormalism 64
5.2.3 RelationtoHypotheses 65
5.3 ProcessforQualitativeDataAnalysis 65
5.3.1 Introduction 65
5.3.2 StepsintheAnalysis 66
5.3.3 Techniques 68
5.3.4 Toolsupport 70
5.4 Validity 71
5.4.1 ConstructValidity 71
5.4.2 InternalValidity 71
5.4.3 ExternalValidity 71
5.4.4 Reliability 72
5.5 ImprovingValidity 72
5.6 QuantitativeDataAnalysis 74
5.7 Conclusion 76
6 REPORTINGANDDISSEMINATION 77
6.1 Introduction 77
6.2 WhyReportandDisseminate 78
6.3 TheAudiencefortheReport 79
6.4 AspectsoftheCaseStudytoReportandDisseminate 80
6.5 WhentoReportandDisseminate 81
6.6 GuidelinesonReporting 82
6.6.1 TheGenericContentofanAcademicReport 82
6.6.2 ReportingRecommendationsfromEvaluativeCase
Studies 84
6.6.3 ReportingtoStakeholders,IncludingSponsor(s) 85
6.6.4 ReportingtheContextoftheCaseStudy 87
www.it-ebooks.info
viii CONTENTS
6.6.5 ReportingtoStudents 89
6.6.6 AdHocandImpromptuReporting 90
6.7 FormatsandStructuresforaReport 91
6.8 WheretoReport 94
6.9 EthicsandConfidentiality 94
6.10 Conclusion 95
7 SCALINGUPCASESTUDYRESEARCHTOREAL-WORLD
SOFTWAREPRACTICE 97
7.1 Introduction 97
7.2 TheAimsofScalingupCaseStudies 98
7.3 DimensionsofScale 99
7.4 LongitudinalCaseStudies 100
7.5 MultipleCaseStudies 102
7.5.1 MultipleCasesandReplications 102
7.5.2 SelectingtheCases 104
7.6 MultiresearcherCaseStudies 105
7.7 Conclusion 107
8 USINGCASESTUDYRESEARCH 109
8.1 Introduction 109
8.2 ReadingandReviewingCaseStudies 109
8.2.1 DevelopmentofChecklists 110
8.2.2 ChecklistsforConductingCaseStudyResearch 111
8.2.3 ChecklistsforReadingandReviewingCaseStudies 111
8.2.4 DevelopmentofPractice 111
8.3 IdentifyingandSynthesizingUseCaseResearch 111
8.3.1 IdentifyingPrimaryStudies 112
8.3.2 SynthesisofEvidencefromMultipleCaseStudies 113
8.3.3 CurrentStateofSynthesis 117
8.4 TheEconomicsofCaseStudyResearch 118
8.4.1 CostsandBenefitsofEvaluationTechniques 119
8.4.2 EvaluationoftheDESMETMethodology 119
8.4.3 FrameworksforOrganizingMethodsofEvaluation 119
8.5 SpecializingCaseStudyResearchforSoftwareEngineering 121
8.5.1 TheLongitudinalChronologicalCaseStudyResearch
Strategy 122
8.5.2 ControlledCaseStudies 123
www.it-ebooks.info
CONTENTS ix
8.6 CaseStudiesandSoftwareProcessImprovement 123
8.7 Conclusion 125
PART II EXAMPLES OF CASE STUDIES
9 INTRODUCTIONTOCASESTUDYEXAMPLES 129
9.1 Introduction 129
10 CASESTUDYOFEXTREMEPROGRAMMINGINA
STAGE–GATECONTEXT 133
10.1 Introduction 133
10.1.1 MethodologicalStatus 133
10.2 CaseStudyDesign 134
10.2.1 Rationale 134
10.2.2 Objectives 134
10.2.3 CasesandUnitsofAnalysis 135
10.2.4 TheoreticalFrameofReference 136
10.2.5 ResearchQuestions 136
10.3 Planning 136
10.3.1 MethodsofDataCollection 136
10.3.2 SelectionofData 137
10.3.3 CaseSelectionStrategy 137
10.3.4 CaseStudyProtocol 137
10.3.5 EthicalConsiderations 137
10.4 DataCollection 139
10.5 DataAnalysis 139
10.5.1 ThreatstoValidity 144
10.6 Reporting 144
10.6.1 Academics 144
10.6.2 Practitioners 144
10.7 LessonsLearned 146
11 TWOLONGITUDINALCASESTUDIESOFSOFTWARE
PROJECTMANAGEMENT 149
11.1 Introduction 149
11.2 BackgroundtotheResearchProject 149
11.3 CaseStudyDesignandPlanning 150
www.it-ebooks.info
x CONTENTS
11.3.1 Rationale 150
11.3.2 Objective 150
11.3.3 DefinitionoftheCase 150
11.3.4 UnitsofAnalyses 151
11.3.5 TheoreticalFrameofReferenceandResearch
Questions 151
11.3.6 CaseSelection 151
11.3.7 ReplicationStrategy 152
11.3.8 CaseStudyProtocol 152
11.3.9 QualityAssurance,Validity,andReliability 152
11.3.10 Legal,Ethical,andProfessionalConsiderations 153
11.4 DataCollection 154
11.4.1 SourcesofData 154
11.5 DataAnalysis 157
11.6 Reporting 159
11.6.1 InternalReportingofResults 160
11.6.2 DisseminationofArtifacts 160
11.7 LessonsLearned 160
12 ANITERATIVECASESTUDYOFQUALITYMONITORING 163
12.1 Introduction 163
12.2 CaseStudyDesign 164
12.2.1 Objectives 164
12.2.2 CasesandUnitsofAnalysis 165
12.2.3 TheoreticalFrameofReference 165
12.2.4 ResearchQuestions 165
12.3 Planning 165
12.3.1 MethodsofDataCollection 165
12.3.2 CaseSelectionStrategy 167
12.3.3 CaseStudyProtocol 167
12.3.4 EthicalConsiderations 167
12.3.5 DataCollection 168
12.3.6 ExploratoryStudy 168
12.3.7 ConfirmatoryStudy 168
12.3.8 ExplanatoryStudy 168
12.4 DataAnalysis 169
12.5 Reporting 169
12.6 LessonsLearned 169
www.it-ebooks.info
CONTENTS xi
13 ACASESTUDYOFTHEEVALUATIONOFREQUIREMENTS
MANAGEMENTTOOLS 171
13.1 Introduction 171
13.2 DesignoftheCaseStudy 172
13.2.1 Rationale 172
13.2.2 Objective 172
13.2.3 TheCaseandItsContext 173
13.2.4 TheUnitsofAnalyses 174
13.2.5 TheoreticalFramework 175
13.2.6 ResearchQuestions 175
13.2.7 Propositions,Concepts,andMeasures 175
13.2.8 CaseStudyProtocol 175
13.2.9 MethodsofDataCollection 176
13.2.10 MethodsofDataAnalysis 176
13.2.11 CaseSelectionStrategy 177
13.2.12 DataSelectionStrategy 177
13.2.13 ReplicationStrategy 177
13.2.14 QualityAssurance,Validity,andReliability 177
13.3 DataCollection 178
13.4 DataAnalysis 179
13.5 ReportingandDissemination 180
13.6 LessonsLearned 181
14 ALARGE-SCALECASESTUDYOFREQUIREMENTS
ANDVERIFICATIONALIGNMENT 183
14.1 Introduction 183
14.2 CaseStudyDesign 184
14.2.1 Rationale 184
14.2.2 Objectives 184
14.2.3 CasesandUnitsofAnalysis 185
14.2.4 TheoreticalFrameofReference 186
14.2.5 ResearchQuestions 187
14.3 Planning 188
14.3.1 MethodsofDataCollection 189
14.3.2 CaseSelectionStrategy 190
14.3.3 SelectionofData 191
14.3.4 CaseStudyProtocol 191
14.3.5 EthicalConsiderations 192
www.it-ebooks.info