ASSESSING REUSABILITY IN AUTOMATED ACCEPTANCE TESTS Mohsin Irshad

206 Pages·2017·9.26 MB·English

A
S
S
ABSTRACT E
S
S
I ASSESSING REUSABILITY IN AUTOMATED
Context: Automated acceptance tests have Results: The results from the SLR shows N
G
become a core practice of agile software that text-based requirements reuse app-
ACCEPTANCE TESTS
R
development (e.g. Extreme Programming). roaches are commonly used in the industry.
E
These tests are closely tied to requirements Structuring these requirements and iden- U
S
specifications and provide a mechanism for tifying the reusable requirements by mat-
A
continuous validation of software require- ching are the two commonly used methods B
I
ments. Software reuse has evolved with the for enabling requirements reuse. The results L
I
introduction of each new reusable artefact from industrial experience report indica- T
Y
(e.g., reuse of code, reuse of frameworks, tes that defect reports can be formulated
I
N
tools etc.). In this study, we have investiga- in template and defect triage meetings can

A
ted the reusability of automated acceptance be used to identify important test cases.
U
tests keeping in view their close association The results from these two studies were T
with requirements. considered when identifying approaches to O Mohsin Irshad
M
measure reuse potential of BDD test cases.
A
Objective: This study attempts to address The two proposed approaches; Normalised T
the following aspects: (i) what important Compression Distance (NCD) and Similari- E
D
reuse characteristics should be considered ty Ratio, for measuring reuse potential were
A
when measuring reusability of automated evaluated in the industry. The evaluation in- C
acceptance tests? (ii) how reusability can be dicated that similarity ratio performed bet- C
E
measured in automated acceptance tests? ter than the NCD approach, however, the P
And (iii) how cost avoided through reuse of results from both approaches were compa- T
A
automated acceptance tests can be calcula- rable with the results gathered with the help N
ted? of expert analysis. The cost related aspects C
E
of reusable acceptance tests were addressed

T
Method: We have used a combination of re- and evaluated using a method that calculates E
S
search methods to answer different aspects the cost-avoidance through reuse.
T
of our study. We started by identifying reu- S
sability related characteristics of software Conclusions: The evidence from this study
requirements, with help of systematic lite- shows that the automated acceptance tests
rature review (SLR). Later, we identified the are reusable, similar to text-based software
reusability related characteristics of defect requirements and their reuse potential can
reports. After identifying the characteristics be calculated. The industrial evaluation of
from the previous two studies, we used these the three studies shows that the overall re-
characteristics on two case studies conduc- sults are applicable to the industry. However,
ted on Behaviour driven development test further work is required to evaluate the reu-
cases (i.e., acceptance tests of textual natu- se potential of automated acceptance tests
re). We proposed two approaches that can in different contexts.
M
identify reuse potential of automated accep- o
h
tance tests and evaluated these approaches s
in
in the industry. Later, to calculate the cost
Ir
avoided through reuse, we proposed and s
h Blekinge Institute of Technology
evaluated a method that is applicable to any a
d
reusable artifact. Licentiate Dissertation Series No. 2018:01
2
0
ISSN: 1650-2140 1
2018:01 ISBN: 978-91-7295-347-5 8:01 Department of Software Engineering
Assessing Reusability In Automated
Acceptance Tests
Mohsin Irshad

Blekinge Institute of Technology Doctoral Dissertation Series
No 2015:09
Blekinge Institute of Technology Licentiate Dissertation Series
No 2018:01
Social Sustainability within
the Framework for Strategic
Sustainable Development
Assessing Reusability In Automated
Acceptance Tests
Merlina Missimer
Mohsin Irshad
Doctoral Dissertation in
Strategic Sustainable Development
Licentiate Dissertation in
Software Engineering
Department of Software Engineering
Department of Strategic Sustainable Development
Blekinge Institute of Technology
Blekinge Institute of Technology
SWEDEN
SWEDEN
2018 Mohsin Irshad
Department of Software Engineering
Publisher: Blekinge Institute of Technology
SE-371 79 Karlskrona, Sweden
Printed by Exakta Group, Sweden, 2018
ISBN: 978-91-7295-347-5
ISSN:1650-2140
urn:nbn:se:bth-15633
A dedication to my wife, Naila.

Abstract
Context: Automated acceptance tests have become a core practice of agile
softwaredevelopment(e.g.ExtremeProgramming).Thesetestsareclosely
tied to requirements specifications and provide a mechanism for continu-
ous validation of software requirements. Software reuse has evolved with
theintroductionofeachnewreusableartefact(e.g.,reuseofcode,reuseof
frameworks, tools etc.). In this study, we have investigated the reusability
ofautomatedacceptancetestskeepinginviewtheircloseassociationwith
requirements.
Objective: This study attempts to address the following aspects: (i) what
important reuse characteristics should be considered when measuring
reusabilityofautomatedacceptancetests?(ii)Howreusabilitycanbemea-
sured in automated acceptance tests? And (iii) how cost avoided through
reuseofautomatedacceptancetestscanbecalculated?
Method: We have used a combination of research methods to answer dif-
ferent aspects of our study. We started by identifying reusability related
characteristicsofsoftwarerequirements,withhelpofsystematicliterature
review (SLR). Later, we identified the reusability related characteristics of
defect reports. After identifying the characteristics from the previous two
studies, we used these characteristics on two case studies conducted on
Behaviour driven development test cases (i.e., acceptance tests of textual
nature). We proposed two approaches that can identify reuse potential of
automated acceptance tests and evaluated these approaches in the indus-
try. Later, to calculate the cost avoided through reuse, we proposed and
evaluatedamethodthatisapplicabletoanyreusableartifact.
Results: The results from the SLR shows that text-based requirements
reuse approaches are commonly used in the industry. Structuring these
requirements and identifying the reusable requirements by matching are
thetwocommonlyusedmethodsforenablingrequirementsreuse.There-
sultsfromindustrialexperiencereportindicatesthatdefectreportscanbe
formulated in template and defect triage meetings can be used to identify
important test cases. The results from these two studies were considered
whenidentifyingapproachestomeasurereusepotentialofBDDtestcases.
The two proposed approaches; Normalised Compression Distance (NCD)
and Similarity Ratio, for measuring reuse potential were evaluated in the
industry. The evaluation indicated that similarity ratio performed better
than the NCD approach, however, the results from both approaches were
comparable with the results gathered with the help of expert analysis.
vii
The cost related aspects of reusable acceptance tests were addressed and
evaluatedusingamethodthatcalculatesthecost-avoidancethroughreuse.
Conclusions: The evidence from this study shows that the automated ac-
ceptance tests are reusable, similar to text-based software requirements
and their reuse potential can be calculated. The industrial evaluation of
thethreestudiesshowsthattheoverallresultsareapplicabletotheindus-
try. However, further work is required to evaluate the reuse potential of
automatedacceptancetestsindifferentcontexts.
viii
Acknowledgments
IwouldliketoexpressmygratitudetomysupervisorsProf.KaiPetersen
andSimonPouldingfortheirsupport,guidanceandfruitfulcollaboration
thatweremandatoryforthesuccessofthisthesis.
Karlskrona,September1,2017
ix