Explaining Testing to Anybody - James Bach

17 Pages·2001·0.27 MB·English

Explaining Testing to
Anybody
James Bach, Satisfice, Inc.
[e-mail]
www.satisfice.com
Uhoh…
Howdoyourespondwhensomeone
youworkwithdoesn’tunderstand
howtestingworks?
1
“In the testing process, test plans are made,
test cases specified, tests databases and
files set up, test output created, and so on.
It is important to regard this testware not as
a disposable product, but as one that has to
be filed, managed, and handed over for
are you listening?”
reuse upon—
head… heavy…
strength… fading…
zzzzzzzz…
Basics of Explaining
(cid:131) Be quick, then stop and check in.
(cid:131) Be humble. Exceptions and alternatives abound.
(cid:131) You can be wrong, as long as you’re thoughtful.
(cid:131) Be real. Own the explanation.
(cid:131) Be respectful. Honor objections and questions.
(cid:131) Be patient. Often, experience must come first.
(cid:131) Be relevant. How does it relate to them? What’s
the bottom-line? What do you want them to do?
2
Basics of Explaining
(cid:131) Be prepared
− Quickpoint (afewsentences at most.)
− Simile &Metaphor(relatetosomethingfamiliar)
− Vocabularycheck (are youusingwords thesame way?)
− 5-minutewhiteboardtalk(diagrams arepowerful)
− Concreteexamples or demonstration(in-housedata)
− Anticipatecommonquestions andobjections
− References &supportingmaterial (haveit handy)
Explanations
(cid:131) Confusions
− “What is testing?”
− “What’s sohardabout testing?”
− “Howdo youknow when you’redone?”
− “Whydidn’t youfindthat bug?”
(cid:131) “Let’s {do my crazy plan},okay?”
− “…changetheproduct at thelast minute…”
− “…specifythosetests inadvance[andautomatethem]…
3
“What is Testing?”
(cid:131) Quick points (choose one):
− Testingis organizedskepticism.It’s thebeliefthat things
maynot beas theyseem; that things couldbedifferent.
− Testingis comparingthe ambiguous totheinvisible,soas
toavoidtheunthinkablehappeningtotheunknown.
− Testingbrings vital informationtolight,sowecansee
where weareandmakebetterdecisions.
− Testingis asupport functionthat helps developers look
goodbyfindingtheirmistakes before anyoneelse does.
“What is Testing?”
(cid:131) Similes and Metaphors
− Policepatrol (scanningfortroubleinacomplexworld)
− Secret Service (makingriskyactivities safer)
− Radar (threat detectionwhileavoidingfalsealarms)
− Copyediting (savingauthors from embarrassment)
− Soccer/hockeydefense(wedefendthegoal)
− Science(conjectureandrefutation)
4
“What is testing?”
Project
Environment
Test
Techniques
Quality Product
Criteria Elements
Perceived
Quality
“What is testing?”
Logistics
Problem
Project
Environment
Coverage
Oracle Problem
Test
Problem
Techniques
Quality Product
Criteria Elements
Reporting
Perceived
Problem Quality
5
“What is testing?”
(cid:131) Supporting Material
− http://www.kaner.com/imposs.htm
Explainstheimpossibilityofcompletetesting.
− http://www.satisfice.com/articles/test_automation_snake_oil.pdf
− http://www.kaner.com/lawst1.htm
Explainthechallengesofregressiontestautomation.
− http://www.satisfice.com/articles/good_enough_testing.pdf
Explainstestinginthecontextofhowmuchisenough.
− http://www.rstcorp.com/marick/classic/mistakes.html
Agoldmineofmarvelouslittleexplanations.
Explanation Hint
Try highlighting one key dynamic,
instead of explaining the whole issue.
6
“What’s so hard about
testing?”
Data
smoke
testing
reliabilitytesting
Functions
function
testing
risktesting
Explanation Hint
Try beginning by explaining things
in terms of the mission, then get
more concrete from there.
7
“How do you know when
you’re done?”
1. When management has enough information to
make an informed decision about the benefits
and risks associated with the product.
AND
2. Management decides to release or cancel the
project.
Bothmustbetrueor
testingisn’tdone.
Explanation Hint
Use a diagram that helps you examine
different scenarios and issues as
part of the dialog.
8
“How do you know when
you’re done?”
Test Cycle Convergence
GoodEnoughQuality
Testsfailed:knownbad
y CycleStart CycleEnd
t
ali } Testsnotrun:
u Unknown
Q
Build
UntestableQuality Testspassed:knowngood
Time
“How do you know when
you’re done?”
Partial Convergence:
duetoincompletetestsuite
y
t
i
l
a
u
Q
Time
9
“How do you know when
you’re done?”
Slow Convergence:
duetoinefficienttestingorlowtestability
y
t
i
l
a
u
Q
Time
“How do you know when
you’re done?”
No Convergence:
duetointerruptedtestcycle
y
t
i
l
a
u
Q
Time
10