Scenarios for Trusted Autonomous Systems

17 Pages·2016·1.44 MB·English

UNCLASSIFIED
Scenarios for Trusted Autonomous
Systems
Brandon Pincombe
Head Land Organisational and Management Science,
Land Capability Analysis Branch
Joint and Operational Analysis Division
Defence Science and Technology Group
1
UNCLASSIFIED
Norway: April and June 1940
 9 April 1940: Germany invaded Norway
– Rapidly consolidated hold on south.
– British TF landed in central Norway from
12 April; force out by 2 May.
– Franco-Polish-British TF landed in north
from 14 April; supporting Norwegian 6th
Division; made gains, but …
– 10 May Germany invaded France, Belgium
and Netherlands; allied priorities changed;
– Recapture Narvik 28 May but Norwegian
royal family and cabinet evacuated on 7
June & allied forces followed on 8 June;
– HMS Glorious was on her fourth mission
to northern Norway;
– German SAG had left Kiel 4 June to strike
allied positions in northern Norway.

2
UNCLASSIFIED
8 June 1940: Norwegian Sea
VE MTA V OW2 O-U 2BC 308 15 030 154GQOX 11 BT 1615 IMI
3
UNCLASSIFIED
Outline
 The future is uncertain, so use preparatory scenarios to assess the
adaptability of options.
– This can be applied to autonomous systems.
 Have used experts to identify critical variables and build scenarios within
the dimensional space created by these.
– Experientially based – captures variables that have been critical.
– Used to test adaptability; some ways of inculcating adaptability may
• Extend outside the ranges considered on variables; or (rarely?)
• Extend to other variables
 Open question: autonomous generation of scenarios?
4
UNCLASSIFIED
The Future is Uncertain
… the major cultural barrier to scenario implementation stems from the way we define
managerial competence. Good managers, we say, know where they are, where
they’re going, and how they’ll get there. We equate managerial competence with
“knowing,” and assume that decisions depend on facts about the present and about
the future. Of course, the reality is that we have no facts about the future.
-- Ian Wilson (2000) “From Scenario Thinking to Strategic Action” Technological Forecasting and Social Change, 65(1) 23-29.

Some managers think that it just takes the right experts to be able to predict the future … but:
– “The phonograph … is not of any commercial value”
Thomas Edison (inventor of the phonograph) c. 1880
May in Tanunda
– “Heavier than air flying machines are impossible.”
Hottest Ever 28.9°
Lord Kelvin (Royal Society president) c.1895
– “I think there is a world market for about five computers.” Average Max. 17.5°
Thomas J. Watson (Chairman of IBM) 1943
Average Min. 6.9°
– “There is no reason for any individual to have a computer in their home.”
Ken Olson (Chairman of DEC) 1977
Coldest Ever -2.1°

Av. Rainy Days 7.3
5
UNCLASSIFIED Venus
Hottest Ever 462°
Plasticity only exists within bounds …
Coldest Ever 462°
 Autonomy: ability of an organism or system, within its social context, to operate
under unpredictable future environmental states.
– Plasticity: the adaptability of an organism to changes in its environment or differences between its
various habitats.
Australia Earth May in Tanunda
Hottest Ever 50.7° Hottest Ever 56.7° Hottest Ever 28.9°
Coldest Ever -23.0° Coldest Ever -89.2° Average Max. 17.5°
Average Min. 6.9°
Mercury Mars
Coldest Ever -2.1°
Hottest Ever 695° Hottest Ever 20°
 What are the minimum bounds
Av. Rainy Days 7.3
Coldest Ever -220° Coldest Ever -153°
on the autonomous capabilities
we need in certain
circumstances?
 How do we identify, evaluate and
prioritise autonomous
capabilities?
 What are the maximum bounds
on the autonomy of various
systems?
6
UNCLASSIFIED
Predictive vs. Preparatory Scenarios
 Preparatory scenarios require us to identify the critical uncertainties … these define the bounds
of the space considered (& are the dimensions of the scenario space).
 Then scenarios can be developed by selecting points or regions within the scenario space … or
scenarios can be eschewed and dimensions used directly.
 Robust options deal with all scenarios; Plastic ones can adapt to all scenarios.
 First … find the dimensions.
Predictive Scenarios Preparatory Scenarios

 Each situation that needs to be dealt with • Focus on covering the critical
has one scenario. uncertainties (the dimensions).
– May need many scenarios. – Shell's pre-oil-shock scenarios didn't
predict an oil embargo but prepared
– Solutions are predicted futures.
for a medium term supply squeeze.
– Brittle if predictions wrong.
– Divide into ‘intuitive logic’ and
– Enemies tend to change themselves
‘indeterminist’ approaches.
to make your predictions wrong.
– Right critical uncertainties vital!
– Even experts have futurology

problems.

• Pincombe, Blunden, Pincombe & Dexter (2012) “Ascertaining a Hierarchy of Dimensions from Time-Poor Experts: linking tactical vignettes to strategic scenarios”
Technological Forecasting and Social Change http://dx.doi.org/10.1016/j.techfore.2012.05.001
• Wright &Cairns (2011) Scenario Thinking: Practical Approaches to the Future. Houndmills, UK: Palgrave Macmillan.

• Derbyshire & Wright (2014) Preparing for the future: Development of an ‘antifragile’ methodology that complements scenario planning by omitting causation. Technological
Forecasting and Social Change, 82, pp. 215-225 doi:10.1016/j.techfore.2013.07.001.
• Giffin & Reid (2003) “A Woven Web of Guesses, Canto Two: Network Centric Warfare and the Myth of Inductivism” http://www.dodccrp.org/events/8th_ICCRTS/pdf/109.pdf
7
UNCLASSIFIED
Delphi-like process for ascertaining critical variables
 The fundamentals of the Delphi process are:
– anonymity throughout
– individual contributions to an initial round of questions
– assessment of responses
– controlled feedback to participants with revised questions
– opportunity to comment, revise/amend contributions based on the feedback
– start out qualitatively and end up quantitatively
– cycle through until consensus (usually at least 3 surveys)
– arrive at consensus or otherwise based on statistical sampling
• Dalkey & Helmer (1951) The Use of Experts for the Estimation of Bombing Requirements. RM727
• Jablin & Sussman (1978) An exploration of communication and productivity in real brainstorming groups, Hum. Commun. Res. 4 329–337.
• Pincombe, Blunden, Pincombe & Dexter (2012) “Ascertaining a Hierarchy of Dimensions from Time-Poor Experts: linking tactical vignettes to
strategic scenarios” Technological Forecasting and Social Change http://dx.doi.org/10.1016/j.techfore.2012.05.001
8
UNCLASSIFIED
Experts for dimensions of future land warfare
contexts for the Australian Army …
 Obviously who we ask is very important:
– Some 8-year old boys I polled had definite ideas on future land warfare:
• very large nuclear powered tanks with rotating mincing flails;
• giant flying androids dispensing viridian green poison gas;
• genetically engineered trained paranoid Canadian arboreal octopi; …
– Maharishi Mahesh Yogi reckons that yogic flying is the way to go. The NLP’s
1997 platform promised an army of 7,000 yogic flyers and thought our main
threat was evil sentient Antarctic penguins.
 We need military experts … for this work we asked:
– 93 experienced and educated military experts drawn from all corps [as well
as 25 ACSC students].
– Representation of all stakeholder groups is very important.
 In the process of ascertaining the dimensions for the use of autonomous
systems in HADR operations
• Pincombe, Blunden, Pincombe & Dexter (2012) “Ascertaining a Hierarchy of Dimensions from Time-Poor Experts: linking tactical vignettes to
strategic scenarios” Technological Forecasting and Social Change http://dx.doi.org/10.1016/j.techfore.2012.05.001
9
UNCLASSIFIED
Computational Scenarios & Feasible Scenario Space
 The scenario space is a set of variables that can be fed into a multi-variate function:
– For software autonomous agents: they are multi-variate functions that could be tested directly;
– For physical autonomous agents: a model could be established and tested.
• Errors could be introduced in the modelling process.
 FSS: a surface which covers the set of scenario parameters for which a given capability set
can achieve success, within acceptable levels of inherent risk.
– About robustness rather than plasticity.
 Not identifying a critical variable produces big problems for both.

 On-going CS work: Hussein Abbass -- postgrad project in
“computational scenario planning for disaster management”
[see: https://www.unsw.adfa.edu.au/computational-scenario-
planning-disaster-management].
Baker, Bender, Abbass & Sarker “A Scenario-based Evolutionary Scheduling Approach for Assessing Future Supply Chain Capabilities” pp. 485-511 in Evolutionary Learning
Vol 49 of Studies in Computational Intelligence. Springer.
Abbass, Bender, Dam, Baker, Whitacre & Sarker (2008) “Computational scenario based capability planning” GECCO’08 1437-1444. 10.1145/1389095.1389378
Abbass, Bender, Gaidow & Whitbread (2011) “Computational Red Teaming: Past, Present and Future” IEEE Computational Intelligence, 6(1) 30-42. 10.1109/MCI.2010.939578
Bowden, Pincombe & Williams (2015) “Feasible Scenario Spaces: a new way of measuring capability impacts” http://www.mssanz.org.au/modsim2015/D3/bowden.pdf
10