Validation Models of Adversary behavior

51 Pages·2016·17.07 MB·English

Validation Models of
Adversary behavior
Jing Zhang, Jun Zhuang, University at Buffalo
1
• Validating Models ooff AAddvveerrssaarryy BBeehhaavviioorrss
!
• Should models be validated? YES!
!
• Can models be validated? Sometimes…
!
• How would models be validated?
!
• Based on real scenario;
!
• With/without data
!
• Thought Experiments, Expert opinion, Interviews
!
• Validation Exercises, Simulation, Case Studies
!

SSuummmmaarryy
2
Introduction
• Hundreds of billions of dollars have been spent on
!
homeland security since September 11, 2001.
• Numerous models have been developed to study the
!
strategic interactions between defenders and adversaries.
• Unfortunately, few if any models have yet been validated
!
using empirical data, limiting the application of those
&)$ ''((""++$$
models in practice. ')$ ')"+$
*+$
*'"%$ *'"+$
')$ **"#$
*)"*$
('"($
/*)$
. (!"&$
- ()"!$ (#"#$
+
,
*+,()$ %%"&$ %'"!$
)
(
%&%)$
$ !!!!""##$$
#
"
!!)$
#)$ 3
)$
!))!$ !))%$ !))($ !))*$ !))'$ !))&$ !)),$ !))+$ !)#)$ !)##$ !)#!$ !)#%$ !)#($ !)#*$ !)#'$
0%12'
Conferences on validating models of adversary behavior
4
Conference word clouds
5
Model Validation Exercises
• Dataset I: Time Series Data for Terrorist Events
!
• This dataset contains time series data for the annual
!
number of terrorist incidents and the annual number of
fatalities from terrorist attacks from 1981 to 2014.
6
• Dataset II: Terrorism Consequences and Counterterrorism
!
Investments for US Urban Areas
• This dataset gives counter-terrorism information on 47 US
!
major urban areas.
7
Sample results:
Total UASI funding against Incidents
1800 18
1600 16
1400 14
1200 12
1000 10
800 8
600 6
400 4
200 2
0 0
New York, NY Chicago San Francisco Washington, DC Los Angeles-Long Beach Philadelphia, PA-NJ Boston, MA-NH Houston Newark, NJ Seattle-Bellevue-Everett Jersey City Detroit Las Vegas, NV-AZ Oakland, CA Orange County, CA Cleveland San Diego Miami, FL Minneapolis-St.Paul, MN-WI Denver Baltimore Atlanta Dallas St. Louis, MO-IL Portland-Vancouver, OR-WA Phoenix-Mesa San Jose otte-Gastonia-RockHill, NC- Kansas City, MO-KS Milwaukee-Waukesha, WI New Haven-Meriden, CT Buffalo-Niagara Falls Pittsburgh Cincinnati, OH-KY-IN pa-St. Petersburg-Clearwater New Orleans Columbus, OH Indianapolis Sacramento Louisville, KY-IN Orlando Memphis, TN-AR-MS Albany-Schenectady-Troy Richmond-Petersburg San Antonio Baton Rouge Fresno
harl Tam
C
Total funding Total Incidents
Correlation of UASI funding to incidents
experienced by a city is 0.69 by the end of 10
years
8
Initially (2004) the correlation is 0.18
• Dataset III: Target Type and Attack Type: Terrorist Preferences
!
• This dataset provides the annual number of terrorist incidents and the
!
annual number of fatalities from terrorist attacks during 1991-2014,
broken down into different attack types and different target types.
9
• Dataset IV: Effectiveness of Security Countermeasures on
!
Critical Infrastructure
• This hypothetical dataset provides the dollar amount spent by 25
!
critical infrastructures on security countermeasures in 2010,
ranging from $0 to $497,500, with a total of $3.52 million.
10