Securing Advanced Metering Infrastructure (AMI) in - Zeyar Aung

92 Pages·2012·1.3 MB·English

Securing Advanced Metering
Infrastructure (AMI) in Smart Grid
using Intrusion Detection System
(IDS)

By
Mustafa Amir Faisal

A Thesis Presented to the
Masdar Institute of Science and Technology
in Partial Fulfillment of the Requirements for the Degree of
Master of Science
In
Materials Science and Engineering

© 2012 Masdar Institute of Science and Technology
All rights reserved
Abstract
Advancedmeteringinfrastructure(AMI)isanimperativecomponentofsmartgrid,
as it is responsible for collecting, measuring, analyzing energy usage data, and
transmittingthesedatatothedataconcentratorandthentoacentralsysteminthe
utility side. Therefore, the security of AMI is one of the most concerning issues
in smart grid implementation. In this research, we propose an intrusion detection
system (IDS) architecture for AMI which will act as a complimentary with other
security measures. This IDS architecture consists of three local IDSs placed in
smartmeters,dataconcentrators,andcentralsystem(AMIheadend).
For detecting anomaly, we use stream data mining approach on public KD-
DCUP 1999 data set for each three components in AMI. Seven stream mining
classifiersarestudiedandtheirfeasibilityisanalyzedforthesecomponents. From
our result and analysis, stream data mining techniques show promising potential
for solving security issues in AMI. Moreover, possible required characteristics of
algorithmsareidentifiedforthesecomponentsforfutureresearch.
ThisresearchwassupportedbytheGovernmentofAbuDhabitohelpfulfillthe
visionofthelatePresidentSheikhZayedBinSultanAlNayhanforsustainable
developmentandempowermentoftheUAEandhumankind.
Acknowledgments
Firstofall, IthankthealmightyAllahtogivethestrengthtofulfilthistask. Then
my gratitude goes to my family members without their help and support, it is im-
possible.
I would like to take this opportunity to thank those who actively guided and
helpedmeinthisresearch. Foremost,Iwouldliketoexpressmydeepappreciation
tomyadvisorDr. ZeyarAungforhiscontinuoussupportformyM.Sc. studyand
research. His guidance, patience, motivation, and support helped me to develop a
deep understanding of the subject. Besides my advisor, I would like to thank my
thesissupervisorycommitteemembers: Dr. WeiLeeWoonandDr. JacobCrandall
fortheirvaluabletime,comments,andadvices.
Finally, and most importantly, once again my special thanks go to my family
membersfortheirsupportandlove. Idedicatethisthesistomymother.
MustafaAmirFaisal,
MasdarInstitute,May15,2012.
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Thesisstatement–Objectives. . . . . . . . . . . . . . . . . . . . 3
1.3 ResearchContributions . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 RelevancetoMasdar . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 ThesisOrganization . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 AdvancedMeteringInfrastructure 5
2.1 FeaturesandBenefits . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 AMIComponents . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 LiteratureReview 10
3.1 AMIsecurityIssues . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 StreamDataMininginIntrusionDetectionSystem . . . . . . . . 12
3.3 IntrusionDetectionSysteminAMI . . . . . . . . . . . . . . . . . 13
4 ProposedIDSArchitectureforAMI 15
6
4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2 ProposedArchitecture . . . . . . . . . . . . . . . . . . . . . . . 16
5 ExperimentalSetUp 20
5.1 DataSet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.2 UsedTools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.2.1 WaikatoEnvironmentforKnowledgeAnalysis(WEKA) . 23
5.2.2 MassiveOnlineAnalysis(MOA). . . . . . . . . . . . . . 23
5.3 EvaluationProcedures . . . . . . . . . . . . . . . . . . . . . . . 25
5.3.1 EvaluatePrequential . . . . . . . . . . . . . . . . . . . . 25
5.3.2 EvaluateInterleavedTestThenTrain . . . . . . . . . . . . . 26
5.4 EvaluationMetrics . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.4.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.4.2 KappaStatistic . . . . . . . . . . . . . . . . . . . . . . . 27
5.4.3 FalsePositiveRate(FPR)andFalseNegativeRate(FNR) 27
5.4.4 Evaluationtime(seconds) . . . . . . . . . . . . . . . . . 28
5.4.5 Modelcost(RAM-Hours) . . . . . . . . . . . . . . . . . 28
5.4.6 Modelsize(KiloBytes) . . . . . . . . . . . . . . . . . . 28
5.5 AlgorithmsExplored . . . . . . . . . . . . . . . . . . . . . . . . 28
5.5.1 HoeffdingTree . . . . . . . . . . . . . . . . . . . . . . . 29
5.5.2 AdaptiveWindowing(ADWIN) . . . . . . . . . . . . . . 30
5.5.3 AccuracyUpdatedEnsemble(AUE) . . . . . . . . . . . . 31
5.5.4 ActiveClassifier . . . . . . . . . . . . . . . . . . . . . . 33
5.5.5 LeveragingBagging . . . . . . . . . . . . . . . . . . . . 35
5.5.6 LimitedAttributeClassifier(LimAttClassifier) . . . . . . 36
5.5.7 OzaBaggingwithADWIN(OzaBagAdwin) . . . . . . . 38
5.5.8 Oza Bagging with Adaptive Size Hoffding Tree (OzaBa-
gASHT) . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.5.9 SingleClassifierDrift . . . . . . . . . . . . . . . . . . . 40
5.6 ExperimentalFlow . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.6.1 DataPreprocessing . . . . . . . . . . . . . . . . . . . . . 42
5.6.2 ExperimentalEnvironment . . . . . . . . . . . . . . . . . 42
5.6.3 Tuningparametervalues . . . . . . . . . . . . . . . . . . 43
6 ResultsandAnalysis 45
6.1 FullVersionDataSet . . . . . . . . . . . . . . . . . . . . . . . . 45
6.2 ImprovedVersionDataSet . . . . . . . . . . . . . . . . . . . . . 53
7 Discussion 59
7.1 Identifyingsuitablealgorithms . . . . . . . . . . . . . . . . . . . 59
7.2 Comparisonwithexistingworks . . . . . . . . . . . . . . . . . . 62
7.3 EvaluationofOurproposedArchitecture . . . . . . . . . . . . . . 62
8 FutureWork 64
9 Conclusion 66
A TunedParameterValues 68
A.1 TunedparametervaluesforFullVersionDataSet . . . . . . . . . 68
A.2 TunedparametervaluesforImprovedDataSet . . . . . . . . . . 69
B Abbreviations 72
List of Tables
4.1 Characteristicsofsmartmeter,dataconcentrator,andAMIheadend. 16
5.1 Attacksintrainandtestdatasets . . . . . . . . . . . . . . . . . . 21
5.2 BasicfeaturesofindividualTCPconnections . . . . . . . . . . . 21
5.3 Contentfeatureswithinaconnectionsuggestedbydomainknowl-
edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.4 Trafficfeaturescomputedusingatwo-secondtimewindow . . . . 22
5.5 StatisticsforKDDCup99DataSet. . . . . . . . . . . . . . . . . 23
5.6 Variousattacks’samplesizesforimprovedversiondataset. . . . . 23
5.7 ConfusionMatrix . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.8 StepsforLimitedAttributeClassifier . . . . . . . . . . . . . . . . 37
5.9 ExperimentalEnvironment . . . . . . . . . . . . . . . . . . . . . 42
6.1 Performancecomparisonforclassifiersforfullversion. . . . . . . 46
6.2 PerformancecomparisonforclassifiersforTestdataset. . . . . . 47
6.3 Performance comparison for classifiers for improved version of
trainingdataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
9
6.4 Performancecomparisonforclassifiersforimprovedversion. . . . 53
7.1 Orderofclassifiersaccordingtotheirperformanceforfullversion
testdata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.2 Order of classifiers according to their performance for improved
versiontestdataset. . . . . . . . . . . . . . . . . . . . . . . . . . 60
A.1 TunedparametervaluesforAccuracyUpdatedEnsemble. . . . . . 68
A.2 TunedparametervaluesforActiveClassifier . . . . . . . . . . . . 69
A.3 TunedparametervaluesforOzaASHT. . . . . . . . . . . . . . . . 69
A.4 TunedparametervaluesforLimAttClassifier. . . . . . . . . . . . 69
A.5 TunedparametervaluesforOzaBagAdwin. . . . . . . . . . . . . 69
A.6 TunedparametervaluesforSingleClassifierDrift. . . . . . . . . . 69
A.7 TunedparametervaluesforLeveragingBag. . . . . . . . . . . . . 69
A.8 TunedparametervaluesforAccuracyUpdatedEnsemble. . . . . . 70
A.9 TunedparametervaluesforActiveClassifier. . . . . . . . . . . . 70
A.10 TunedparametervaluesforLeveragingBag. . . . . . . . . . . . . 70
A.11 TunedparametervaluesforLimAttrClassifier. . . . . . . . . . . . 70
A.12 TunedparametervaluesforLimAttrClassifier(numAtts). . . . . . 70
A.13 TunedparametervaluesforOzaBagAdwin. . . . . . . . . . . . . 70
A.14 TunedparametervaluesforOzaBagASHT. . . . . . . . . . . . . 71
A.15 TunedparametervaluesforSingleClassifierDrift. . . . . . . . . . 71
List of Figures
2.1 OverviewofAMIcomponentsandnetworks. . . . . . . . . . . . 9
4.1 (a)SmartmeterwithIDS.(b)IDSforasmartmeter. . . . . . . . . 17
4.2 ArchitectureofwholeIDSinAMI. . . . . . . . . . . . . . . . . . 18
4.3 IntrusiondetectionprocedurefromsmartmetertoAMIheadend. . 18
5.1 Classificationcycleofdatastream[40] . . . . . . . . . . . . . . . 25
5.2 Pseudo-codeforADWIN[39]. . . . . . . . . . . . . . . . . . . . 32
5.3 Pseudo-codeforAUE[20]. . . . . . . . . . . . . . . . . . . . . . 34
5.4 StrategyFrameworkActiveLearning[20]. . . . . . . . . . . . . 34
5.5 LeveragingBagging[17]. . . . . . . . . . . . . . . . . . . . . . . 36
5.6 ExperimentalFlow(lefttoright). . . . . . . . . . . . . . . . . . . 43
6.1 Accuracyforfullversiontestdatasetwithtunedparametervalues. 48
6.2 Correctdetectionsofvariousattacksfortestdataset. . . . . . . . 48
6.3 Memoryconsumedbyvariousclassifiers(TestDatasetforfullver-
sion). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
11