Full Thesis including appendices (239-page pdf)

239 Pages·2011·19.87 MB·English

© 2011, Jennifer L. Bayuk. All rights reserved.
iii
MEASURING SYSTEMS SECURITY:
AN INITIAL SECURITY THEORETICAL CONSTRUCT FRAMEWORK
ABSTRACT
System security metrics have evolved side by side with the advent of cyber security tools
and techniques. They have been derived from the techniques rather than specified as
system requirements. This dissertation surveys the evolution and state of the practice of
system security metrics from both a technical and historical perspective. The survey leads
to the conclusion that currently accepted methodology for measuring system security has
no empirical basis. This research provides new criterion with which to evaluate security
metrics, and proposes a new methodology for security theory attribute construction
(“STAC”). The STAC framework has been applied to case studies in Cloud Computing
and Mobile Communications. Specific research in a variety of system security topics is
recommended to reinforce these results, and provide theoretical foundation for more
effective tools and techniques for systems security engineering.
Author: Jennifer L. Bayuk
Advisor: Ali Mostashari
Date: December 1, 2011
Department: School of Systems and Enterprises
Degree: Doctor of Philosophy
iv

ACKNOWLEDGMENTS
This has truly been a journey into uncharted territory for which every available
navigation aid assumes a different set of environmental conditions. I am fully indebted to
my committee members who lent every available methodology within their grasp of
expertise to my cause. Ali Mostashari led this team of diverse adventurers with a rigorous
scientific approach, iteratively measuring the extent to which each endeavor brought us
closer to our destination. Barry Horowitz maintained our focus on the community goal of
security improvement by consistently framing successive attempts at security problem
resolution in terms of efficacy against an adversary. Paul Rohmeyer repeatedly quantified
this research in terms of its contribution to the field of system security, and ensured that
no stone was left unturned to strengthen our bonds with scientific thinkers in the security
community. Brian Sauser intuitively drew parallels between systems security problems
and other complex problems that benefit from applied systems thinking, which ultimately
led to the development of the STAC framework. In addition, though not an official
member of the committee, Stevens adjunct professor Bill Miller provided valuable insight
on how to connect the research hypothesis to survey results. Thank you all for your
efforts to bring this journey to fruition.
I also acknowledge my husband for his support throughout this journey, though support is
far too meek a word to describe the sustenance he provides. Thank you Michael.
v

TABLE OF CONTENTS

Abstract .............................................................................................................................. iii
Acknowledgments.............................................................................................................. iv
List of Tables ..................................................................................................................... ix
List of Figures ......................................................................................................................x

1. Introduction and Problem Setting ............................................................................... 1
1.1. Introduction .......................................................................................................... 1
1.2. Overview of the Problem ..................................................................................... 1
1.3. Problem Statement ............................................................................................... 3
1.4. Research Objectives ............................................................................................. 5
1.4.1. Research Hypothesis and Implications ......................................................... 6
1.4.2. Hypothesis Validation ................................................................................. 10
1.4.3. Research Approach ..................................................................................... 11
1.5. Uniqueness of this Research .............................................................................. 12
1.6. Dissertation Organization and Structure ............................................................ 13
2. Security Metrics Literature Review .......................................................................... 15
2.1. Face Valid .......................................................................................................... 17
vi
2.2. Content Valid ..................................................................................................... 20
2.3. Criterion Valid.................................................................................................... 25
2.4. Construct Valid .................................................................................................. 29
2.5. Taxonomy........................................................................................................... 34
2.6. Outlook ............................................................................................................... 39
3. Security Survey ......................................................................................................... 41
3.1. Survey Design .................................................................................................... 43
3.2. Survey Results .................................................................................................... 47
3.2.1. Qualifications .............................................................................................. 47
3.2.2. Rank Results ............................................................................................... 49
3.2.3. Subsequent Analysis and Feedback ............................................................ 50
4. Security Theory Attribute Construction.................................................................... 54
5. System Security Engineering Case Studies .............................................................. 64
5.1. Cloud Computing ............................................................................................... 64
5.1.1. Cloud Computing Security Problem ........................................................... 64
5.1.2. A Structured Expression ............................................................................. 66
5.1.3. System Definition ....................................................................................... 68
5.1.4. Conceptual Model ....................................................................................... 69
5.1.5. Comparison of the Model to the Structured Problem ................................. 75
vii
5.1.6. Identify Feasible Changes in Structure, Procedure, and Attitude ............... 82
5.1.7. Recommend Action to Improve the Situation ............................................ 85
5.1.8. Cloud Computing Security Validation........................................................ 86
5.2. Mobile Communications .................................................................................... 89
5.2.1. Mobile Communications Security Problem ................................................ 89
5.2.2. A Structured Expression ............................................................................. 90
5.2.3. System Definition ....................................................................................... 91
5.2.4. Conceptual Model ....................................................................................... 93
5.2.5. Comparison of the Model to the Structured Problem ................................. 96
5.2.6. Identify Feasible Changes in Structure, Procedure, and Attitude ............... 97
5.2.7. Recommend Action to Improve the Situation .......................................... 100
5.2.8. Mobile Communications Security Validation .......................................... 100
5.3. Case Study Conclusions ................................................................................... 102
6. Summary and Conclusions ..................................................................................... 103
Appendix A – Hypothesis Derivation Logic .................................................................. 108
Appendix B – Survey Design ......................................................................................... 112
Appendix C – Survey Analysis Detail ............................................................................ 117
Appendix D – Survey Questions and Answers ............................................................... 136
Appendix E – Group Independence Tests ...................................................................... 204
Appendix F - Descriptive statistics for all security attributes ......................................... 214
viii
References ....................................................................................................................... 216
Vita ................................................................................................................................. 223
ix

LIST OF TABLES

Table 1: Attribute Rank Order for Survey Responses ...................................................... 49
Table 2: Clusters of Ranked Attributes ............................................................................. 51
Table 3: Cloud Computing Requirements ........................................................................ 70
Table 4: Example Verification Metrics ............................................................................. 84
Table 5: Cloud Computing Validation Results ................................................................. 89
Table 6: Mobile Communications Requirements ............................................................. 93
Table 7: Mobile Communications Validation Results .................................................... 102
x
LIST OF FIGURES
Figure 1: Security Systemigram........................................................................................ 20
Figure 2: Security Standards as a Theoretical Construct .................................................. 32
Figure 3: Security Models ................................................................................................. 32
Figure 4: Example Security Metrics ................................................................................. 36
Figure 5: Example Business-Oriented Security Metrics Taxonomy ................................ 37
Figure 6: Taxonomy of Security Metrics .......................................................................... 39
Figure 7: Survey Respondent Demographics ................................................................... 48
Figure 8: Example Security Architecture Framework ...................................................... 55
Figure 9: Roadmap Path for Security................................................................................ 57
Figure 10: Set-Theoretic Illustration of the system Level Approach ............................... 58
Figure 11: Security Metrics Framework Overlay on the Vee Model ............................... 60
Figure 12: Security Theory Attribute Construction Framework....................................... 63
Figure 13: Cloud Computing Problem .............................................................................. 66
Figure 14: Structured Cloud Problem ............................................................................... 68
Figure 15: Cloud System Definition ................................................................................. 69
Figure 16: Cloud Security Model ..................................................................................... 75
Figure 17: Cloud Computing Metrics Taxonomy ............................................................. 85
Figure 18: Cloud Computing STAC Metrics Report ........................................................ 88
Figure 19: Mobile Communications Problem................................................................... 90
Figure 20: Structured Mobile Communications Problem ................................................. 92
Figure 21: Mobile System Definition ............................................................................... 92