Architecting Secure Software Systems

480 Pages·2009·3.48 MB·English

by  Asoke K. Talukder

CCRRCC__AAUU77884433__FFMM..iinndddd ii 1111//1100//22000088 44::5566::1155 PPMM
CCRRCC__AAUU77884433__FFMM..iinndddd iiii 1111//1100//22000088 44::5566::1166 PPMM
Asoke K. Talukder
Manish Chaitanya
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
AN AUERBACH BOOK
CCRRCC__AAUU77884433__FFMM..iinndddd iiiiii 1111//1100//22000088 44::5566::1166 PPMM
Auerbach Publications
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2009 by Taylor & Francis Group, LLC
Auerbach is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-13: 978-1-4200-8784-0 (Hardcover)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For orga-
nizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Talukder, Asoke K.
Architecting secure software systems / Asoke K. Talukder and Manish Chaitanya.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-1-4200-8784-0
ISBN-10: 1-4200-8784-3
1. Computer security. 2. Computer architecture. 3. Computer networks--Security measures. I.
Chaitanya, Manish. II. Title.
QA76.9.A25T34 2008
005.8--dc22 2008024408
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the Auerbach Web site at
http://www.auerbach-publications.com
CCRRCC__AAUU77884433__FFMM..iinndddd iivv 1111//1100//22000088 44::5566::1166 PPMM
Table of Contents
Abbreviations .....................................................................................................................xxi
1 Security in Software Systems .......................................................................................1
1.1 Need for Computer Security .....................................................................................1
1.1.1 I nformation Age ............................................................................................1
1.1.2 D igital Assets ................................................................................................3
1.1.2.1 S tatic Assets ....................................................................................4
1.1.2.2 Assets on Transit ............................................................................4
1.1.2.3 Securing Digital Assets ...................................................................4
1.2 V ulnerability and Attacks..........................................................................................4
1.2.1 E xploiting Vulnerability ................................................................................5
1.2.2 P assive Attacks ..............................................................................................5
1.2.3 A ctive Attacks ...............................................................................................6
1.2.4 Ha cking ........................................................................................................6
1.2.5 S ocial Engineering ........................................................................................7
1.2.6 I dentity Ā e ft ................................................................................................7
1.3 Various Security Attacks ...........................................................................................9
1.3.1 B rute-Force Attacks .......................................................................................9
1.3.2 A uthentication Attacks .................................................................................10
1.3.2.1 D ictionary Attack ..........................................................................10
1.3.2.2 R eplay Attack ................................................................................11
1.3.2.3 P assword Guessing ........................................................................11
1.3.2.4 P assword Sniffi ng .........................................................................12
1.3.3 S poofi ng Attacks .........................................................................................12
1.3.4 Den ial-of-Service Attacks ............................................................................13
1.3.4.1 Distributed Denial-of-Service Attack ...........................................13
1.3.4.2 Half-Open Attack or SYN-Flooding .............................................14
1.3.4.3 Denial-of-Service through User-ID Lock Attack ...........................15
1.3.4.4 Ping of Death Attack .....................................................................15
1.3.4.5 S murf Attack .................................................................................15
1.3.5 P acket Sniff er ...............................................................................................16
1.3.5.1 Tcpdump and Ethereal ..................................................................16
1.3.6 Taking Control of Application .....................................................................16
1.3.6.1 O verfl ow Attack ............................................................................17
1.3.6.2 Stack Smashing Attack ..................................................................17
v
CCRRCC__AAUU77884433__FFMM..iinndddd vv 1111//1100//22000088 44::5566::1177 PPMM
vi (cid:1) Table of Contents
1.3.6.3 Remote Procedure Call Attack .......................................................17
1.3.6.4 Code Injection Attacks ...................................................................17
1.3.6.5 Lu ring Attack .................................................................................18
1.4 C omputer Security ...................................................................................................18
1.4.1 P hysical Security ..........................................................................................18
1.4.2 Operating System Security ...........................................................................19
1.4.2.1 S hell Security .................................................................................19
1.4.2.2 File System Security ......................................................................20
1.4.2.3 K ernel Security ...............................................................................21
1.4.3 N etwork Security .........................................................................................21
1.5 Counter External Ā re ats ........................................................................................22
1.5.1 S topping Attacker ........................................................................................22
1.5.2 F irewall ........................................................................................................22
1.5.3 Intrusion Detection System .........................................................................23
1.5.4 Intrusion Prevention System ........................................................................24
1.5.5 H oneypot .....................................................................................................25
1.5.6 Penetration Test and Ethical Hacking ..........................................................25
1.6 S ecurity Programming .............................................................................................25
1.6.1 S ecurity Attributes .......................................................................................26
1.6.1.1 C onfi dentiality ..............................................................................26
1.6.1.2 I ntegrity ........................................................................................26
1.6.1.3 A vailability ....................................................................................27
1.6.1.4 A uthentication ..............................................................................27
1.6.1.5 A uthorization ................................................................................27
1.6.1.6 A ccounting....................................................................................27
1.6.1.7 A nonymity ....................................................................................28
1.6.2 S ecured Programming .................................................................................28
1.6.3 Sa fe Programming .......................................................................................28
1.6.4 V ulnerability Remediation ...........................................................................28
1.7 Dat abase Security ...................................................................................................29
1.7.1 Dat abase Authentication..............................................................................29
1.7.2 Dat abase Privileges ......................................................................................30
1.7.3 S ecure Metadata ...........................................................................................31
1.7.4 Customize Access to Information .................................................................31
1.7.5 Virtual Private Database ..............................................................................32
1.7.6 High Availability Database ...........................................................................33
1.7.7 Dat abase Encryption ....................................................................................33
1.7.8 PL/SQL Code Obfuscation .........................................................................34
1.8 C ommon Criteria ....................................................................................................34
1.8.1 Evaluation Assurance Levels .........................................................................35
1.9 S ecurity Standards ..................................................................................................36
1.9.1 Public-Key Cryptographic Standards...........................................................36
1.9.1.1 Advanced Encryption Standard ....................................................38
1.9.1.2 T ransport Layer Security ...............................................................38
1.9.2 C ERT ..........................................................................................................39
1.9.3 Open Web Application Security Project ......................................................40
CCRRCC__AAUU77884433__FFMM..iinndddd vvii 1111//1100//22000088 44::5566::1177 PPMM
Table of Contents (cid:1) vii
1.9.4 National Institute of Standards and Technology ..........................................40
1.9.5 Organization for the Advancement of Structured Information Standards ...40
1.9.6 System Security Engineering Capability Maturity Model .............................41
1.9.7 ISO 17799 ....................................................................................................41
1.10 Summary ................................................................................................................42
References .........................................................................................................................42
2 Architecting Secure Software Systems .......................................................................45
2.1 Bu ilding Secured Systems .......................................................................................45
2.1.1 S ecurity Development Lifecycle .................................................................46
2.2 Security Requirements Analysis .............................................................................47
2.2.1 Functional versus Nonfunctional Requirements ........................................48
2.2.2 U se Case ....................................................................................................48
2.2.3 M isuse Case ...............................................................................................49
2.2.4 Corepresenting Use and Misuse Cases .......................................................50
2.2.5 D efi ning Security Requirements .................................................................51
2.3 Ā re at Modeling ....................................................................................................52
2.3.1 S TRIDE ....................................................................................................54
2.3.2 A ttack Tree ................................................................................................54
2.3.3 D READ.....................................................................................................56
2.3.4 A ttack Surface ............................................................................................57
2.3.5 Putting It All Together ...............................................................................58
2.4 S ecurity Design .....................................................................................................58
2.4.1 Patterns and Antipatterns ...........................................................................58
2.4.2 A ttack Patterns ............................................................................................59
2.4.3 S ecurity Design Patterns .............................................................................59
2.4.3.1 Single Access Point ......................................................................62
2.4.3.2 C heckpoint .................................................................................63
2.4.3.3 R oles ...........................................................................................63
2.4.3.4 S ession .........................................................................................64
2.4.3.5 Full View with Errors ..................................................................64
2.4.3.6 L imited View ..............................................................................64
2.4.3.7 Secure Access Layer .....................................................................64
2.4.3.8 L east Privilege ..............................................................................65
2.4.3.9 J ournaling ....................................................................................65
2.4.3.10 C lose Gracefully ...........................................................................65
2.4.4 A uthentication ...........................................................................................66
2.4.4.1 Delay Authentication Prompt ......................................................66
2.4.4.2 Encrypt the Password ..................................................................66
2.4.4.3 S trong Password ...........................................................................67
2.4.4.4 Prevent Replay Attack on Password ..............................................67
2.4.4.5 O ne-Time Password .....................................................................67
2.4.4.6 Prevent Password Guessing ..........................................................67
2.4.4.7 M ultikey Authentication .............................................................68
2.4.4.8 M ultifactor Authentication .........................................................68
2.4.4.9 Build Knowledgebase on Password Usage ...................................68
CCRRCC__AAUU77884433__FFMM..iinndddd vviiii 1111//1100//22000088 44::5566::1177 PPMM
viii (cid:1) Table of Contents
2.4.4.10 C hallenge Questions .................................................................69
2.4.4.11 Pass Sentences and Passphrases ..................................................69
2.4.4.12 M nemonic Password .................................................................70
2.4.4.13 R andomized Password ...............................................................70
2.4.4.14 Reverse Turing Test ...................................................................70
2.4.4.15 Storing the Password .................................................................71
2.4.4.16 S ingle Sign-On ..........................................................................71
2.4.5 A uthorization .............................................................................................72
2.4.5.1 R ole-Based Security.....................................................................72
2.5 S ecurity Coding ....................................................................................................72
2.5.1 S ecurity Algorithms ...................................................................................73
2.5.1.1 Symmetric Key Cryptography .....................................................73
2.5.1.2 Public Key Cryptography .............................................................74
2.5.1.3 Secret Sharing and Ā re shold Cryptography ...............................75
2.5.1.4 D igital Signature ..........................................................................76
2.5.2 S ecurity Protocol ........................................................................................77
2.5.3 K ey Generation ..........................................................................................78
2.5.3.1 Key for Symmetric Cryptography................................................79
2.5.3.2 Keys for Public Key Cryptography ..............................................79
2.5.4 S ession Management ..................................................................................80
2.5.5 Logging and Auditing .................................................................................81
2.6 Sa fe Programming ..................................................................................................81
2.6.1 A rtifi cial Hygiene .......................................................................................82
2.6.1.1 A rtifi cial Hygiene in Networking Applications ...........................83
2.6.1.2 A rtifi cial Hygiene in Business Applications .................................83
2.7 S ecurity Review .....................................................................................................84
2.7.1 Step 1: Identify Security Code Review Objectives ......................................84
2.7.2 Step 2: Perform Preliminary Scan ..............................................................84
2.7.3 Step 3: Review Code for Security Issues ......................................................85
2.7.4 Step 4: Review for Security Issues Unique to Architecture .........................89
2.8 Generating the Executable .....................................................................................89
2.8.1 Tools for Checking Code ...........................................................................89
2.8.1.1 L int .............................................................................................89
2.8.1.2 P REfast .......................................................................................90
2.8.1.3 Fx Cop .........................................................................................90
2.8.1.4 A ppVerif ......................................................................................90
2.8.2 Windows Compilation Option ...................................................................91
2.8.2.1 / GS Option ..................................................................................91
2.8.2.2 / SAFESEH Option ......................................................................91
2.8.2.3 / NXCOMPAT Option .................................................................91
2.9 S ecurity Testing .....................................................................................................92
2.9.1 V ulnerability Assessment ............................................................................92
2.9.1.1 E xternal Vulnerability Assessments .............................................92
2.9.1.2 Internal Vulnerability Assessments ..............................................92
2.9.1.3 Vulnerability Assessments Tools ..................................................92
2.9.2 Code Coverage Tools .................................................................................93
2.9.3 Negative or Nonoperational Testing...........................................................93
CCRRCC__AAUU77884433__FFMM..iinndddd vviiiiii 1111//1100//22000088 44::5566::1177 PPMM
Table of Contents (cid:1) ix
2.9.4 P enetration Testing ....................................................................................93
2.9.5 Et hical Hacking .........................................................................................94
2.9.6 F uzz Testing ...............................................................................................94
2.9.7 F ault Injection ............................................................................................95
2.9.7.1 Fault Injection through Traps ......................................................95
2.9.7.2 Fault Injection through Debugger ...............................................95
2.9.8 Common Criteria and Evaluation Assessment Level ..................................96
2.9.8.1 Evaluation Assessments Level ......................................................96
2.10 S ecured Deployment .............................................................................................98
2.11 S ecurity Remediation ............................................................................................99
2.11.1 Deb ugging .............................................................................................100
2.12 S ecurity Documentation .....................................................................................100
2.12.1 U ser Documentation ..............................................................................101
2.12.2 S ystem Documentation ..........................................................................101
2.13 Security Response Planning .................................................................................101
2.14 Sa fety-Critical Systems .........................................................................................102
2.14.1 F ormal Methods .....................................................................................102
2.15 S ummary .............................................................................................................104
References .......................................................................................................................104
3 Constructing Secured and Safe C/UNIX Programs .................................................107
3.1 UNIX and Linux History ......................................................................................107
3.1.1 Extremely Reliable Operating System .........................................................108
3.1.2 Why UNIX Is Important ...........................................................................109
3.2 UNIX and Linux Security .....................................................................................109
3.2.1 C apability-Based System .............................................................................110
3.2.2 Security Holes in UNIX .............................................................................110
3.3 Privileges in UNIX ................................................................................................111
3.3.1 Elevation of Privilege in UNIX ..................................................................111
3.3.2 Writing Secure Set User ID Programs ........................................................112
3.3.3 Principle of Least Privilege ..........................................................................113
3.4 S ecured Network Programming .............................................................................114
3.4.1 Generic Security Service Application Program Interface .............................114
3.4.2 Secure Network Programming ...................................................................115
3.4.3 Open Secure Socket Layer Application Program Interface ..........................117
3.4.4 S ockets .......................................................................................................119
3.4.5 R aw Socket .................................................................................................119
3.5 U NIX Virtualization ............................................................................................120
3.6 UNIX Security Logging ........................................................................................121
++
3.7 C /C Language .................................................................................................123
++
3.8 Common Security Problems with C/C ............................................................125
3.8.1 Memory Availability Vulnerability .............................................................125
3.8.1.1 M emory Leak ...............................................................................125
3.8.2 Memory Corruption Vulnerability ............................................................126
3.8.2.1 M emory Overfl ow .......................................................................126
3.8.2.2 S tack Smashing ............................................................................129
3.8.2.3 H eap Smashing ............................................................................132
CCRRCC__AAUU77884433__FFMM..iinndddd iixx 1111//1100//22000088 44::5566::1177 PPMM