Applied Network Security Monitoring This page intentionally left blank Applied Network Security Monitoring Collection, Detection, and Analysis Chris Sanders Jason Smith David J. Bianco, Technical Editor AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON NEW YORK (cid:129) OXFORD (cid:129) PARIS (cid:129) SAN DIEGO SAN FRANCISCO (cid:129) SINGAPORE (cid:129) SYDNEY (cid:129) TOKYO Syngress is an imprint of Elsevier AcquiringEditor:ChrisKatsaropoulos EditorialProjectManager:BenjaminRearick ProjectManager:PunithavathyGovindaradjane Designer:MatthewLimbert Copyeditor:EllenSanders SyngressisanimprintofElsevier 225WymanStreet,Waltham,MA02451,USA Copyright#2014ElsevierInc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronic ormechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem, withoutpermissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformation aboutthePublisher’spermissionspoliciesandourarrangementswithorganizationssuchasthe CopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite: www.elsevier.com/permissions. ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher (otherthanasmaybenotedherein). Notices Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroaden ourunderstanding,changesinresearchmethodsorprofessionalpractices,maybecomenecessary. Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluating andusinganyinformationormethodsdescribedherein.Inusingsuchinformationormethodsthey shouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhave aprofessionalresponsibility. Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assume anyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability, negligenceorotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideas containedinthematerialherein. LibraryofCongressCataloging-in-PublicationData Applicationsubmitted BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:978-0-12-417208-1 PrintedandboundintheUnitedStatesofAmerica 14 15 16 17 18 10 9 8 7 6 5 4 3 2 1 ForinformationonallSyngresspublications,visitourwebsiteatstore.elsevier.com/Syngress This book is a product of strength gained through love. This book is dedicated to God, my wife Ellen, and allthosewho continue to love andsupportme. “But those who hope in the Lord will renew their strength. They will soar on wingslikeeagles,theywillrunandnotgrowweary,theywillwalkandnotbefaint.” Isaiah 40:31 (NIV) This page intentionally left blank Contents Acknowledgements...................................................................................................xi Aboutthe Authors.................................................................................................xiii Foreword..................................................................................................................xv Preface..................................................................................................................xvii CHAPTER 1 ThePracticeofApplied Network Security Monitoring...............1 Key NSM Terms.............................................................................3 IntrusionDetection.........................................................................5 NetworkSecurity Monitoring........................................................6 Vulnerability-Centric vs. Threat-Centric Defense.........................9 The NSM Cycle:Collection, Detection, and Analysis..................9 Challenges toNSM.......................................................................11 Definingthe Analyst....................................................................12 Security Onion..............................................................................19 Conclusion....................................................................................24 SECTION 1 COLLECTION CHAPTER 2 Planning DataCollection......................................................27 The AppliedCollection Framework (ACF).................................28 Case Scenario:OnlineRetailer....................................................34 Conclusion....................................................................................42 CHAPTER 3 TheSensor Platform.............................................................43 NSM Data Types..........................................................................45 Sensor Type..................................................................................47 Sensor Hardware...........................................................................49 Sensor OperatingSystem.............................................................61 Sensor Placement..........................................................................61 Securingthe Sensor......................................................................70 Conclusion....................................................................................73 CHAPTER 4 Session Data........................................................................75 Flow Records................................................................................76 Collecting SessionData................................................................81 Collecting and AnalyzingFlow Data with SiLK........................83 Collecting and AnalyzingFlow Data with Argus.......................92 SessionData Storage Considerations...........................................95 Conclusion....................................................................................97 vii viii Contents CHAPTER 5 FullPacket CaptureData......................................................99 Dumpcap.....................................................................................101 Daemonlogger.............................................................................102 Netsniff-NG................................................................................104 Choosing the Right FPCCollection Tool..................................105 Planning for FPCCollection......................................................106 Decreasingthe FPCData Storage Burden.................................111 Managing FPCData Retention..................................................115 Conclusion..................................................................................120 CHAPTER 6 Packet String Data.............................................................121 Defining Packet StringData.......................................................122 PSTR Data Collection................................................................124 Viewing PSTR Data...................................................................135 Conclusion..................................................................................146 SECTION 2 DETECTION CHAPTER 7 DetectionMechanisms,Indicators of Compromise, and Signatures...................................................................149 Detection Mechanisms...............................................................149 Indicators of Compromise and Signatures.................................151 Managing Indicators and Signatures..........................................162 Indicator andSignature Frameworks.........................................168 Conclusion..................................................................................173 CHAPTER 8 Reputation-BasedDetection................................................175 PublicReputation Lists..............................................................176 AutomatingReputation-BasedDetection...................................184 Conclusion..................................................................................201 CHAPTER 9 Signature-Based Detection with Snort andSuricata..............203 Snort............................................................................................205 Suricata.......................................................................................208 Changing IDS Engines inSecurityOnion.................................211 InitializingSnort and Suricata for IntrusionDetection.............211 ConfiguringSnortand Suricata..................................................214 IDS Rules....................................................................................229 Viewing Snort andSuricataAlerts.............................................252 Conclusion..................................................................................254 Contents ix CHAPTER 10 TheBro Platform................................................................255 Basic BroConcepts....................................................................256 Running Bro................................................................................257 Bro Logs.....................................................................................258 CreatingCustom DetectionTools with Bro..............................262 Conclusion..................................................................................287 CHAPTER 11 Anomaly-BasedDetection with Statistical Data.....................289 Top Talkerswith SiLK...............................................................289 ServiceDiscovery with SiLK.....................................................294 Furthering Detection with Statistics...........................................299 VisualizingStatistics with Gnuplot............................................302 VisualizingStatistics with Google Charts.................................306 VisualizingStatistics with Afterglow........................................310 Conclusion..................................................................................316 CHAPTER 12 Using Canary Honeypots for Detection.................................317 Canary Honeypots......................................................................318 TypesofHoneypots....................................................................319 Canary Honeypot Architecture...................................................320 Honeypot Platforms....................................................................323 Conclusion..................................................................................338 SECTION 3 ANALYSIS CHAPTER 13 Packet Analysis.................................................................341 Enter the Packet..........................................................................342 Packet Math................................................................................344 Dissecting Packets......................................................................350 Tcpdump for NSM Analysis......................................................355 TShark for Packet Analysis........................................................359 Wireshark for NSM Analysis.....................................................363 Packet Filtering...........................................................................376 Conclusion..................................................................................384 CHAPTER 14 Friendly and Threat Intelligence..........................................385 The Intelligence Cycle for NSM................................................386 Generating Friendly Intelligence................................................390 Generating ThreatIntelligence...................................................401 Conclusion..................................................................................420