Logging and log management: The authoritative guide to understanding the concepts surrounding logging and log management

698 Pages·2012·5.99 MB·English

by  Anton A. Chuvakin

Logging and Log Management
The Authoritative Guide to Understanding the
Concepts Surrounding Logging and Log
Management
Dr. Anton A. Chuvakin
Kevin J. Schmidt
Christopher Phillips
Technical Editor
Partricia Moulder
Table of Contents
Cover image
Title page
Copyright
Acknowledgments
Dr. Anton A. Chuvakin
Kevin J. Schmidt
Christopher Phillips
About the Authors
About the Technical Editor
Foreword
Preface
Intended Audience
Prerequisites
Organization of the Book
Chapter 5: Case Study: syslog-ng
Chapter 6: Covert logging
Chapter 7: Analysis Goals, Planning and Preparation: What Are
We Looking for?
Chapter 8: Simple Analysis Techniques
Chapter 9: Filtering, Matching and Correlation
Chapter 10: Statistical Analysis
Chapter 11: Log Data Mining
Chapter 12: Reporting and Summarization
Chapter 13: Visualizing Log Data
Chapter 14: Logging Laws and Logging Mistakes
Chapter 15: Tools for Log Analysis and Collection
Chapter 16: Log Management Procedures: Escalation, Response
Chapter 17: Attacks Against Logging Systems
Chapter 18: Logging for Programmers
Chapter 19: Logs and Compliance
Chapter 20: Planning Your Own Log Analysis System
Chapter 21: Cloud Logging
Chapter 22: Log Standard and Future Trends
Chapter 1. Logs, Trees, Forest: The Big Picture
Introduction
Log Data Basics
A Look at Things to Come
Logs Are Underrated
Logs Can Be Useful
People, Process, Technology
Security Information and Event Management (SIEM)
Summary
References
Chapter 2. What is a Log?
Introduction
Logs? What logs?
Criteria of Good Logging
Summary
References
Chapter 3. Log Data Sources
Introduction
Logging Sources
Log Source Classification
Summary
Chapter 4. Log Storage Technologies
Introduction
Log Retention Policy
Log Storage Formats
Database Storage of Log Data
Hadoop Log Storage
The Cloud and Hadoop
Log Data Retrieval and Archiving
Summary
References
Chapter 5. syslog-ng Case Study
Introduction
Obtaining syslog-ng
What Is syslog-ngsyslog-ng?
Example Deployment
Troubleshooting syslog-ng
Summary
References
Chapter 6. Covert Logging
Introduction
Complete Stealthy Log Setup
Logging in Honeypots
Covert Channels for Logging Brief
Summary
References
Chapter 7. Analysis Goals, Planning, and Preparation: What Are
We Looking for?
Introduction
Goals
Planning
Preparation
Summary
Chapter 8. Simple Analysis Techniques
Introduction
Line by Line: Road to Despair
Simple Log Viewers
Limitations of Manual Log Review
Responding to the Results of Analysis
Examples
Summary
References
Chapter 9. Filtering, Normalization, and Correlation
Introduction
Filtering
Normalization
Correlation
Common Patterns to Look For
The Future
Summary
Reference
Chapter 10. Statistical Analysis
Introduction
Frequency
Baseline
Machine Learning
Combining Statistical Analysis with Rules-based Correlation
Summary
References
Chapter 11. Log Data Mining
Introduction
Data Mining Intro
Log Mining Intro
Log Mining Requirements
What We Mine For?
Deeper into Interesting
Summary
References
Chapter 12. Reporting and Summarization
Introduction
Defining the Best Reports
Network Activity Reports
Resource Access Reports
Malware Activity Reports
Critical Errors and Failures Reports
Summary
Chapter 13. Visualizing Log Data
Introduction
Visual Correlation
Real-time Visualization
Treemaps
Log Data Constellations
Traditional Log Data Graphing
Summary
References
Chapter 14. Logging Laws and Logging Mistakes
Introduction
Logging Laws
Logging Mistakes
Summary
Reference
Chapter 15. Tools for Log Analysis and Collection
Introduction
Outsource, Build, or Buy
Basic Tools for Log Analysis
Utilities for Centralizing Log Information
Log Analysis Tools—Beyond the Basics
Commercial Vendors
Summary
References
Chapter 16. Log Management Procedures: Log Review, Response,
and Escalation
Introduction
Assumptions, Requirements, and Precautions
Common Roles and Responsibilities
PCI and Log Data
Logging Policy
Review, Response, and Escalation Procedures and Workflows
Validation of Log Review
Logbook—Evidence of Exception of Investigations
PCI Compliance Evidence Package
Management Reporting
Periodic Operational Tasks
Additional Resources
Summary
References
Chapter 17. Attacks Against Logging Systems
Introduction
Attacks
Summary
References