Foundations of Mac OS X Leopard Security

487 Pages·2008·35.287 MB·English

by  Barker, William, Edge, Charles S., Smith, Zack

cyan yelloW
MaGenTa Black
panTone 123 c
Books for professionals By professionals® The eXperT’s Voice® in Mac os X
Companion
Charles S. Edge, Jr., eBook Available
author of Foundations of Mac OS X Leopard Security
Mac Tiger Server
Foundations of
Little Black Book Dear Reader,
M
As instructors, course authors, systems administrators, and consultants for Mac
networks big and small, we have run into hundreds of security problems at our
clients and discussed them with our students for years. We have watched the a Foundations of
perception of OS X security progress from being considered hogwash that only
c
snake-oil salesmen would sell to something that is a legitimate concern that we
all must consider. We have watched exploits and new vulnerabilities come out Mac OS X
and even discovered some of our own. And now we want to share what we have O
learned over the years with you.
By default, the Mac is a pretty darn secure computer. But many of the things S
you do to the computer after you turn it on for the first time can increase or

decrease how secure it is. This book is about security from the perspective of
X
a systems administrator, or a Mac user, once you turn on that computer. For
some, this means securing your personal machine, your home network, or Leopard Security
your small-business network from attacks. For others, it means securing your L
enterprise from corporate miscreants. Whatever environment you are pro-
e
tecting, the principals are the same: provide the least amount of access that
is required while maintaining a satisfactory measure of usability. Through o
detailed descriptions, step-by-step instructions, and command-line examples,
p
we present best practices for the home user and the enterprise security architect.
Some of the examples and walk-throughs in this book come from our work in
a
the field, perfecting hundreds of such procedures over the years. Some of the
examples, though, are new, written just for this book, based on our feedback r
from the community. d
Once you are finished reading this book, you will have a clearer understanding

about the challenges that you will face as the person responsible for maintaining
S Mac OS X client and server security,
the network.
from the home to the enterprise
We hope you will find that this book helps you solve those everyday security
e
challenges and helps give you a new level of understanding about security and
the Mac. c
Charles Edge, William Barker, and Zack Smith u
Companion eBook
r
i
ES t
L
t y
ti
See last page for details Ed
on $10 eBook version at
L
E
R

E Charles S. Edge, Jr.,
ISBN-13: 978-1-59059-989-1 Bd
S g
www.apress.com ISBN-10: 1-59059-989-6 mae
53999 ithrke, Jr William Barker, and Zack Smith
US $39.99 r,.,
Shelve in
Mac
User level:
9 781590 599891
Beginner–Intermediate
this print for content only—size & color not accurate spine = 0.9237" 488 page count

Edge_Barker_9896FRONT.fm Page i Tuesday, April 1, 2008 9:47 AM
Foundations of Mac OS X
Leopard Security
■ ■ ■
Charles S. Edge, Jr., William Barker,
and Zack Smith
Edge_Barker_9896FRONT.fm Page ii Tuesday, April 1, 2008 9:47 AM
FOUNDATIONS OF MAC OS X LEOPARD SECURITY
Copyright © 2008 by Charles S. Edge, Jr., William Barker
All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means,
electronic or mechanical, including photocopying, recording, or by any information storage or retrieval
system, without the prior written permission of the copyright owner and the publisher.
ISBN-13 (pbk): 978-1-59059-989-1
ISBN-10 (pbk): 1-59059-989-6
ISBN-13 (electronic): 978-1-4302-0646-0
ISBN-10 (electronic): 1-4302-0646-2
Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1
Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence
of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark
owner, with no intention of infringement of the trademark.
Lead Editor: Jeffrey Pepper
Technical Reviewers: Mike Lee, Frank Pohlmann
Editorial Board: Clay Andres, Steve Anglin, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan
Gennick, Matthew Moodie, Joseph Ottinger, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke,
Dominic Shakeshaft, Matt Wade, Tom Welsh
Project Manager: Candace English
Copy Editor: Kim Wimpsett
Associate Production Director: Kari Brooks-Copony
Senior Production Editor: Laura Cheu
Compositor: Susan Glinert Stevens
Proofreader: Nancy Bell
Indexer: Julie Grady
Cover Designer: Kurt Krames
Manufacturing Director: Tom Debolski
Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor,
New York, NY 10013. Phone 1-800-SPRINGER, fax [phone], e-mail [email protected], or
visit http://www.springeronline.com.
For information on translations, please contact Apress directly at 2855 Telegraph Avenue, Suite 600,
Berkeley, CA 94705. Phone [phone], fax [phone], e-mail [email protected], or visit http://
www.apress.com.
Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use.
eBook versions and licenses are also available for most titles. For more information, reference our Special
Bulk Sales–eBook Licensing web page at http://www.apress.com/info/bulksales.
The information in this book is distributed on an “as is” basis, without warranty. Although every precaution
has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to
any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly
by the information contained in this work.
Edge_Barker_9896FRONT.fm Page iii Tuesday, April 1, 2008 9:47 AM
This book is dedicated to my loving wife, Lisa.
—Charles
To my family and friends, who incessantly inspire me to follow my dreams.
—William
Edge_Barker_9896FRONT.fm Page iv Tuesday, April 1, 2008 9:47 AM
Contents at a Glance
About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xix
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxi
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxiii
PART 1 The Big Picture
■ ■ ■
■ CHAPTER 1 Security Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
■ CHAPTER 2 Security Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
■ CHAPTER 3 Securing User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
PART 2 Security Essentials
■ ■ ■
■ CHAPTER 4 Malware Security: Combating Viruses, Worms, and Root Kits . . . 75
■ CHAPTER 5 Securing Web Browsers and E-mail . . . . . . . . . . . . . . . . . . . . . . . . . . 93
■ CHAPTER 6 Reviewing Logs and Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
PART 3 Network Security
■ ■ ■
■ CHAPTER 7 Securing Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
■ CHAPTER 8 Setting Up the Mac OS X Firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
■ CHAPTER 9 Securing a Wireless Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
PART 4 Sharing
■ ■ ■
■ CHAPTER 10 File Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
■ CHAPTER 11 Web Site Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
■ CHAPTER 12 Remote Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
■ CHAPTER 13 Server Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
iv
Edge_Barker_9896FRONT.fm Page v Tuesday, April 1, 2008 9:47 AM
PART 5 Workplace Security
■ ■ ■
■ CHAPTER 14 Network Scanning, Intrusion Detection, and
Intrusion Prevention Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
■ CHAPTER 15 Backup and Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
■ CHAPTER 16 Forensics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
■ APPENDIX A Xsan Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
■ APPENDIX B Acceptable Use Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
■ APPENDIX C Secure Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
■ APPENDIX D Introduction to Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
■ INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
v
Edge_Barker_9896FRONT.fm Page vi Tuesday, April 1, 2008 9:47 AM
Edge_Barker_9896FRONT.fm Page vii Tuesday, April 1, 2008 9:47 AM
Contents
About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xix
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxi
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxiii
PART 1 The Big Picture
■ ■ ■
■ CHAPTER 1 Security Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Securing the Mac OS X Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Customizing System Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Accounts Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Login Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Security Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
FileVault. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Infrared Controls in Security Preferences . . . . . . . . . . . . . . . . . . . . . 12
Other System Preferences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Software Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Bluetooth Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Printer Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Sharing Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Securely Erasing Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Using the Secure Empty Trash Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Using Encrypted Disk Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Securing Your Keychains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
■ CHAPTER 2 Security Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
What Can Be Targeted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
The Accidental Mac Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Kinds of Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
OS 9 vs. OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Darwin vs. Aqua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
vii
Edge_Barker_9896FRONT.fm Page viii Tuesday, April 1, 2008 9:47 AM
viii ■ CONTENTS
Unix Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
In the Beginning…the Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Physical Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Equipment Disposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Physical Devices and Optical Media. . . . . . . . . . . . . . . . . . . . . . . . . . 38
Firmware and Firmware Password Protection . . . . . . . . . . . . . . . . . . . . . 38
Open Firmware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
EFI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Firmware Protection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Multifactor Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Keeping Current: The Cat-and-Mouse Game . . . . . . . . . . . . . . . . . . . . . . 45
The NSA and the Mac . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
A Word About Parallels and Boot Camp . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
■ CHAPTER 3 Securing User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Introducing Authentication, Authorization, and Identification . . . . . . . . . 47
Managing User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Introducing the Account Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Adding Users to Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Enabling the Superuser Account. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Setting Up Parental Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Managing the Rules Put in Place . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Restricting Access with the Command Line: sudoers . . . . . . . . . . . . . . . 62
Securing Mount Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
SUID Applications: Getting into the Nitty-Gritty . . . . . . . . . . . . . . . . . . . . . 69
Creating Files with Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
PART 2 Security Essentials
■ ■ ■
■ CHAPTER 4 Malware Security: Combating Viruses,
Worms, and Root Kits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Classifying Threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
The Real Threat of Malware on the Mac . . . . . . . . . . . . . . . . . . . . . . 77
Script Virus Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Socially Engineered Viruses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79