PRAISE FOR SECURITY METRICS “Throw out the security religion and make informed business decisions now!” —Mark Curphey ISBPM,Inc. “Connecting People,Process and Technology” “I’m very excited that Jaquith has written a text on metrics,and expect this will be the standard reference for years to come.” —Adam Shostack “Andrew devotes an innumerable amount oftime and effort to helping our profession out at SecurityMetrics.org.His book is wonderful,entertaining,and well thought-out.I found myselfnodding my head in agreement more than a few times.” —Alex Hutton CEO,Risk Management Insight “Andrew has written a book that most people who work in information protection and those who manage and work with them should read,not because it is particu- larly informative about information protection,but because it is highly informative about the challenges ofmeasuring protection programs effectively.While lots of books are out there about this or that aspect ofsecurity,from a security manage- ment standpoint,you cannot manage what you cannot measure,and Andrew puts his stake in the ground with this book about what you should measure and how to do it.” —Dr.Fred Cohen CEO,Fred Cohen & Associates http://all.net/ “To paraphrase Lord Kelvin’s famous quote,‘You cannot improve what you cannot measure.’Computer security has inhabited this sorry state for years,leaving too much room for snake oil,scare tactics,and plain old bull feathers.Andy’s book helps to remedy this problem by sending a strong clear message that metrics are both necessary and possible.Buy this strikingly well-written book today and help put an end to security nonsense.” —Gary McGraw,Ph.D. CTO,Cigital Author ofSoftware Security: Building Security In This page intentionally left blank Security Metrics This page intentionally left blank Security Metrics R F , U , D EPLACING EAR NCERTAINTY AND OUBT Andrew Jaquith Upper Saddle River,NJ • Boston• Indianapolis • San Francisco New York • Toronto •Montreal • London•Munich • Paris • Madrid Cape Town • Sydney • Tokyo • Singapore • Mexico City Many ofthe designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book,and the publisher was aware ofa trademark claim,the designations have been printed with initial capital letters or in all capitals. The author and publisher have taken care in the preparation ofthis book but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions.No liability is assumed for incidental or consequential damages in connection with or arising out ofthe use ofthe information or programs contained herein. The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales,which may include electronic versions and/or custom covers and content particular to your business,training goals,marketing focus,and branding interests.For more information,please contact: U.S.Corporate and Government Sales 800-382-3419 [email protected] For sales outside the United States,please contact: International Sales [email protected] Visit us on the Web:www.awprofessional.com Library ofCongress Cataloging-in-Publication Data: Jaquith,Andrew. Security metrics :replacing fear,uncertainty,and doubt / Andrew Jaquith. p.cm. Includes bibliographical references and index. ISBN 0-321-34998-9 (pbk.:alk.paper) 1. Risk management.2. Decision making. I.Title. HD61.J37 2007 658.4’7015195—dc22 2006103239 Copyright © 2007 Pearson Education,Inc. All rights reserved.Printed in the United States ofAmerica.This publication is protected by copyright,and permission must be obtained from the publisher prior to any prohibited reproduction,storage in a retrieval system,or transmission in any form or by any means,electronic,mechanical,photocopying,recording,or likewise.For information regarding permissions,write to: Pearson Education,Inc. Rights and Contracts Department One Lake Street Upper Saddle River,NJ 07458 Fax:(201) 236-3290 ISBN 0-32-134998-9 Text printed in the United States on recycled paper at RR Donnelley,Crawfordsville,Indiana First printing,March 2007 To dreamers and contrarians This page intentionally left blank Contents Foreword xv Preface xix Acknowledgments xxv About the Author xxviii Chapter 1 Introduction:Escaping the Hamster Wheel of Pain 1 Risk Management Is Where the Confusion Is 1 Metrics Supplant Risk Management 5 Summary 7 Chapter 2 Defining Security Metrics 9 Security Measurement Business Drivers 11 Roadblocks to Data Sharing 12 Modeling Security Metrics 13 Modelers Versus Measurers 13 Quality Assurance Literature 15 Public Health Terminology and Reporting Structure 16 Portfolio Management 17 Accelerated Failure Testing 17 Insurance 18 What Makes a Good Metric? 19 “Metric”Defined 21 Consistently Measured 23 Cheap to Gather 23 ix