Product Description: <>The Definitive Guide to Quantifying, Classifying, and Measuring Enterprise IT Security Operations
Security Metrics is the first comprehensive best-practice guide to defining, creating, and utilizing security metrics in the enterprise.
Using sample charts, graphics, case studies, and war stories, Yankee Group Security Expert Andrew Jaquith demonstrates exactly how to establish effective metrics based on your organization’s unique requirements. You’ll discover how to quantify hard-to-measure security activities, compile and analyze all relevant data, identify strengths and weaknesses, set cost-effective priorities for improvement, and craft compelling messages for senior management.
Security Metrics successfully bridges management’s quantitative viewpoint with the nuts-and-bolts approach typically taken by security professionals. It brings together expert solutions drawn from Jaquith’s extensive consulting work in the software, aerospace, and financial services industries, including new metrics presented nowhere else. You’ll learn how to:
• Replace nonstop crisis response with a systematic approach to security improvement
• Understand the differences between “good” and “bad” metrics
• Measure coverage and control, vulnerability management, password quality, patch latency, benchmark scoring, and business-adjusted risk
• Quantify the effectiveness of security acquisition, implementation, and other program activities
• Organize, aggregate, and analyze your data to bring out key insights
• Use visualization to understand and communicate security issues more clearly
• Capture valuable data from firewalls and antivirus logs, third-party auditor reports, and other resources
• Implement balanced scorecards that present compact, holistic views of organizational security effectiveness
Whether you’re an engineer or consultant responsible for security and reporting to management–or an executive who needs better information for decision-making–Security Metrics is the resource you have been searching for.
Andrew Jaquith, program manager for Yankee Group’s Security Solutions and Services Decision Service, advises enterprise clients on prioritizing and managing security resources. He also helps security vendors develop product, service, and go-to-market strategies for reaching enterprise customers. He co-founded @stake, Inc., a security consulting pioneer acquired by Symantec Corporation in 2004. His application security and metrics research has been featured in CIO, CSO, InformationWeek, IEEE Security and Privacy, and The Economist.
Foreword
Preface
Acknowledgments
About the Author
Chapter 1 Introduction: Escaping the Hamster Wheel of Pain
Chapter 2 Defining Security Metrics
Chapter 3 Diagnosing Problems and Measuring Technical Security
Some gaps, but useful nonetheless Andrew Jaquith's book on security metrics is refreshing in its approach. Instead of a neverending cycle of risk assessments and vulnerability patching (a process which the author humorously calls the "hamster wheel of pain"), we are told to focus on core operational security processes and measurement of key indicators.
The central premise of the book is that a "risk management" approach, as promoted by many security vendors, doesn't work. The reason it doesn't work is that it is extremely difficult to get a good handle on the true value of assets, and an accurate estimate of risk. As the author puts it, "identifying problems is easy ... quantifying and valuing risk is much harder."
The thorough discussion of information security metrics makes this book worthwhile reading. However, there is a hint of sloppy thinking sprinkled throughout, which tends to undermine one's trust in the author's intellectual honesty. For example, when discussing the importance of tracking not only inbound viruses, but outbound as well, the author makes the following statement:
BEGIN QUOTE - Another twist I have added to the traditional antivirus statistics is a simple metric documenting the number of outbound viruses or spyware samples caught by the perimeter mail gateway's content filtering software. Why it matters is simple--it is an excellent indicator of how "clean" the internal network is. Organizations that practice good hygiene don't infect their neighbors and business partners. My friend Dan Geer relates this quote from the CSO of a Wall Street investment bank:
"Last year we stopped 70,000 inbound viruses, but I am prouder of having stopped 500 outbound."
In other words, the bank's internal network is cleaner than the outside environment by a factor of 140 to 1. - END QUOTE
Certainly, the conclusion in the last sentence cannot be supported without additional information. The volume of inbound email is likely to be drastically higher, which may account for the difference. The bank's outbound detection/prevention mechanism also may not be as efficient as the inbound.
Moreover, the metrics analysis chapter is very rudimentary and incomplete. Basic concepts like mean, median, and standard deviation are briefly discussed, but there is no mention of statistical random sampling techniques and confidence levels, which would surely be of significant importance when measuring key indicators across large populations, where a complete enumeration is either impossible, or too expensive and time-consuming. Sometimes, metrics which are "meaningful", are not the ones that are "tangible" and "easy to measure". A certain degree of statistical sophistication can be helpful in such situations.
In summary, the book offers some useful insight and practical advice for those who are charged with running an information security management program, but a healthy skepticism of the assumptions underlying the author's conclusions is warranted. In order to develop truly meaningful information security metrics, a much more sophisticated approach than what is described in this book will likely be needed.
Good resource for infosec professionals Nutshell review - This is a great book on security metrics. Practical, applicable, well written, well presented and will serve as an excellent resource for security professionals.
A necessary paradigm shift for information security Upon completion of this book, I began to muse: what percentage of security professionals have given any thought to security metrics? For those that have actually considered the topic, with what level of frequency do they entertain thoughts of security metrics? Yearly? Monthly? Daily? Gee, I think to myself, I'd like to see a time series analysis exhibit of that...
Based on the fact that I sit here torturing myself with these thoughts, I contend that Security Metrics has already influenced my approach toward security management. Indeed, Jaquith has done an excellent job of exposing an area that is critical to effective security management, but to which many security practitioners (myself included) have previously paid lip service. Security Metrics offers valuable insight to organizations seeking to provide a greater level of intelligence and meaning around their security program(s).
In addition to how well the ideas of the book resonated with my own professional and academic background, the choice to give a 5 star rating was based on its organization, readability, entertaining quips, and the fact that many of the alternative publications in the realm of security metrics are triple or more the cost of this one. Though I've not yet read or reviewed other similar works, the bar has been set high.
Every security professional (or wannabe) should read this book I'm not sure what I can write to sway you to buy or read the book if 5 star reviews from Ben Rothke and Richard Bejtlich don't sway you but I'll throw my likes and dislikes in here anyway. I'm not a "metrics guy" in fact, I'm still not , but I do think the book puts the concept of using them into perspective for the person that may not use any metrics in their security work.
I've been summing up the book to people at work by using the example (and I'll badly paraphrase) from the book of "if your spam gateway blocks 100,000 spam messages a day is that a good metric?" Initially you may say yes, that is a good metric. In fact most people at work said the same thing. But, as the author explains it is a poor metric. Better metrics are useful percentages like the percentage of missed spam or the percentage of false positives. Saying that 100,000 spam message are being stopped only tells us that you have a ton of spam on your network.
Some of the things I liked about the book were the author's discussions on how to make charts more readable and efficient at portraying information. I had to read the Tufte books in college and have to admit that I got more out of chapter 6 (visualization) than I feel I learned that whole semester of class. Chapter 2 discussing what makes good metrics was extremely useful, as well were chapters 3 & 4 because they gave good examples of metrics you can use to measure an organizations various defenses like perimeter security or application security. The discussion of using COBIT, ITIL and Security Frameworks in Chapter 4 was also good.
I only had two minor gripes. First was that toward the end of the book the author talks about colors of slides and charts which obviously doesn't do us any good since the book is in black and white and second, that he does use some big words throughout the book and I did find myself having to go back and reread things. Could he have put it into simpler terms, probably, but that doesn't make the book bad, just means I need to work on my vocab :-)
Overall it was a good entrance to the world of security metrics for me and took and away some of the perceived boredom of them. It definitely gave me some tools to look more critically at the numbers and stats that some of the vendors throw our way as well as how to deliver data and information in a more useful matter.
I liked it better than Cats! What a book. Seriously, I laughed, I cried. I shouted in frustration, only to be placated on the next page. I got a better understanding of what Andy has been banging on about with Security Metrics. And it helps me do my job better.
Anything Goes Trivia
Trevor
Megaton Man
