Security Metrics, A Beginner's Guide

Book written by Caroline Wong

Book review by Rick Howard

Bottom Line

Good niche book, I don't recommend this nonfiction book for the Cybersecurity Canon Hall of Fame, but if you are interested in the topic, this is a good one to read.

Review

There are some good parts here.

• I learned how to make a whisker plot.
• I re-learned what linear regression is and can now make one in a spreadsheet.
• The author demystified the logarithmic scale for me.

I also liked her convention of using headers like

• In My Humble Opinion
• In Actual Practice
• Tips
• Lingo
• Notes

But, there is a lot of fluff in here about how to run projects in a company that is just general purpose leadership and management skills. In a book about security metrics, she offers no examples of her own that are particularly useful to the security practitioner. Most importantly, she contradicts her own purpose by saying this about risk:

"As information security professionals, we are in the business of managing risk; specifically, if necessary, we must reduce the current level of risk in the information security area to align with the level of risk the company is willing to tolerate."

Yes - this is correct. But then she says this:

"How this alignment should be achieved is not always easy to identify quantitatively, but even very qualitative judgments with respect to risk reduction are useful during prioritization exercises. This may sound unusual in the context of a book about information security metrics, but not everything has to be about the numbers." 

No, this is just wrong. Metrics and analytics should be used by security practitioners to reduce our uncertainty about risk forecasting (See Howard and Abbas below). Qualitative judgements, like heat maps, are just bad science. There is not just one research study that says so, there are dozens (See Hubbard and Seiersen Below).

To be fair, Wong published her book in 2011, years before these other researchers (who are considered experts in the field), published theirs:

Howard / Abbas - 2013.

Freund / Jones - 2014.

Tetlock / Gardner - 2015.

Hubbard / Seiersen - 2016.

The truth is that none of the books, Wong's included, represent the complete picture. And even after reading all of them, I still struggle with the math to make forecasting risk easy. I was hoping Wong would provide that for me. She did not.

To use Wong's convention, in my humble opinion, this book is not a must read for most security practitioners. It's not even a must read for risk forecasters. But, if you are interested in the metrics and analytics, it's a good niche book.

Source

"Security Metrics, a Beginner's Guide," by Caroline Wong, Published by McGraw-Hill Companies, 20 October 2011.

References

‌"Book Review: How to Measure Anything in Cybersecurity Risk," by Steve Winterfeld, Cybersecurity Canon Project, Ohio State University.

"Book Review: Measuring and Managing Information Risk: A Fair Approach," by Ben Rothke, Cybersecurity Canon Project, Ohio State University.

"Book Review: Security Metrics: Replacing Fear, Uncertainty and Doubt" by Rick Howard, Cybersecurity Canon Project, Ohio State University.

"Foundations of Decision Analysis," by Ronald Howard and Ali Abbas, Published by Pearson, 1 March 2013.

"How to Measure Anything in Cybersecurity Risk," by Douglas Hubbard and Richard Seiersen, Published by Wiley, 25 April 2016.

"Measuring and Managing Information Risk: A Fair Approach," by Jack Freund and Jack Jones, Published by Butterworth-Heinemann, 1 January 2014.

“Security Metrics: A Beginner’s Guide’ Review,” by Ben Smith, Cybersecurity Canon Project, Ohio State University, 2021.

"Superforecasting: The Art and Science of Prediction," by Philip E. Tetlock and Dan Gardner, Published by Crown, 29 September 2015.

We modeled the Cybersecurity Canon after the Baseball Hall of Fame and the Rock & Roll Hall of Fame, except it’s a canon for cybersecurity books. We have more than 25 books on the initial candidate list, but we are soliciting help from the cybersecurity community to increase the number. Please write a review and nominate your favorite. 

The Cybersecurity Canon is a real thing for our community. We have designed it so that you can directly participate in the process. Please do so!