Getting Started With Security Metrics

A common question about security metrics is: “What are the top metrics that I should keep?” That’s usually asked in the context of someone who is thinking about getting started with a metrics program. And—spoiler alert—I’m not going to answer that question.

Metrics can make security relevant to the business

Here’s why: a metrics process is how you tie your security program to your business objectives. In other words, it is an expression of how well you understand what it is that you’re supposed to be doing. If you’ve ever heard security practitioners say that they wish they could talk to business executives in terms they understand – metrics are the necessary language. If you’re a security manager and you start asking “what metrics should I keep?” you’re really saying “I don’t understand how what my security organization does is relevant to the business.” Don’t do that.

Hopefully, you read the preceding paragraph and thought “that does not apply to me!” Well, then you’re in good shape because you do know how your security organization is relevant to the business. Next, all you have to do is start collecting information and presenting it in a way that shows that relevance. My preferred definition of a “metric” is “a data-set and a process for reducing it into a picture that tells a story.” The important concept is that of relevance: the story has to have a protagonist, a plot-arc, and a conclusion. If your reaction to what I’m saying is “I know how my security practice affects the business!” then your problem is simply to quantify that impact, along the contact-point where your organization actually does affect the business.

To optimize your security processes you have to measure them

In some organizations, that may mean preventing downtime of an e-commerce site by monitoring vulnerabilities, triaging them, and closing them before they are used in attacks. Let’s imagine that is the case. One metric you might derive from that scenario is tracking the exposure-time between identification of vulnerability and its closure; that’s a good one. Another might be to track the time after a vulnerability is closed and when you begin to see attacks attempting to exploit the vulnerability. That might serve as a measure of “number of attacks that would have succeeded if we hadn’t closed vulnerabilities.” Paired with a metric on “number of attacks that succeeded because we identified a vulnerability we didn’t fix” and you have a powerful story about the effectiveness of a patch management process. The first metric might be amplified with ‘waypoints’ – how long does it take for the security practice to learn about a vulnerability, triage it, and make a recommendation to the systems administration team, then how long does the vulnerability remain open before it is closed? When you are thinking about the waypoints in a process, you want to think in terms of the question “where do we spend our time?” because once you can answer that, you can begin to figure out where you can optimize your process.

Explain how what you do impacts the business and why

“Top metrics” tend to be just “speeds and feeds” – what I call ‘gee, wow” numbers. Think in terms of showing your relevance; look at what you do, and how much time you spend doing it, then ask yourself “why?” Because when you understand why you do something, then you can explain how what you do impacts the business and why, then you can track where and how you improve on doing it.

If you liked this blog from Marcus and are interested in more of his insights, read more Marcus Ranum blogs on the Tenable blog.