CSO: Does this take hours? Days? Weeks?

JOHNSTON: It depends. If you're looking at a very complex security program, you may want to spend two or three weeks just kind of freewheeling. But you don't just sit around and do ideas. You generate nutty ideas, and then you go back to the program or the hardware and play around a little bit to see if those nutty ideas might have some merit. Then you get back together again, and you think of more nutty ideas based on what you learned. We're very much in favor of hands-on work, and not just thinking in abstractions. Toss the device around. Chat up the security guards. Kick the fence. Play with the system and try to understand how it behaves.

CSO: When the CSO tells his or her company about a vulnerability, we've seen that there can be a kind of "shoot the messenger" effect. (See "Don't Shoot the Messenger" from the August 2006 CSO.) What are ways they can avoid that or at least mitigate the effect?

JOHNSTON: We try to encourage people think about a vulnerability not as bad news. It's great news. When you find a vulnerability, you can do something about it.

CSO: But you still have to take people down the path of, something terrible could happen.

JOHNSTON: All our vulnerability assessment reports start out by pointing to the good things. There are <em>always</em> good things. Sometimes they're an accident, but by pointing them out, you get them recognized. Also, at the very beginning we always point out that we're going to find more vulnerabilities than they can possibly mitigate. We're going to make more suggestions for changes than you can possibly implement. That's OK. The bottom line is, vulnerability assessors are not here to tell you what changes to make. We're here to point out what we think are problems and what we think may be solutions. It's up to you to decide what you do with the findings.

This binary thinking about security--that something is either secure or not secure, or that we have to have all the vulnerabilities covered or we're not doing our job--is really nonsense. Security is a continuum, and there are always going to be vulnerabilities you can't do anything about. That doesn't mean anybody is screwing up. That's just the way security works.

CSO: In coming up with this laundry list of problems and possible solutions, is there oftentimes an 80/20 thing at play, where you can solve 80 percent of the problems with 20 percent of the solutions?