Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: The Open Group.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.
Today, we present a sponsored podcast discussion coming to you from The Open Group’s
Enterprise Architecture Practitioners Conference in Seattle on Feb. 2, 2010.We've assembled a panel to examine the need for IT security to run more like a data-driven science, rather than a mysterious art form. Rigorously applying data and metrics to security can dramatically improve IT results and reduce overall risk to the business.
By employing and applying more metrics and standards to security, the protection of IT becomes better, and the known threats can become evaluated uniformly. People can understand better what they are up against, perhaps in close to real-time. They can know what's working -- or is not working -- both inside and outside of their organization.
Standards like Information Security Management Maturity Model (ISM3) are helping to not only gain greater visibility, but also allowing IT leaders to scale security best practices repeatably and reliably.
We're here to determine the strategic imperatives for security metrics, and to discuss how to use them to change the outcomes in terms of IT’s value to the business.
Please join me in welcoming a security executive from The Open Group, as well as two experts on security who are presenting here at the Security Practitioners Conference. I want to welcome Jim Hietala, Vice President for Security at The Open Group. Hi, Jim.
Jim Hietala: Hi Dana.
Gardner: We are also here with Adam Shostack, co-author of The New School of Information Security. Welcome, Adam.
Adam Shostack: Hey, Dana. Great to be here.
Gardner: And also Vicente Aceituno, director of the ISM3 Consortium. Welcome.
Vicente Aceituno: Thank you very much.
Gardner: Now that we have got a sense of this need for better metrics and better visibility, I wonder if I could go to you, Jim. What is it to be a data-driven security organization, versus the alternative?
Hietala: In a sentence, it's using information to make decisions, as opposed to what vendors are pitching at you or your gut reaction. It's getting a little more scientific about gathering data on the kinds of attacks you're seeing and the kinds of threats that you face, and using that data to inform the decisions around the right set of controls to put in place to effectively secure the organization.
Gardner: Is it fair to say that organizations are largely not doing this now?
All over the map
Hietala: It's probably not a fair characterization to say that they're not. A presentation we had today from an analyst firm talked about people being all over the map. I wouldn’t say there's a lot of rigor and standardization around the kinds of data that’s being collected to inform decisions, but there is some of that work going on in very large organizations. There, you typically see a little more mature metrics program. In smaller organizations, not so much. It's a little all over the map.
Gardner: Perhaps it'
s time to standardize this a little bit?Hietala: We think so. We think there's a contribution to make from The Open Group, in terms of developing the ISM3 standard and getting it out there more widely.
Gardner: Adam, what, in your perception, is different now in terms of security than say two, three, or four years ago?
Shostack: The big change we've seen is that people have started to talk about the problems that
they are having, as a result of laws passed in California and elsewhere that require them to say, "We made a mistake with data that we hold about you," and to tell their customers.We've seen that a lot of the things we feared would happen haven't come to pass. We used to say that your company would go out of business and your customers would all flee. It's not happening that way. So, we're getting an opportunity today to share data in a way that’s never been possible before.
Gardner: Is it fair to say we are getting real about security?
Shostack: We've been real about security for a long time, but we have an opportunity to be a heck of a lot more effective than we have been. We can say, "This control that we all thought was a really good idea -- well, everyone is doing it, and it's not having the impact that we would like." So, we can reassess how we're getting real, where we're putting our dollars.
Gardner: Vicente, perhaps you could help us understand the application of metrics and data for security with external factors, and then internal. What's the difference?
Aceituno: Well, you can only use metrics to manage internal factors, because metrics are all about controlling what you do and being able to manage the outputs that you produce and that contribute value to the business.
I don’t think it brings a bigger return on investment (ROI) to collect metrics on external things that you can't control. It’s like hearing the news. What can you do about it? You're not the government or you're not directly involved. It's only the internal metrics that really make sense.
Gardner: From your perception, what needs to be a top priority in terms of this data-driven approach to security inside your own organization?
What you measure
Aceituno: The top priority should be to make sure that the things you measure are things that
are contributing positivity to the value that you're bringing to business as a information security management (ISM) practitioner. That’s the focus. Are you measuring things that are actually bringing value or are you measuring things that are fancy or look good?Gardner: We've heard "fit for purpose" applied to some other aspects of architecture and IT. How does this notion, being fit for purpose, apply to your security efforts?
Aceituno: Basically, we link business goals, business objectives, and security objectives in a way that’s never been done before, because we are painfully detailed when we express the outcomes that you are supposed to get from your ISM system. That will make it far easier for practitioners to actually measure the things that matter.
Gardner: We've been talking fairly generally about metrics and data. Jim, what do we really talk about? What are we defining here? Is this about taxonomy and categories, metadata, all the above -- or is there something a bit more defined that we're trying to measure?
Hietala: There's some taxonomy work to be done. One of the real issues in security is that when I say "threat," do other people have the same understanding? Risk management is rife with different terms that mean different things to different people. So getting a common taxonomy is something that makes sense.
The kinds of metrics we're collecting can be all over the map, but generally they're the things that
would guide the right kind of decision making within an IT security organization around the question, "Are we doing the right things?"Today, Vicente used an example of looking at vulnerabilities that are found in web applications. A critical metric was how long those vulnerabilities are out there before they get fixed by different lines of business, by different parts of the business, looking at how the organization is responding to that. We're trying to drive that metric toward the vulnerabilities being open for less time and getting fixed quicker.
Gardner: Adam, in your book, I believe you addressed some of these issues. How do look at metrics? How do you characterize them? I know it could go on for an hour about that, but at the high level ...
Shostack: At the high level, Vicente’s point about measuring the things you can control is critical. Oftentimes in security, we don’t like to admit that we've made mistakes and we conceal some of the issues that are happening. A metrics initiative gives you the opportunity to get out there and talk about what's going on, not in a finger pointing way, which has happened so often in the past, but in an objective and numerically centered way. That gives us opportunity to improve.
Gardner: I suppose this is a maturation of security. Is that fair to say that we're bringing this to where some other aspects of business may have been, in say manufacturing, 30, 40, or 50 years ago?
Learning from other disciplines
Shostack: I think that’s a fair statement. We're learning a lot from other fields. We're learning a lot from other disciplines. Elements of that are going to uncomfortable for some practitioners, and there are elements that will really enable practitioners to connect what they are doing to the business.
Gardner: The stakes here, I imagine, are quite high. This is about the trust you have with your partners, your customers, and the brand equity you have in your company. These are not small considerations.
Hietala: No, they're big considerations, and they do have a big effect on the business. Also, the important outputs of a good metrics program can be that it gives you a different way to talk to your senior management about the progress that you're making against the business objectives and security objectives.
That’s been an area of enormous disconnect. Security professionals have tended to talk about viruses, worms, relatively technical things, but haven't been able to show a trend to senior management that justifies the kind of spending they have been doing and the kind of spending they need to do in the future. Business language around some of that is needed in this area.
Gardner: I have to imagine, too, that if we formalize, structure, and standardize, we can make these repeatable. Then there's not that risk of personnel leaving and taking a lot of the tribal knowledge with them. Is that fair?
I can't think of anything better than for ISM3 to be managed from The Open Group from here on.
Hietala: That's fair as well. That's something that came out today in some of the discussions. Documenting the processes and what you're doing makes it easier to transition to new personnel and that kind of thing.
Gardner: Vicente, tell us a little bit about the ISM3 Consortium, its history, and what it is that you are principally involved with at this time.
Aceituno: The main task of the ISM3 Consortium so far was to manage the ISM3 standard. I'm very happy to say that The Open Group and ISM3 Consortium reached an agreement and, with this agreement, The Open Group will be managing ISM3 from here on in. We'll be devoting our time to other things, like teaching and consulting services in Spain, which is our main market. I can't think of anything better than for ISM3 to be managed from The Open Group.
Gardner: Adam, do you have a sense of this particular standard, the ISM3? Where do you see it fitting in?
Shostack: Actually, I don't have a great sense of where it fits in. There are a tremendous number of standards out there, and what I heard today I am very impressed by. I'm going to go read more about it, but it's not something I have a lot of operational exposure to that really lets me say, "This is where it's working for me."
Gardner: Jim, do you have a sense of where it fits in, and perhaps for those of our listeners who are not that familiar, can you give a quick tutorial?
Business value approach
Hietala: Sure. In terms of where I'd place it in the information security community, it adds a business value approach to information security, a metrics and maturity model approach that you had not necessarily had there with some of the other standards that are out there.
I'd also say that it's approachable from the standpoint that it's geared toward having different target maturity levels for different kinds of enterprises. That makes sense.
One of the things we talk about is that there's an 80-20 rule. You get 80 percent of the benefit from a subset of security controls. You can tailor ISM3 to the organization and get some benefit out of it, without setting the bar so high that it's unachievable for a mid-size or small business. That's the way I would characterize it.
Gardner: I think it's really important that these things are developed and brought into an organization at a practical level for those people who are in the trenches and are down there doing the work. Is there anything about this particular standard that you think is really not academic, but something quite effective in practice?
Hietala: Well, it spans the breadth of information security. You have metrics and control approaches in various areas and you can pick a starting point. You can come at this top-down, if you're trying to implement a big program. Or, you come at it bottoms-up and pick a niche, where you know you are not doing well and want to establish some rigor around what you are doing. You can do a smaller implementation and get some benefit out of it. It's approachable either way.
It was easier to communicate with other teams, and we had metrics to understand the results we were getting from making changes in the process.
Gardner: Adam, any thoughts about this issue of practicality when it comes to security, something that's more scientific and not perhaps a mysterious dark art of some kind?
Shostack: I really liked seeing the practical extracted. "Here are the things we're measuring. Here is why it matters to the business." That's what Vicente was talking about with regards to ISM3 through the day. Getting away from these very broad, hand wavy measures of risk or improvement, down to, "We are measuring this precise thing and this is why we need it to improve," is refreshing.
Gardner: Vincente, do you have any examples of organizations that have taken a lead on this and what sort of results have they been able to provide?
Aceituno: At this moment, the one organization that has implemented the ISM3 is Caja Madrid, which is the fourth biggest financial institution in Spain, and they had very impressive results. We found six times as many vulnerabilities. We were making more than twice as many ethical hacking tests. We could bring down the cost of unethical hacking by a big percentage, and we were getting more vulnerabilities fixed.
It was easier to communicate with other teams, and we had metrics to understand the results we were getting from making changes in the process. We have knowledge management that allows us to change the whole team of people and still carry on doing exactly the same thing in the same way that we were doing it.
I think that Caja Madrid is very happy and, actually, the director of security at Caja Madrid is very impressed with ISM3.
Gardner: Who typically are the folks who would be bringing this into an organization? I suppose there is some variability and the organizational landscape is still quite diverse, but is there a methodology in terms of how to bring this into an organization?
Works either way
Aceituno: It could work either way. Either you're a top-level manager, the CISO, or whatever, and you can think, "Okay, I want to do this" and you can implement a top-down implementation of the method.
Or, you can have no support from higher management and understand that you need to put in some rigor for management and you can think, "Okay, I'm going to organize my own work around this framework."
It can work either way, as Jim was saying before. You can implement it top down or bottom up and get benefit from it.
Gardner: Jim, this is a specific Open Group question. Does this work well inside of some other framework activity or architectural initiatives? Are there some other ITIL related activities? Does this have a brotherhood, if you will, in terms of standards and approaches that The Open Group's heritage is a bit more attuned to?
Hietala: I don't know that there's a direct statement you can make about how well this will work in an enterprise architecture framework or something like that. This is more about managing security objectives and operational things that you are going to do in a information security frame within an enterprise.
It's process-oriented. So, in terms of working well with other things, it works well with ITIL. Some of the early implementations have suggested that, but there is a good synergy there. I'll leave it there.
Gardner: Adam, any thoughts, from your perspective, on how this fits into some larger initiatives around security?
We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.
Shostack: We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.
The real success here and the real step with ISM3 is that it gives people a prescriptive way to get started on building those metrics.
You can pick it up and look at it and say, "Okay, I'm going to measure these things. I'm going to trend on them." And, I'm going to report on them."
As we get toward a place, where more people are talking about those things, we'll start to see an expectation that security is a little bit different. There is a risk environment that's very outside of people's control, but this gives people a way to get a handle on it.
Gardner: Vicente, it seems quite important, as a first step, to know where you are, in order to know how you've progressed. This seems to be an essential ingredient to being able to ascertain your risks over time.
Aceituno: The very first step, when it comes to the usual implementing, is to understand the needs and the goals of the business and the obligations of the business, because that's what drives the whole design of the ISM system There is no need to align security goals and business goals, because there are no goals outside of business goals. You have to serve the business first.
Gardner: There really isn't much difference between the goals of security and the general goals of the business. They are inexorably tied.
Aceituno: Yes, of course, they are.
Gardner: We've been learning more about security, some new metrics, and the ability to tie this into business outcomes. I want to thank our panel. We've been talking to Jim Hietala, Vice President for Security at the Open Group. Thank you, Jim.
Hietala: Thank you, Dana.
Gardner: Adam Shostack, co-author of the book, The New School of Information Security. Thank you.
Shostack: Thank you.
Gardner: And, also Vicente Aceituno, who is the Director of the ISM3 Consortium. Thank you.
Aceituno: Thanks so much.
Gardner: We are coming to you from The Open Group Security Practitioners Conference in Seattle, the week of Feb. 1, 2010.
This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for listening to this BriefingsDirect podcast, and come back next time.
Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: The Open Group.
Transcript of a sponsored BriefingsDirect podcast on ISM3 and security standards recorded live at The Open Group’s Enterprise Architecture Practitioners Conference in Seattle. Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.
You may also be interested in:
