SOA Demands Broader Skills and Experience for Enterprise Architects Across Technical and Political Domains, Says Open Group Panel

Transcript of BriefingsDirect podcast recorded at the 19th Annual Open Group Enterprise Architect's Practitioners Conference on July 22, 2008, in Chicago.

Listen to the podcast. Listen on iTunes/iPod. Sponsor: The Open Group.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today, a sponsored podcast discussion coming to you from a live panel at the 19th Annual Open Group Enterprise Architect's Practitioners Conference on July 22, 2008, in Chicago. We're going to discuss the role and impact of skills and experience for Enterprise Architects (EAs) in the context of services-oriented architecture (SOA).

We're going to talk about both the public sector, that is to say, government organizations, as well as the private sector, because these issues cut across all areas of information technology (IT).

First, let me introduce our panel of experts and guests. We are joined by Tony Baer, a senior analyst at Ovum. We're also joined by Eric Knorr, editor-in-chief of InfoWorld, as well as Joe McKendrick, author, consultant, prolific blogger, and IT industry analyst. Andras Szakal, the chief architect at IBM's Federal Software Group is here. And lastly, David Cotterill joins us. He’s head of innovation at the U.K. Government Department for Work and Pensions, which is generally equivalent to the Social Security Administration here in the United States.

We've heard a lot at this conference about the role of enterprise architects, how it's swiftly evolving. There is a need for alignment between business and IT at multiple levels. There is also concern about security and managing complexity as organizations move toward SOA methods and principles.

But when I talk to developers, architects and operations people -- and when we get down to what makes them successful -- they often come back to hiring. They talk about the people that they need to bring into their organizations to succeed, and the correct way to promote and encourage those people within their organizations.

A lot of that, of course, is important to The Open Group and The Open Group Architecture Framework (TOGAF), as well as its certification processes, IT Architect Certification (ITAC). It seems important for us to discuss why certification and frameworks need to take into account the full architect, the full person, not only in terms of their knowledge -- their book knowledge, their a priori knowledge -- but also their experience and skills.

We're asking so much of these people, in the context of politics, collaboration, and transformation. My first question then goes to Andras at IBM. Are the skill sets for EA and for SOA the same, or are they significantly different?

Andras Szakal: That's a good question. I believe that they are significantly different. Obviously, an enterprise architect needs to have a background in an EA method. They have to be able to apply the EA method. There are a whole set of governance requirements that an enterprise architect is involved with that an SOA practitioner may not be involved with.

There is quite a bit of intersection between the SOA architect and the implementation of the business processes that they are responsible for. So that's the intersection between EA and SOA. Certainly, they're not equivalent -- even if you believe some of the media out there.

Some of the SOA advertisements suggest that maybe SOA is equivalent to EA, but it's not. For SOA you need to have a background in understanding how to transform the business processes into actual implementation, versus enterprise architects who are following the EA methods and following the practices for producing the deliverables that are handed over to the EA practitioner.

Gardner: Is there a general rule of thumb about the balance? Is this an 80-20 equation, 60-40, 50-50? From your experience in the government organizations, what's the proper balance, when you're hiring an architect, between being technically savvy, and the other skills around organization and management?

Szakal: Within the government, EA is, I would say, trending more toward the political aspect, to the executive-level practitioner, than it is, say, an integrator who has hired a set of well-trained SOA practitioners.

The enterprise architect is really more focused on trying to bring the organization together around a business strategy or mission. And, certainly, they understand the tooling and how to translate the mission, vision, and strategy into an SOA architecture -- at least in the government. If you look at folks like some of the past enterprise architects and chief architects, like Dick Burk [former Chief Architect and Manager, Federal Enterprise Architecture Program Office of E-Government and Information Technology Office of Management and Budget, Executive Office of the U.S. President] … I would say he was more of an administrator than a practitioner.

If you look at the SOA practitioners who come from the systems integrators, who are the majority of the individuals who implement systems in the U.S. federal government's base, they are focused more on IT and the implementation.

Gardner: Good, thanks. David Cotterill, in the U.K. you are a very large and substantial agency. How do you view this balancing act between the skills and knowledge? Is this about book skills? Is it about experience?

Of course, we are dealing with the most variable of variables -- people. How are you seeing the breakdown and where do you see the most important skill sets for “new age,” if you will, enterprise architects?

David Cotterill: Well, I think the technical background can be taken as a given for an enterprise architect. We expect that they have a certain level of depth and breadth about them, in terms of the broadest kind of technology platforms. But what we are really looking for are people who can then move into the business space, who have a lot more of the softer skills, things like influencing … How do you build and maintain a relationship with a senior business executive?

Those are kind of the skills sets that we're looking for, and they are hard to find. Typically, you find them in consultants who charge you a lot of money. But they're also skills that can be learned -- provided you have a certain level of experience.

We try to find people who have a good solid technical background and who show the aptitude for being able to develop those softer skills. Then, we place them in an environment and give them the challenge to actually develop those skills and maintain those relationships with the business. When it works, it's a really great thing, because they can become the trusted partner of our business people and unmask all the difficulties of the IT that lies behind.

We also have a more applications-focused space, in terms of a delivery-focused space. We need people who have a greater, more technical understanding of what the IT would be from our perspective, so that they can challenge the integrators and the suppliers -- just to make sure that they are doing the right thing, that we're keeping as open and flexible as we would like to be, and so that we're not tied into any given supplier.

Gardner: A lot of these roles are new and they're unexplored. We are into new territory. What do you look for in terms of the right kind of experience, the right mix of experience, for the people that you want to bring into these roles?

Cotterill: We hire a lot of people from financial services, because working with an organization the size of ours, you have to not be fazed by the scale of what you are trying to do. So, it helps having worked in a large organization, and understanding the complexities of scale. Typically, we're looking for people who have presented at a board level before, so they have demonstrated the ability to engage with board-level executives. That's really what we're trying to get.

Gardner: Let's go to some of our analysts. Tony Baer, you've been covering SOA for some time. We often hear a lot of prescriptives and definitions, methodologies, reference implementations and so forth -- and not a lot of attention is given to this dynamic management at the human level. Do the software vendors need to come out with a different messaging approach to help these organizations as they try to transform themselves, not just in terms of product, but in terms of this business-IT alignment?

Tony Baer: The short answer is “yes,” but the long answer is that vendors are having an awfully hard time trying to bridge the gap. I'll just give an example of IBM. I'm not trying to single out IBM here. IBM has probably been one of the companies that been most honest about this.

Just to give an idea in terms of trying to bridge different silos, I was talking with Danny Sabbah at IBM’s Rational division, and one of the interesting results of IBM's acquisition of Rational was that they have tried to cross-purpose some of the assets from Rational, where it could apply to some of the other product groups.

The obvious place is in the area of concern over how your systems operate at runtime. So they applied elements of the Rational Unified Process to Tivoli, but they also found at the same time that each brand was used to marketing to different entry points within the organization. They had a hard time trying to bridge that, because there was a mutual distrust.

As I said, I'm not singling out IBM here. I think IBM has been more ambitious in trying to tackle this problem, more honest about it. The same thing here applies with SOA and EA, and with trying to define and hopefully raise the consciousness within the EA profession that you need to have more of a business-level sensibility.

So the short of it is, the answer is yes. The long of it is, it's going to be awfully difficult to get there.

Gardner: Well, let's go to IBM for a second and give them a chance. I'm sure that within their own organization they are also dealing with, "Which came first, the chicken or the egg," except it's professional services or software, and you have mentioned, Andras, that you have this discussion. It's a tough problem.

Szakal: When we are engaging with the customer, we try to have one face. It depends on the business problem that they are solving. So, do you have a discussion about how you implement SOA, or do you first kind of try to roll back the discussion to the point where you can say, "What is the business problem?" and then apply some kind of framework and methodology to mete that out effectively, and then begin to map on the technologies that are necessary to be there.

That's the balance my group tries to play. We are software architects, but we are really trying to solve the business problem. We do this through a framework that we call the SOA Entry Points. We call this people, process, information, connectivity, and reuse. We apply a framework we now call the Smart SOA Approach, which allows you to try to define where you are on the continuum of maturity, and how you apply that with respect to the needs of the business and the business function that you are implementing. An organization that is essentially immature will want to begin to implement SOA infrastructure before they begin to integrate business processes or dynamically adopt SOA at very higher maturity rate.

Gardner: I would also like to bounce a question off of you, Andras. When you're hiring, what do you look for in the requirements? Is there any surprise for this audience about what you look for on the resume of the people that you like to bring into be architects for you at IBM?

Szakal: That's a good question, because I do hire quite a few architects. I would look for people who have deep technical skills, and have had experience in implementing systems successfully. I have seen plenty of database administrators come to me and try to call themselves an architect, and you can't be one-dimensional on the information side, although I am an information bigot too.

So you're looking for a broad experience and somebody who has methodology background in design, but also in enterprise architecture. In that way, they can go from understanding the role of the enterprise architect, and how you take the business problem and slice that out into business processes, and then map the technology layer on to it.

Gardner: Are there any things that you might see on a resume that would disqualify someone in your mind from this role of modern-day enterprise architect focusing more on SOA?

Szakal: I think there are few that would disqualify. You have to be very articulate, both written and verbally. Obviously, you cannot work with people if you have no people skills. You have to have a broad background in technology, both hardware and software. You have to understand the value of EA and business-process choreography. So if you are simply a database administrator and you are very focused on design, that's not really architecture.

If you are simply one-dimensional, the architecture that you are applying is the architecture that you know, and you are not really acting as an “architect.” If you only implemented an IBM solution, a DB2 solution, or say a Microsoft .NET solution, are you really an architect -- or are you just somebody who is a very good specialist at implementing .NET or an information management solution?

Gardner: Joe McKendrick, it sounds like we are defining a newer super-human role here for someone. Not too stuck in one aspect of their IT experience, but not overly technical either. They need to be a politician, a chef, a gardener, and a house painter. It sounds like a difficult thing for any one person to step up to. Is this all feasible? Is this logical? Should we certify individuals, or is this really something that probably requires more of a team?

Joe McKendrick: Thanks, Dana. It's a great point, and I want to take it back it to your previous question about how the vendors are addressing this. IBM is a kind of an exception to the rule here, but what we are seeing in the industry is that the vendors are talking very actively about business transformation, which is a very high-level activity. Ultimately, the goal of SOA is a fairly major transformation of the business to achieve more agility and more flexibility. However, most of the vendor community targets the IT department from the CIO on down.

The big question is whether the vendor community, at this stage in the game, may be asking a little too much of IT departments. Do IT personnel, IT managers, really have the bandwidth or the training, the wherewithal, or the organizational support to go out to the rest of the organization, and say, "Okay, folks, we are going to transform you now." I think our leading IT vendors, in particular, are leaning a little bit too heavily on their IT department, where SOA is a community effort.

In fact, the analogy I'd like to think of in terms of SOA is a condominium association. If you look at a condo association, you don't have one single owner. You have multiple owners. No one really has complete ownership of the building or the community. What happens is that each owner turns over management of the community to this governance group, the condo management association. IT is one of those owners. The condo turns shared services, be it gardening, landscaping, trash pick up and so forth to the management association.

Gardner: Well, I have to say, I hope that the IT department work a little better than some of the condo associations I have been affiliated with. That brings up another issue, and so let's go to Eric Knorr from InfoWorld. This sounds like balancing on so many different levels … group versus individual, command-and-control versus collaboration.

It reminds me of what's taking place in development on the design-time side of things around, Agile and Scrum. I know that Agile and Scrum might not be germane to an architect, but they have been designed to deal with teams and complexity and managing many moving parts by creating more of a team approach with a leader, sometimes called a master, and lots of iterative-stage meetings. Does this make sense, talking about not so much an individual or a team, but really a new way to manage complexity and innovation in a fast-paced environment?

Eric Knorr: Within the context of SOA, one thing we might not have touched on yet is that often, at least in the most successful examples that we have seen, there has to be some sort of visionary leadership. In case study after case study, you run into a chief architect, or even a chief technology officer sometimes, who has really made that connection, in an SOA context, between not only looking at the business processes, but breaking them down into business services and figuring out how to map a technical infrastructure against that.

That leadership is so important, because SOA is such an elusive concept that it's very easy to fall back into the old habits of enterprise application integration (EAI), and thinking in terms of point-to-point integration and not thinking in terms about the last presentation, that strategic value.

At the same time, SOA doesn't happen by decree. It happens with a feedback loop from the bottom up. The demands on the leadership skills are really very high. In organizing these teams that you are talking about here, a board of review is an essential part of that. Maybe you start your governance with an interoperability framework, so everybody knows what protocols are being used.

Then, you get deeper into the design-time governance rules, and you don't want to get to a point where you have such a granular level of rules that your best developers feel like they are being strangled … "Oh, here come the governance cops." So, you need to have that sort of reality feedback, and I think that takes a high level of managerial skill to pull off and still keep everybody on the same track, because if you don't exert enough control, you are going to have people building redundant services again. It's a real balancing act.

Gardner: I suppose it often happens, both in development and in IT operations, that dictates will come down, methodologies will be established, lists will be made, matrices will be presented, people will nod their heads, and they'll go off and do whatever they think it takes to get the job done, which gives us a little bit of a problem in terms of maintaining security, and maintaining the expectations and requirements to the business side of the house. Let's go back to David Cotterill. In the real world, in your organization, do you find that people often ignore what becomes the supposed standards of operating procedure?

Cotterill: Not that often, actually, because in the U.K. we've had some recent examples of where people have ignored operating procedures around security, and the message is pretty clear now that we've put in a lot of governance and compliance procedures to try and remove that level of, "I am just going to ignore and do what I feel is right. I'll publish something which I feel is the right thing to do."

We know that the impacts of that are not IT impacts, but they affect the business, the organization, and they affect ministers. Our friends in the press like to get hold of those things. So, we are very, very sensitive to not allowing that kind of “skunkworks” kind of thing to see the light of day.

The challenge, therefore is -- and this goes back to think to the point about visionary leadership -- how do you establish the right governance and platforms that allow people to be innovative in terms of the solutions that they bring forward, but also has that right level of control that says, "Okay, you are not going to publish something which opens up the entire department's customer information to attack." It's a real balancing act, and that's where leadership comes in I think.

Gardner: I am assuming that the Information Technology Infrastructure Library (ITIL) plays a significant role in your organization?

Cotterill: Absolutely, in terms of the operations management.

Gardner: And with ITIL, this moves you toward the shared services approach of IT management, the understanding that the users in your business at large or, in your case, the government department, are the real customers. Perhaps market forces come into play. That is to say, supply and demand. You don't get paid if the bids don't come in. The bids don't come in if the experience isn't good and rewarding over time for the end users. Do you think that whenever we deal with complexity, on the level of we are talking about, that supply and demand, letting the free forces of competition come into play -- does that help?

Cotterill: I think it helps. Government traditionally has a very command-and-control approach to innovation, and that stifles innovation effectively. I know it's a most effective method of stifling innovation. So when working with suppliers what we are always trying to do is introduce that level of competition. It's not necessarily just a vendor-customer relationship that we are talking about, because we are dealing with services that affect citizens and public information that should be available to citizens. We actually have to look about how the citizens want to use the information that government holds.

As an example, in the U.K. recently the government just launched a competition called "Show Us A Better Way," which is open to the public to identify uses of U.K. government data. How would they like to see U.K. government data mashed up to present new innovative solutions for the public? That's kind of an interesting thing, which takes us out that traditional supplier customer model.

Gardner: Okay, I'd like to check one of my own premises, and that was that SOA and enterprise architecture is common and similar between large enterprises and the public sector. Let's go to Andras at IBM. Do you find that to be the case from your experience that the way that you are dealing with your federal and government clientele gives you certain advantages or differentiates you from what is going on in the private sector?

Szakal: Obviously, within the U.S. federal government space, EA was adopted as an industry, before many other industries using failure modes and effects analysis (FMEA) and the Department of Defense Architecture Framework (DoDAF). IBM is very focused on trying to create a normative model for enterprise architecture.

We started off doing that with DoDAF in the Object Management Group (OMG), and now we've kind of settled on Unified Modeling Language (UML), the next innovative version of that standard which establishes a normative metamodel. You can actually flow artifacts from the EA into the actual IT architecture and design layer, so that there is seamless integration. We can have tool-based round tripping, so that we can actually have a dynamic asset management repository that is pulled in dynamically at the lowest levels into the EA layer that we can then use as part of our normative models.

Then we can push that down into the design layer, and it all connects and makes sense. We can make sure that there is a normalization of the artifacts and not just pretty pictures at the A level being handed over to an IT architect, who then has a process of going through and making their own determinations about the meaning of that picture. Those are some of the challenges that we face as a community.

Gardner: Well, I think clearly what we've heard form our panelists aligns well with many of the discussions today -- how transparency and breaking out of silos is important, not just for technology, but for how people behave in, and operate, and in understanding the business goals, and communicating them appropriately.

Even as many moving parts become essential, it is indeed a talented person who can cross the boundaries of the technology issues, and foster this collaboration clearly by an evangelism, across not only technology, but the business domains. I certainly congratulate those of you who are doing that, and hope that we can find a lot more folks in the field that are willing to step up and take on such a large responsibility.

I'd like to thank our panel of guests as we close today. We've been joined by Tony Baer, senior analyst at Ovum; Eric Knorr, editor-in-chief at InfoWorld; Joe McKendrick author and IT analyst and blogger; Andras Szakal, chief architect, IBM's Federal Software Group; and David Cotterill, head of innovation -- a good title, if you can manage to get it -- at the U.K. Government Department for Work and Pensions.

This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. I'd like to thank Allen Brown and The Open Group for helping put this together. Thanks, and have a good day.

Listen to the podcast. Listen on iTunes/iPod. Sponsor: The Open Group.

Transcript of BriefingsDirect podcast recorded at the 19th Annual Open Group Enterprise Architect's Practitioners Conference on July 22, 2008, in Chicago.

SOA Insights Analysts Discuss Likely Future of SOA at Open Group Conference

Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition podcast, recorded July 23, 2007.

Listen to the podcast here. To learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Dana Gardner: Hello, and welcome to a special presentation of BriefingsDirect SOA Insights Edition, an ongoing series of podcast discussions on Services Oriented Architecture (SOA)-related news and events with a panel of industry analysts and guests. I'm your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions.

We are recording this special podcast on "The Future of SOA" before a live audience at the July 23^rd, 2007 plenary session of the Open Group’s Enterprise Architecture Practitioners Conference in Austin, Texas. Welcome to you all.

Our panel today consists of Eric Knorr, executive editor-at-large at InfoWorld. Also, Tony Baer, principal at onStrategies; Todd Biske, principal architect at MomentumSI, based here in Austin, and, Beth Gold-Bernstein, vice president of ebizQ Learning Center. Welcome to the panel.

Our topic today is “The Future of SOA.” We want to take a look at what might happen if a lot of the things we’ve been considering for SOA actually happen.

One of the interesting things we heard this morning from Dave Linthicum was that he's anticipating that, in as few as five years, the role of enterprise architect and the role of SOA architect will meld.

I thought that was a little ambitious, and I wonder if I could put that to our panel. Before we get into the future of SOA, we should actually determine when the future of SOA will be important.

Beth, what do you think about five years? Do you think we are going to see SOA actually become part and parcel of all enterprise architecture activities?

Beth Gold-Bernstein: Five years is definitely ambitious. All the polls that we’ve done at ebizQ have shown that the market is really in the early stages of adoption. From my point of view, the sooner the SOA architect’s role is rolled into enterprise architecture in terms of governance the better. It is a subpart. And it is, as Dave said, best practice in architecture. We’ve known that for a couple of decades, actually.

We are getting there. Standards have gotten in. But there is more to that than just architecture. It fundamentally changes the way we create applications. That means developers need to change the way they are architecting applications, and that’s very different. It's going to take quite a while until we build up the different levels of services.

Gardner: I’ve always thought that SOA was a journey, in which you never actually get to the destination. Todd, what do you think? Is this an ongoing journey, or is there a future point for SOA when we can determine, “We’ve made it; it’s working”?

Todd Biske: I definitely agree that it's a journey. One of the things that I have seen in my work as a consultant, and prior to that as a practicing enterprise architect in a major enterprise, is that you're always going to have these new initiatives come along that may start out from a point solution.

Just as was said this morning, if solves a small percentage of my problem, if you're only looking at a narrow scope, and you’ve got boundaries around it, you are not seeing the whole enterprise.

So, in enterprise architecture practice the enterprise is not going to go away. It will continue. It’s always a journey that we have to integrate, whether it’s SOA, BPM, or any of these efforts into those long-term initiatives that keep going on.

If the company goes away, well your SOA probably will go away with it or at least be folded into someone else's. But definitely it’s a journey that involves culture change, and that takes a long time.

Gardner: There's an interesting whitepaper being delivered at this event, and it’s part of this notion of "boundaryless information flow." It appears to me that we're all going to be creating new information on an ongoing basis throughout our organizations. So, the boundaries will always be created, and therefore will need to be torn down.

Let’s go to you, Tony. What do you think about this notion of "people" as an important element about the future of SOA?

Tony Baer: There’s no question about it. You and I have been bouncing off each other a new acronym -- and I’ll have to credit you on it -- human-oriented architecture (HOA). Those of you who have been looking at the headlines over the last few weeks, have seen that the usual suspects on Oasis -- IBM, SAP, Oracle and others -- have formally said, "We are going to submit a proposal to incorporate human workflow into BPEL" -- actually into SOA in general.

It’s a recognition on their part that very few processes are completely automated. That’s one side of the story. The other side is that, when you are looking at defining processes, you have to take a look at the context of people's roles, whether the process is automated or not. So, it goes both ways. You can’t have SOA without people, whether you call it human-oriented architecture or whatever.

Gardner: Alright, Eric, to you now. We might actually drop the words, "Services Oriented Architecture." We might still call it generally, "architecture," but it does seem as if this is always going to be a chicken-and-egg relationship.

As we get more kinds of processes, we're going to come up with different standards to try to tear down different walls around siloed platforms or information stores. If we don’t call it SOA in five years, what do you think we should call it?

Eric Knorr: Dave had it exactly right this morning. It’s something that will be subsumed within enterprise architecture, as a whole. It’s part of the ongoing saga of making IT more efficient.

If you listened to Dave’s keynote this morning, he also mentioned that out of the implementations that he had seen, 80 percent were in trouble. Adoption, on a broad level, is still relatively at an early stage right now.

So there is a certain danger right now in SOA in terms of the acronym and its impact on the industry. It maybe another one of these initiatives that looks like it’s going to be very successful. You go through the hype curve, and then it begins to fall away. When that happens, I think it will be absorbed into enterprise architecture. The basic principles won't go away.

Everybody is gravitating toward service orientation within the enterprise, and there are all sorts of reasons why that makes sense from management and architecture reasons, redundancy development, and other things. That will continue to go on, but as a trend, it may have a definite life span -- and that may be only a couple of more years.

Gardner: An interesting observation from the last presentation from Deutsche Bank was that the costs seemed to be going down, according to their forecast over the next several years, and that benefits are going up, at least in terms of one subset of SOA activities. That would lead us to believe that, at least from Deutsche Bank’s perspective, SOA is going to be an ongoing productivity benefit.

If we are able to create a more boundaryless enterprise, if we can get people to work more harmoniously with processes, and agility and change become de facto attributes of IT, then we really will be coming up with something different.

So, let’s look at that. Let’s assume that much of what we’ve heard today is prologue to the future, whether it’s five, seven, 10 years -- and whether we call it SOA or not is not so important.

How is this going to first affect the IT department? Then how is it going to affect the business? Then, take a step further and ask how this might change the characteristics of what we consider companies and how they relate to one another.

Let’s start with the IT department. Todd, you’ve worked in an IT department. If you had a boundaryless information flow, if you had agility, and you could have IT work at the pace of business, what do you think the impact would be on how IT departments actually behave?

Biske: It would be a dramatic shift from what we see today. As Beth pointed out, and other people have said, adopting SOA is a fundamental change in the way that IT operates. It’s a cultural change, and an example that I point to is the notion of a service provider.

During the Austin Energy presentation today, I asked, “A power company is a service provider. They are used to that concept. How does IT act as a service provider, whether they are doing internal services used by their colleagues on another application or are producing services that are being used by customers, business partners?"

We're used to building a solution, putting it into production, and going on to the next project. IT is a very project-based culture.

If you move to SOA, you have to shift more toward a product-based culture, where you have a life cycle that goes on over multiple versions and doesn’t end until you take that service out of production.

You have to deal with customer management, looking at the different scope -- one of consumers and how they utilize those services. How you leverage that information in building new services is where it may tie into your business intelligence (BI) efforts and technologies associated with that. This move from a project-based culture to a product-based culture will be the biggest shift.

If you want a good example, look at companies that practice product management, and at the things that they sell, and you will probably gain a good idea on how IT needs to operate.

Gardner: Beth, you’ve mentioned that you think there are a number of technologies that are going to be important in terms of how they relate to SOA activities. What’s your take on how this agility that we’re anticipating will affect the adoption of technology within IT departments?

Gold-Bernstein: There are related technologies on this. BI is one of them. It’s very important to get the business value out of the SOA infrastructure. BPM is another that is separate technology, but related to SOA.

Very directly it can be used to create the overall end-to-end process. While underlying services are delivering the functionality of the overall application, business process management (BPM) will give you visibility and monitoring.

Also, event-driven architecture (EDA) processing is a separate discipline, but an event-driven SOA delivers higher agility. It provides even predictability, and the technologies are coming forward.

So once you have the SOA infrastructure down, the discipline of creating very loosely coupled components, and layering on top of that BPM and event processing -- then you will be able to deliver to the business the value, the intelligence, the visibility, the monitoring, the predictability that will make the business more agile.

Gardner: Are you anticipating that, as SOA is adopted successfully, it acts as a catalyst, accelerant, or a multiplier to the adoption of other technologies that can then be brought to bear on business issues?

Gold-Bernstein: Absolutely.

Gardner: What’s your take on that, Eric? Are you as optimistic, or do you have any cynicism on that?

Knorr: I wouldn’t say I am cynical about that, but what Beth says does make sense. It’s interesting, too, to see what happens to applications like BI that right now have their own proprietary integration infrastructure in the enterprise.

That’s a big part of what the BI players offer. What’s left? Well, the analytics. As you see SOA methodology spreading through the organization and the ISVs, you'll begin to see a more component-oriented way of developing applications that will permeate the commercial software vendors.

That’s a very long-running project. SAP has been talking about that since 2000. Oracle seems to have jumped in a little bit further with the Fusion platform, while that remains to be seen in the way it plays out. But I do think it will penetrate those areas, and the event-driven angle is very important.

Gardner: Before we leave the implications for the business, we also need to step back and revisit what Dave Linthicum said this morning about other mega trends. He mentioned Software As a Service (SaaS). We have heard other blanks or X’s "as a service," whether it’s integration, platform, or infrastructure.

What do you think will be the impact on IT and the business, as SOA not only opens the floodgates to some of these other technologies, but also opens the floodgates to more ways of acquiring and consuming services outside the organization?

Tony?

Baer: In a way, that’s déjà vu all over again. I remember that during the bubble, when we were all having a lot of fun, we were talking just B2B exchanges. It was the idea of, “Well, you can find something.” It was like an eBay for the supply chain.

You can see how far that went, because it went against established practice in the supply chain. Twenty years of supply chain management theory was: Consolidate your partners, get to know them better, and don’t partner anonymously.

My sense is that what has to come of all this is not just a random coupling. Ten years ago we said all these components out there are not going to just couple randomly. There are going to be partnerships. We were starting to talk the other day about semantic integration, but behind every successful semantic integration is a successful human partnership.

Gardner: So part of a SOA success trajectory would be the ability to consume, as an enterprise, services in a marketplace, where such pressures or forces as competition and a drive for lower costs and higher benefits could play quite nicely.

Baer: And partnerships …

Gardner: And partnerships, so that you find areas where you might overlap with someone in your supply chain. And you must start handing off some of your services, and you might adopt some of theirs.

Baer: Or, perish the thought, start within your own enterprise -- if you can get beyond all the political roadblocks.

Gardner: There are political roadblocks inside of your departments and your companies. Imagine the political roadblocks you’d fight trying to partner with others companies entirely?

Baer: Exactly.

Gardner: So we do come back to this "people" and "what I learned in kindergarten"-mentality.

All right, so let’s not think SOA is all just peaches and cream. You mentioned this semantic issue, Tony.

If I've got a Tower of Babel inside my enterprise, or in a department, where different types of doing things or identifying them are scattered across whatever someone puts in an Excel spreadsheet, then we have this ongoing battle around how we identify that which we want to use, consume, or produce within services.

Todd, let’s get back to you. Let’s discuss what I believe is an inhibiting factor, which is this whole semantic issue. What do you see happening around standardization that is a counter force to that?

Biske: I don’t know that it’s so much standardization that’s the counter force. Certainly it is, but I look at the information integration problem, or master data management, enterprise, logical data model, whatever you want to call it. It's actually a good space to look at and say, “Okay, what do we need to fix to do SOA right?”

Groups trying to do a master data management (MDM) effort or come up with an enterprise canonical data model create this group that goes off on an island. They sit there, lock themselves away, and try to come up with this uber-model, but they miss making the connection back to the people that are working on the projects that actually need to use that information.

It's pointed out that it takes too long to get it done, or it's not meeting my needs. The same thing can happen with SOA if you’re trying to define services in the enterprise context. We don’t want this to happen.

In both of these cases, we need to figure out how to make this information relevant to the projects that need to execute properly and take those incremental steps to get us there.

Clearly, having a consistent semantic model is critical to the success of SOA. If we don’t get the consistency across our services, we wind up creating more work for the consumers. And as the previous presenter from Deutsche Bank said, it’s all about consumption. It’s not about producing. It's about creating services that are easy for our consumers to use.

Gardner: Beth, how do you think that some of the technologies that you mentioned earlier, intelligence and management of applications and services, can be brought to bear on the semantic issue? They seem somewhat divorced at this time.

Gold-Bernstein: Actually a different technology that’s being brought to bear is semantic integration technology. There are standards bodies. I know that the Open Group is working on some of that as well, and some vendors have put forth some semantic integration or enterprise information integration (EII) products.

Software AG came up with one. They have inference engines underneath them. I can remember being at the conference when IBM introduced SNA and told everyone, “You have to build your enterprise data dictionary.”

I worked with organizations that did that. They had the books on their shelves, but it didn’t do anything. They were just books on the shelves. The difference now is that we have the integration technology, and with these semantic integration technologies, we can build these taxonomies incrementally on a project-by-project basis, and then add to them over time, grow this in an organic way, and still be able to move forward that way.

I think that will be the successful way. Semantic integration is not receiving a lot of press. Software AG brought out a great product a year and a half ago or so, and is going nowhere with that so far. That is the future and a very important one, because a dirty little secret about integration, is that 80 percent of the budget is spent on semantic integration.

Gardner: Eric, in addition to the semantic issue internally, as we discussed, being more influenced from outside the organization, we already see popularity of things like mashups, RSS feeds, and content brought to bear on business processes.

Do you think that, as SOA matures and we look to the future, there needs to be a delineation between internal and external content, and who's going to be in role of managing that boundary?

Knorr: I'm not sure who's going to be in that role, but mashups have an immediate role now for SOA adoption. Basically, if you were able to wave the magic wand today and transform your entire application infrastructure to make it SOA-based, you wouldn’t see any difference.

So mashups are a great selling tool. They may not be strictly SOA, but if you got a couple of internal data sources, and maybe Google Maps on the outside, and you throw some Salesforce.com data on there too, you can begin to illustrate for upper management, what agility looks like. That’s one good thing about mashups.

I've been surprised in some of the panels that I have done, with the resistance you get, even to something like Salesforce.com, which is a fairly well-established application right now in the marketplace.

Gardner: It’s a business application right?

Knorr: Right. But, back to the mashups, I think a good way of looking at mashups from a practical perspective and how they fit into all this is that there is a lot of rogue application development going on there under the radar, that nobody in upper management really knows about.

So mashup platforms -- whether they are from BEA or IBM -- they have their first iterations of those types of platforms, or outside the firewall, where you are actually doing development on the outside. These provide a framework for the rogue application development.

Eventually this kind of stuff outside the firewall will be folded into the greater SOA somewhere down the line. In a way, that’s really what's most exciting about SOA, and most difference is the ability to begin to connect to those external services and bring them into the fold.

This goes back to the early days of web services. XML grew out of a desire to open up EDI, and there was this whole thread of web services that were all about B2B connection. Then came Microsoft, saying that one day the Internet will be the operating system, and we will build applications on top of it. Well, that’s interesting to see.

Gardner: I suppose another way of looking at this is that the pendulum keeps swinging across several major variables, one being distributed, and then consolidated. And we go back and forth. And another is complexity that gets solved, but then simply opens up another type of complexity that needs to be solved arrives.

And so if SOA is successful, it seems like we're dealing with a complexity of integration, but then that opens up the complexity of semantic issues, and people and behavioral issues, and then boundary and political and government issues.

So assuming that we can get boundarylessness on this sort of scale, and we have the pendulum going back and forth among these complexities, does the business recognize enough return on investment to say that SOA was worth it? And when will we reach that sort of an economic business rationale? Tony?

Baer: I hate to say this, but I just recalled what David Linthicum was saying this morning about the inherent tension between-project based SOA and the enterprise architects. We all seem to be talking past each other.

To satisfy your client in the business, you are going to show them that it’s deliverable. And the client is saying, "I am not paying for your enterprise architecture. I want a solution that helps me right now that’s a sure fit, an 100 percent fit. I do care if you make it 80 percent, and I have to pay the cost of generalizing it to the rest of the enterprise." So, there is an inherent tension there.

I like what Eric was saying about using mashups as a step forward. That’s what you were saying about semantic integration. We need to work this from the ground up instead of the grand enterprise data model.

We have to take an incremental approach, and don’t try a project to boil the ocean. Then, after you’ve done that, if you can somehow sell it to the business, there might be some internal budgeting mechanism or brownie points, where there can be some sort of internal trading system. And then maybe there is a way to subsidize that extra 20 percent of development.

Gardner: Now, wait. I thought that SOA was going to make agility the number-one requirement that we are going to meet in IT departments. Whether you're Deutsche Bank with a five- to seven-year window on recovering costs -- or a bit more of a common company that has to deal with quarterly returns, and are seemingly always under pressure to cut costs -- there's got to be a better business payback here.

Does anyone, as we discuss the future of SOA, have a sense of where the business people recognize that this is successful and we have made it? What sort of benchmarks would that require?

Knorr: I have a comment on that one. It all has to come back to the business strategy. There are a number of organizations out there that don’t have an enterprise architecture practice, yet, they're trying to do SOA.

As a result you have SOA applied to projects, and you are missing the point that’s on the strategic side of this. We've heard the term "boil the ocean."

We can go to the opposite extreme as well and say, “Okay, I am going to just redo all of IT. We're going to change all of the fundamentals behind this and do it all at once.” That’s not going to work, to completely do the opposite extreme in saying, "everything is ground up."

You need this middle-out approach, but it has to be driven by the business strategy. The way to determine "Have I been successful?" is, "Have I been successful in adopting my business strategy and meeting my business goals?" If I have, then I am doing the right things.

Every enterprise is going to vary in the extent to which IT contributes to those goals. For some organizations, IT may be a strategic asset. There may be lots of specializations that do what I would call a vertical domain that makes sense to them.

For another organization, IT may not be so much of a strategic asset, and they just use it for the standard, horizontal features of HR and business support services.

The best thing for them may be to go down an outsourcing route with a company, saying, “Okay, make sure you are able to keep up with what we need by adopting SOA on your end. But that’s really a part of you being a good business, not necessarily our being a good business.”

It all comes back to what the business is trying to do, and to try to understand how IT can contribute to that solution. If I don’t have any idea on how IT contributes, I am never going to be able to say I was successful or not.

Biske: I think we focused a lot on organizational resistance today, but that’s not to say that there isn’t a lot of success out there in the marketplace right now.

Where you see that success is in telecom and in financial services. Financial services, of course, because they have the money and they are always first, but also because you are talking about a relatively heterogeneous array of services that they can create, recombine and put together into different products. The market demands that they come up with a very broad variety of products that they can deliver to their customers. They are seeing that agility quicker than anybody else.

Gardner: A mega trend that's with us these days, but we didn’t bring up yet is globalization. Companies are competing in ways that they never had to before. So perhaps competition, the ability to compete and win markets and outflank your direct competitors and partners efficiently, and to do mergers and acquisitions well because your IT department can keep up with the business strategies -- perhaps these are the big payoffs from SOA.

Perhaps we'll we see companies effectively embrace what we’ve talked about today, make agility a de facto means through which IT performs at the pace of business. And then they become big companies, get bigger and better, take over more customers, deliver better services, partner across a larger ecology, become masters because they are at the forefront of their markets, and other players become slaves.

Or, am I being a little bit too loose? What do you think, Tony?

Baer: In this thing you were talking about -- globalization, M&A scenarios -- sometimes circumstances have a nice little way of concentrating the mind.

When you all of a sudden are faced with putting two organizations together, which happens pretty often in the business -- M&A is not exactly the exception these days. At some point you have to say, "Look, we need to take an architectural approach."

"Our tried and true methods have been tried and they are true, but they may not be valuable. We keep just going back on our traditional way, our traditional path of execution, and we're just going to develop ourselves into a brick wall."

Gardner: Eric, do you really think that if you are a successful SOA company, you are going to just drop the SOA and just say, “We’re a successful company?”

Knorr: A successful SOA company as what?

Gardner: As someone who executes well on what we have been discussing -- SOA. Does that really make you a successful company?

Knorr: Absolutely. It can contribute to it in terms of agility. There's no question about that. I can’t imagine somebody hanging out their shingle as an SOA company and having trouble getting over that one.

Gold-Bernstein: You asked when a company can say they are successful? In my view, every SOA project needs to contribute to the benefit of the business. It's not down the road five years. Everything you do needs to show benefit, and incrementally. There are different styles of SOA.

We talked about mashups. There is interactive SOA and event-driven SOA. So you are monitoring things and you are giving business intelligence. There is process-driven SOA, with which I'm optimizing my business processes or I'm using a horizontal business process and reusing what I already have.

So I may be building my architecture incrementally, based on different business problems. It's just the approach I'm taking to give more agility. All along the way, to be successful at all, you have to deliver business value for every single project. I don’t think you can wait at all.

Gardner: Okay, before we go to questions from the audience, Todd, for an IT department that can go to their business leadership and say, "You tell us what you want us to do and we will help you do it," versus one that says, "Get in line and wait, give us a list of requirements, and we will get back to you" -- which one is going to be around in five years?

Biske: I don’t know that either of those would be around in five years. We're talking internal IT. I don’t like the notion of a customer-supplier relationship.

It has to be a partnership. So it's not that IT comes to the business, or the business comes to IT. IT is part of the business. And they all sit down at the table together and have equal discussions on what's the right thing from a strategic point of view for that business.

If ever it's the case of just me going and saying, “Give me your requirements, and I will build what you need" ... that’s like saying, "Why don’t you go ahead and outsource me, because someone else can come in and ask you that same question and probably leverage economies of scale better than I can."

If it's the case of "it's pushing too far," you're not creating that shared understanding of the technologies involved to really enable the business to contribute. It has to go in both directions, you have to mutually understand -- IT has to understand the business better, as well as the business side has to understand technology better.

Gardner: Let's take some questions from the audience. This one is, “SOA can’t exist without governance, and there's been a lot of discussion about technical reference architectures. Where are the governance reference models? Are we there yet with governance? Do we have something we can point to and say, 'That’s the way you do it,' and if not, why not?”

Biske: I'll jump in on that one. Working for a consultant firm, obviously we come in and help with governance. It's interesting that we heard "technical reference architecture," and then "governance reference architecture." But I actually view the reference architecture as part of governance. I like to simplify governance to three simple things: people, policies, and process.

First, you have to have some level of authority to put policies in place. If no one recognizes any authority to establish policies, they're not going to be successful. So that should be enterprise architecture or something. Certainly, at an architectural level, you would expect reference architecture material to come from enterprise architecture as the authoritative source.

The reference architecture itself now is an expression of the policies associated with architecture. I have got something concrete. Later, you are using the style, saying, "must," "must not," "should," "should not," but that’s giving exclusive guidance, creating the policies that now the development projects have to follow.

Finally, you have to have process that goes along with that. So, as Beth pointed out, I don’t want to create a bunch of dictionaries that just sit on the shelf and gather dust. I have to have the process to figure out how I actually get people to follow these policies. It’s unlikely that you're going to be able take a strong-arm view, create your police force and just have your key checkpoints where the law is laid down.

Probably it's going to come back with a lot of resistance from your developers, unless your culture is already used to operating in that manner, which, I suspect, most companies are not.

You need to create transparency. You have to have involved with the establishment of the policies people who are also the ones that work with the projects on a day-to-day basis, so there's a feedback loop from the trenches, as well as guidance from above.

You need transparency into the activities that are going on as they occur, versus sitting back, waiting eight weeks to hit the architecture checkpoint, and when it's too costly to make changes, and say "Oh, you're not doing it according to the reference architecture."

You want to have that constant communication, transparency on both sides into the reference architecture development, as well as the development of the specific solution architectures, and everybody really works harmoniously. Its not going to happen overnight, but you always have to consider your consumers.

Gold-Bernstein: Can I add one thing, Todd? I agree. One other thing you need is management, because in SOA it’s dynamically binding. You don’t know whether the policies or the SOA’s are being delivered upon. You need real-time management and visibility, and automated management. So, if something is out of governance, it’s out of policy that is reported on, and you have procedures there to act upon it.

Those are becoming very important, and the thing is that people aren’t adopting them until they’re way down the road. In the beginning of SOA, you have to begin your governance, it has to develop alongside your architecture from the very beginning.

Gardner: So this is clearly another topic for another day: management and governance, do or don’t they come together?

Here is another question: "There are continuing references to semantic data. Two approaches to adding semantic information to data include Resource Description Framework (RDF) and Topic Maps. Any thoughts on RDF and Topic Maps?"

Baer: I think its too early, frankly. I mean, at this point it’s kind of "chicken and the egg." Very often, you’ll have vendor product come out, and then standards will then follow. I don’t mean to make this sound like this is vendor-driven, but in the area of semantic integration of services, there are a handful of products out there and maybe a handful of people actually working with this at this point. So, my sense is it’s too early to call.

Biske: RDF has one advantage right now in that, there are some close ties between that and REST, and certainly REST is continuing to gain in mindshare. I'd like to describe REST as more resource-orientated architecture than service-oriented architecture, but again that’s a debate for another podcast.

If you take this resource-oriented view, clearly RDF fits very nicely into that. But again, those technologies are still just coming out of the incubation stage. How to properly apply that in an enterprise context is still quite a ways off.

Gardner: And, as an indicator of the interest on this topic of semantics, another semantic question, "How far are organizations in their internal enterprise semantic model activities, and what is the extent of adoption?"

Beth, I think, you said it’s something on the order of 80-90 percent, and you had some percentage of activities devoted to semantic issues?

Gold-Bernstein: Oh, no! What I said was that’s the budget of an integration solving semantic issues, but they are doing it manually, re-solving that every time they hook something up. It’s a huge problem, but as far as solving it at a semantic level with semantic metadata and tools automatically doing it, we’re just not there.

Gardner: So your answer to the question of the extent of adoption is "relatively low."

Gold-Bernstein: Yes. It’s not even on the map.

Knorr: And it’s one of the oldest, ugliest problems within IT.

Gold-Bernstein: Absolutely.

Gardner: And we’ve heard that the data issues need to get worked out before many of the other benefits of SOA can be.

Biske: If you're going to have composite applications in the enterprise, and they're tapping into data stores that contradict each other, it’s not going to work.

Gardner: So get your data and semantic acts together.

Biske: Where we do see a little bit of effort happening, and it's wrong to necessarily call it semantics, is in increased adoption of vertical standards, a set of XML tags for a particular vertical domain.

Is that really semantics? Probably in the purest sense. Is it a step in the right direction? Yes.

If you're trying to say, “How do I represent my messages on my services in a way that’s consistent," you certainly can’t go wrong with starting with an industry-specific model. Is that the full way to leverage semantic technologies? No. You are a long way from that, but it's a step in the right direction.

Gardner: It seems to me that if you want to carve out a leadership position in your vertical, that you will take on the heavy lifting of semantics, and then extend it across your ecology.

Gold-Bernstein: But it doesn’t solve the problem for all of those packaged applications, home-grown applications, and everything else you are trying to reuse and integrate with. So, that’s the real semantic problem.

Gardner: Okay, last question, we’re out of time. "We talked about product-based versus project-based cultures. We have a plethora of products and enterprises, they have to continuously evaluate and determine the right fit. So, this is saying, 'We have a lot of product strategy activities, but if we productize services, won’t that result in a similar world of confusion?' And if we can’t even decide on what products to take to market and when, how can we decide which services are appropriate in order to allow us to then fight over which products to take to market?"

Biske: Let me make one quick answer to that, and let everybody else jump on me. By that logic, eBay should never have worked.

Gardner: Explain.

Biske: The whole idea of eBay and this marketplace of things that people are selling, they are selling probably new products. How are you going to make sense of it? How are you going to find it?

Somehow, eBay has put together mechanisms that are partially based on timeliness. There's some very good semantic integration and very good metadata management. Somehow, they put together what should otherwise, by conventionalism, be an impossible task.

In an enterprise, the process isn't easy, but the scope of the problem is going to be a little more containable than eBay.

Gardner: I hear another mega trend here, and that is social networks and the wisdom of crowds applied to services and products with governance, so that we have somewhat of a free-market approach that allows the wisdom to rise to the top?

Biske: I'd actually turn it around a little bit. The original question was part of the reason why we have all this problem in choosing products is that we didn’t have a product-based view to begin with. We didn’t have a capability-based view of, "These are the capabilities I need. Let’s go find the right thing to fit that." Instead we said, "Let’s go pick a product, and now look at what capabilities it gave us."

If you begin with this capability-driven view, now the products just naturally fit, whether they are internally developed, purchased externally. It creates the opportunity for more of a marketplace, leveraging all the conversations about it that the social networks may create for evaluating and selecting things appropriately, whether its internal or external.

I think we should have the same conversations about internal service providers and the quality of service they provide that we have with external providers.

Gardner: Well, we will have to leave it there. Thanks, you’ve been listening to BriefingsDirect SOA Insights Edition.

I want to thank our panel in a live presentation before the Open Group Architecture Practitioners Conference. We’ve had Eric Knorr, executive editor-at-large at InfoWorld; Tony Baer, principal at onStrategies; Todd Biske, principal architect at MomentumSI, and Beth Gold-Bernstein, vice president of ebizQ’s Learning Center.

I'm Dana Gardner, your host and moderator, principal analyst at Interarbor Solutions. Thanks very much for listening.

Listen to the podcast here. Produced as a courtesy of Interarbor Solutions: analysis, consulting and rich new-media content production.

To learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Transcript of Dana Gardner’s BriefingsDirect Podcast on The Future of SOA. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.