Edited transcript of BriefingsDirect[TM] podcast with Dana Gardner, recorded Dec. 19, 2006.
Podcast sponsor: Borland Software.
Listen to the podcast here.
Dana Gardner: Hi, this is Dana Gardner, and you’re listening to BriefingsDirect. Today, a sponsored podcast on developer productivity, on looking at an holistic approach to development that takes into consideration policy-based approaches -- managing process and people -- and not just the technology. We'll also talk about seeking a feedback approach, where testing is continuous -- and visibility into performance becomes an imperative.
To help us sift through some of these issues today we’re joined by Rob Cheng. He’s a Director of Developer Solutions at Borland Software. Thanks for joining us, Rob.
Rob Cheng: Thanks for having me, Dana.
Gardner: Rob, we want to get into a little bit of a historical context here on developer productivity. Why don’t we start out with understanding some of the basic principles around Agile development? I think that’s a good place to start, as we get into some of the approaches that Borland’s taking, and some of the newer benefits that are available in the field now. Give us a little primer and a little history, if you will, on Agile?
Cheng: Certainly. Agile essentially arose as a reaction against what was perceived as more heavyweight methods like Rational Unified Process (RUP). They were considered to be somewhat inflexible and inefficient, and imposed too high of an overhead on the development organization. The term "Agile" actually covers quite a few different specific methodologies, including extreme programming and Scrum, but they all share some basic principles, things like customer focus and rapid iterations on working software.
Gardner: I suppose if you do have rapidity, and you’re remaining lightweight, you also might lose visibility. So, managing this becomes a bit more important.
Cheng: Absolutely. I don’t know if it’s true that you lose visibility, but you have to be more cognizant of what you need to do to maintain visibility. A rapidly iterating process gives you more opportunities to measure things. You could conceivably end up with better higher fidelity measurements of visibility. That’s really where we’re going from the Borland Gauntlet perspective.
Gardner: And obviously it gives us an opportunity to gather more data at more points in the process.
Cheng: That’s right. In terms of data or metrics, rapid iterations really give you more opportunity to sanity check what you’re building, either from a code perspective or, ideally, with the customer/end user of the product.
Gardner: When one gains more data and input, then one can also make comparisons, and have, in a sense, a feedback opportunity.
Cheng: That’s exactly right. One thing that isn’t talked about enough in this area is the context. You get around to looking at projects in relationship to each other in context and also in relationship to time, looking at how things have changed from previous versions or over the last few months of the development process.
Gardner: I suppose the emphasis, until fairly recently, has been on managing the code, managing the development itself, in bits and bytes, checking in and checking out, and so forth, but with less attention paid to the overall process of how to take a project from beginning to end with a lifecycle approach.
Cheng: I think that’s actually an important point around Agile methodologies. If you look at the Agile Manifesto, or any other documentation around it, it really does try to emphasize that there’s a customer focus in all of this. And that customer focus helps drive some of this end-to-end thinking that we’re talking about.
You can’t have customer focus when you’re ignoring requirements, or when you’re ignoring use cases. Your iteration is not going to be completely successful, unless, at the end of it, there is a validation.
Gardner: And that leads of course to the real Nirvana here, which is to align business goals and interest with the development process, and development in general.
Cheng: That’s right. When you talked about aligning to customer needs, ultimately what you’re doing when you’re aligning the customer needs is aligning with the business needs.
Gardner: Now, what's at stake in getting this right or wrong? How big a deal is this, when we look at different vertical industries? And when we look at competencies within organizations, as they compete with one another, and the impact of globalization, and the need for getting to market quickly. What are the stakes here?
Cheng: The stakes are ultimately the success or failure of your software initiative. When you look at reports like the [Standish Group’s] CHAOS Report, which specified some ridiculously high number -- over 60 percent of software projects failing -- that entails a huge cost to the business. There’s a huge cost to software failure, software project failure, and there’s also a significant cost within projects of having to deal a lot of late stage rework.
Getting this right during the process means that it will eliminate a lot of those costs upfront. We can find out early whether there are issues, and the scope and size of those issues. The second part isn’t addressed quite as often. It’s the high business risk involved in simply not knowing. This goes back to the visibility issue. If you’re heading up a business unit, and you ask your accountants for some financial analysis, and they tell you, “Come back in a few months and then maybe we’ll give educated guesses,” what would you do?
Gardner: I’d be concerned.
Cheng: You bet, and you’d be right to be concerned. That’s not how businesses are run, but this is how software development has been running for decades. The businesses that have been trying to manage these, development organizations, have taken it for granted that this is how development works. This is black art. And, it goes to the heart of problem that we at Borland are trying to help the customers solve.
We’re trying to help them make software delivery more of a managed business process that’s based on metrics and objective measurements, not just guess work and anecdotal evidence. When we’re talking about more agile or rapidly iterative process, this is where it really gets to the heart of it. You can contribute to more frequent and improved measurement, not just from code, but, as we were talking about earlier, from an end-to-end user acceptance perspective.
Gardner: So, I shouldn’t need to wait until the project’s done to know, how well it’s doing.
Cheng: That’s a recipe for failure -- waiting until the product’s done to know whether it’s successful. You could take any other example in the business industry and you would be laughed out of any conference room, if you suggested, “We’ll wait until we’ve delivered our product or come off the manufacturing line to decide whether it’s okay.”
Gardner: This makes good sense. It’s logical and it’s rational. We can compare this to other initiatives in the industry in business over the past 150 years, whether it was manufacturing or the Industrial Revolution. So, there seems to be a natural evolution. But, how do we actually put it into practice? How do we go from theory to execution? You’ve gone out and decided that this is an important and a good business to be in. And, you have done some acquisitions to bolster your approach to this. Tell us a little bit about how you got from theory to execution?
Cheng: The technology that we’re talking about today around Gauntlet arose from an acquisition of some technology from a start-up that was run by several of BEA’s WebLogic Workshop veterans. They saw the problems and the challenges within their previous roles in very high definition and contrast.
They felt this was a serious problem that really impacts every development organization on earth. Borland, at that time, was also looking at how we could help extend some of the best practices and process improvements across the lifecycle -- and earlier into development. You might have noticed that we’d also made recent acquisitions around the testing in Quality Assurance (QA) space.
Gardner: Right.
Cheng: One was with Segue Software. One of the challenges that vendors like Segue or Mercury have had is that, although they’ve been quite effective at doing some automation and improvement around the QA process, they had no visibility into what’s happening in development. So, we still had this problem that you’re testing more effectively, but you’re still doing it very late in the game. After it comes off the assembly line, you have very little impact, and you can make very few changes effectively and cost effectively at that time.
Gardner: When I was doing some research into Gauntlet, I learned that the word Gauntlet comes from the idea that you’re going to put their code, the process, the people, through this gauntlet, making sure that it passes all the tests along the way, which is a good visual idea. It reinforces what’s going on here. Can you give us a bit more detail? What do you mean by test-before-the-test-phase and continuous test?
Cheng: That definitely is the metaphor of Gauntlet. What we mean by testing continuously is essentially that we are marrying quality control with existing version-control processes. Specifically, what’s happening is, as developers are committing their changes into their SCM or version control system, Gauntlet works behind the scenes to test, analyze, and measure those changes.
We can also intercept problems. For example, a very common problem is that you make some change, and because you weren’t careful, you break the build. It causes everybody else not be able to work, because you did something, perhaps some trivial change to a configuration file, that stopped it from working for every one else. Well, Gauntlet can actually interpose itself into this process, do the automated build analysis, and find that either it doesn’t build or fails some important test. Gauntlet won’t allow it to progress. It actually won’t promote the code and integrate it into main lines.
Other developers won’t see that and won’t be impacted by that change. When we talk about continuous test, we’re really talking about an approach to software delivery that emphasizes that quality really should be approached, and should be considered in every role and every stage in the life cycle. It’s not just the responsibility of QA. Unless you’ve made that mental acceptance that this is QA’s job, and you’re already in that place where it’s after the fact. You’re doing all this work but it’s already too late to make effective changes.
Gardner: Of course, the earlier you intercept the problem, the less costly and time-consuming it is to fix it, and the sooner you can bring the rest of the team in and on it.
Cheng: Actually, that’s true for number of reasons. Testing frequently does give you this notion of early detection. If you’re testing constantly and continuously, that means the time between when you create a problem or defect and the time when you find out about it is relatively short. That has a couple of consequences. One is that, it doesn’t have time to grow. It’s a bit much to talk about these defects as viral, but they can impact other projects and other development efforts.
Software is classically complex and interdependent. The longer you take between introducing a problem and discovering it, the longer you have that problem, and the longer you have other components and technologies -- or other software modules -- that depend on that particular flawed implementation. Unrolling that becomes much more expensive when you find out about it much later.
Gardner: I’ve heard some folks say, “Gee! You want us to have quality, and you want us to test continually, but you also want it done fast. Now which is it going to be?” I think there’s been some mentality that testing slows things down, and you’re bringing this in however as an overlay. This isn’t something that people need to do terribly differently. It’s not a rip-and-replace approach. You’re letting people use their same software configuration management (SCM), the same tests, and you’re just providing an organizational matrix on that. Is that correct?
Cheng: Yes that’s right. In fact, we do lower the overhead to some extent. When we talk about testing often, those are manual tests. Those are tests that the developer or QA engineer has to remember to do. They have to do them manually, have to take time out of their implementation to go and run these. Build it locally, test it locally. Presumably you’re doing something with that output as well, and all of that is overhead.
Gauntlet is trying to help streamline this process by automating all this stuff. The building and the testing actually all happens on the server. So, once the developer checks it in, it’s fire and forget. They can go on to the next task and check with the dashboard to see what the result of that was later.
Gardner: So, a rundown Rob, if you will, about what Gauntlet is? How it’s priced? What’s its architecture? Give us just a quick elevator description of what it is we’re actually talking about here.
Cheng: Sure. Gauntlet, as we talked about earlier, is a continuous build-and-test automation solution. Essentially, it’s a server-based product, server software.
It’s charged by the seat. It does interact with the version control systems that work with StarTeam, and we are constantly looking at also increasing the range of version control systems or SCM systems that we support.
Gardner: What have the results been so far? Are you going to come out with a new version or major release in early Q1 of 2007?
Cheng: That’s right.
Gardner: What have been some of the results so far? How has the field reacted to this?
Cheng: Right. The release in early Q1 of 2007 is actually the first official production release. We acquired Gauntlet in February, and since then we have doing a number of internal customer field trials as well as make a public early access release available. The feedback has been quite good.
I’ve been surprised by how quickly people have been able to get up to speed in using it and seeing benefit. The thing that’s really interesting about is that there are organizations, customers, and evaluators that are at all different phases and all different parts of the spectrum, when it comes to their occurrence, build, and development processes.
We have some that barely have unit tests. It was a feat to get them to actually have a build script that they can kick off in an automated fashion. Gauntlet gives them a framework, the central dashboard if you will, to monitor their progress, because it is one of their initiatives to improve. Part of what you need to do to build improve is to measure. You only get what you measure.
Gauntlet not only does the measurement, but it also does the visualization of it. So you can see from day to day, week to week, and project to project how things are progressing. You can see how often you’re able to build successfully, how much testing coverage you have from different projects, and how that is changing overtime. Are you increasing the coverage of testing that you have for your the different classes or different projects?
Then, we have other customers who are way on the other end of the spectrum, where they’re already very rapidly iterative. They are very agile and they see Gauntlet as a way to push that even further. They take their best practices and multiply the impact of them by offloading a lot of manual effort, doing it more on an automated way, and to giving the entire organization much more real-time feedback and better visibility about what’s going on?
Gardner: So, from their perspective of using this as a tool, they can find utility in it across the spectrum of maturation, in terms of where they are in their development and where they hope to go? That’s always a good descriptor of a long-term value. You mentioned something about visualizing a dashboard. Are these things that are designed for the architects or the project manager level, or is it some thing you can talk to the business people and say, “Here’s where we are. No guess work. We’re on track?” Is it designed for business people to engage with this as well?
Cheng: The Gauntlet dashboard currently is designed for development managers, project managers, and individual practitioners: the developers, the QA engineers, QA management, of course. The consequence of Gauntlet is that those managers, those project leads, the project managers, have the objective information to tell the business people what the actual information is.
They can tell what the status of the project is, what the risk is involved, how close they are to being able to actually ship, and if they know what the real stability is. They know what the confidence level is. A really good way of characterizing is that you’re giving developmental IT management much more confidence in how and what they say to business management?
Gardner: They can report to these business side folks with authority, because they do actually know where things are?
Cheng: Exactly. It’s fact based. It’s not just based on educated guesses.
Gardner: That notion of hope as a business strategy doesn’t have to kick in here.
Cheng: Actually, it’s not business strategy; it’s a recipe for failure and disaster. There often is too much -- what was one of the term that [former Federal Reserve Chairman Alan] Greenspan used -- an over abundance of optimism?
Gardner: Exuberance, perhaps?
Cheng: Perhaps that was it, but the overabundance of exuberance is really something that characterizes a lot of development in organizations sometimes. Gauntlet is a nice way to run a sanity check on that.
Gardner: One of the things you mentioned a moment ago about this spectrum of different organizations being able to apply this well gets to the heart of an area that’s always fascinating to me. That’s how to manage behavior, cultural shifts, individuals and in larger groups, getting them to change the way they think and move beyond a certain pattern of accomplishment, even though they feel it has probably been the only way up to a certain point.
Can you tell us little bit about how Gauntlet might be brought to bear on this issue of managed business processes, and how to get people to change the behavior? Can we really think about this as a way of orchestrating teams and behaviors? Or is that biting off a bit too much to chew?
Cheng: Yes and no. There is definitely something required as a forcing function, and it allows the adoption of more and more agile development. There’s usually a big failure or big problem.
One of our beta customers, for example, who is very aggressively adopting this technology, had what they called a brand-damaging, embarrassing failure event on their external customer-facing Website. They felt as if this was a symptom of the process failure that they had, and that was what really initiated their need to drive a change in process and in culture.
So, often there’ll be issues that people know they need to change, but they’re kind of set in their ways. They need something to spur them to action. Gauntlet does help in a sense that once somebody -- it doesn’t even have to be the entire company, the entire organization -- gets it in their head that they need to try this or they want to experiment with the more iterative, more agile approach, Gauntlet really helps them climb that learning curve. They climb that ladder of progressing incrementally towards getting more effective in their processes.
It can be just simple things like tracking builds, making sure that you can automate your builds, and that you’re building successfully. Then, tracking your unit tests. Do you have enough unit tests? Are they succeeding? How much of the code base are your tests actually covering? A lot of those things you could figure out, but it would be a lot of extra overhead to manage that process manually.
Gardner: So, for those organizations that have decided that they need to move from a tactical ad-hoc approach to a more strategic processes-driven approach, this gives them a ramp and a tool to start moving swiftly in that direction.
Cheng: That’s right. When you talk about strategic, we talked earlier about the notion of software delivery as a managed business process. One of the things that characterize other business processes in organizations is that you can do real sophisticated analytics. You can do business intelligence. That’s one of things that Gauntlet is trying to bring to the table, because of being able to continuously test, measure, and collect all the data.
Gauntlet actually gives you this great wealth of information that you can then data mine. When you talk about strategy and process, you also have to think about how to optimize the process. How do organizations continuously improve processes, not just code or technology? How are they are doing the work itself? A lot of that comes down to being able to mine to do ad-hoc analytical queries against the data that’s been collected about how the organization is actually operating. That’s one of the things that Gauntlet really provides -- that data warehouse of activity and metrics around actual development actions.
Gardner: So, I’ll get more data, I‘ll be able to start analyzing it. And then I can start setting policies, and then be able to revisit those policies and, on an iterative basis, improve upon them. What can we tell people about this policy-based approach or policy enforcement up and down the process and up and down the development lifecycle?
Cheng: That’s a very good question. The issue of process is important. A lot of times, process decisions are made for very good reasons at a higher level, in the process office, in the CTO, or the CIO’s office. They’re made in conjunction and collaboration with the management of software development. It often doesn’t get down to the individual practitioners, or it’s hard to enforce, because it’s just something that you’re telling people you’re managing that they have to do. How do you discipline it, and how do you enforce it?
One thing that Gauntlet allows you to do is make policy decisions at the point of insertion, at the point of code entering the system. Earlier, I talked about how we can essentially gate the promotion of code, the integration of code changes into different controls, based on whether they’ll be all that successful or whether test passed? Well, this is also something like you use much more generically as a framework, so that you can embed policy into that point as well. We can make decisions about requirements around maintainability, the complexity of the code, and the readability of the code.
If we have tools, they’ll help us analyze security, vulnerabilities or even such things as license compliance. Are you introducing unapproved third party’s libraries as open source libraries and packages? All these types of issues have the same characteristic cost curve, which is the later you find out about problems, the much more expensive it is to remediate.
Gardner: It also sounds like you can capture knowledge and competencies and then extend them, reuse them in a sense. This leads us into my next set of questions. What’s coming down in the future around these sorts of benefits, as we think about things like managed services and services-oriented architecture, and continuous ecology development of services, where there really isn’t just a cut-off date? As people consume these services, you’re going to adjust them, and they’re going to be working under service-level agreements. Is the Gauntlet approach something that’s going to become necessary as we move into that new arena of constantly dynamic development?
Cheng: Absolutely. In fact, a couple of words that you said were really resonating with me. One of them is “competencies.” From the competency perspective, one of the things that really breaks the back of development organizations sometimes -- and I’ve heard this a number of times from development directors – is that there may be dependencies in your system that you’re not really aware of. You don’t really think about them until they’re broken.
One of those is the human element, where there may be one person in the entire organization who knows how to put the software together. You only have one person who knows how to build the software, and that person leaves the company. You’re kind of hosed. You may have weeks or months worth of catch-up to get that knowledge back, but it ought to be institutional knowledge.
Gardner: Right.
Cheng: That shouldn’t be somebody’s personal expertise. The company or the development organization is running on that knowledge. Gauntlet helps to capture that in a centralized way and in an automated way, and share it with everyone.
This is also true of the situations when you have maybe niche platforms that you are trying to support or niche environments. There’s only one guy doing it, but you feel like it’s okay. Then, later there’s a customer escalation, because they’re that one big customer that happens to use that platform. And, by the way, the institutional knowledge is now gone. Again, the idea that we could capture that in a system, which actually is doubling as a means of collaboration, is an important one.
The second part about being able to have more services-oriented approaches, more dynamic development processes, also speaks to the issue of globally distributed and outsourced teams. One of the things that Gauntlet can bring to the table there is measurement. A big problem with outsourcing is how do we know what the cost benefit is? I know what the cost is, but we don’t know what the benefit is.
Gardner: Right.
Cheng: We don’t know how effective it is? We have no way to measure productivity or quality of the deliverables. Gauntlet, with frequent objective fact-based measurement, can start giving you answers to those kinds of questions. On the other hand, we also have the ability to do that process enforcement we’re talking about or the policy enforcement. Once I made a determination about the relative effectiveness of different teams, I can put different policies on the different teams.
Gardner: That strikes me as essential, if you want to go from the perception of IT as a cost center to a value center, moving the yardstick up in terms of getting more discretionary spending and more innovative types of projects that are going to be brought to the table and funded.
Cheng: That’s absolutely true. When you talk about that perspective and also the SOA arguments, all of it is predicated on the ability of someone in the IT organization convincing those with the purse strings that they can deliver predictably and they can deliver to target in a managed way.
Gardner: Give us a little bit of a road map if you could, as to what the, purview of Gauntlet is. I think that it’s mostly focused on Enterprise Java development at this time. Where do you see its boundaries or how do you expect it to evolve?
Cheng: The great thing about Gauntlet is that from a framework perspective, it’s as agnostic -- particularly of your language and your development --as version control itself. You can store anything. It doesn’t even have to be source code. And, you can do documentation and other types of content management essentially through version control.
Gauntlet does provide that same framework to be able to invoke and then analyze the results of tests and other measurements that you do. But, you’re quite right that in the first release we are very focused on Enterprise Java. That’s where the ecosystem of the test and analytic capabilities is really focused, but we will be aggressively moving over the next year to support additional languages, environments and systems with .NET development, with C, C++ development.
These are all areas in which customers really want to be able to leverage Gauntlet as well. So, we’ll definitely be moving to that direction and we’ll also be expanding, as I mentioned earlier, the version control on SCM support that we’ll have.
Gardner: And this is already something that’s usually used in conjunction with open source and the Eclipse-based development.
Cheng: Absolutely. This acquisition was from a start up that was almost a 100 percent open-source stack, and it does have extremely effective networking with the sub version that was the original SCM version control system. It leverages NAnt and JUnit as tools for building and unit testing. We can configure it to work with a number of other open-source tools and analytical packages.
At a higher level, it’s also a way bringing some of the best practices of open-source development to commercial customers. One of the things about open source development is that it’s very transparent. I can go to the Website or the public repository and see what’s happening. I can see who did what, when, who broke what, who fixed what?
Earlier we talked about how we start nudging cultural change. One great way to do it is having the kind of transparency that promotes peer-to-peer accountability, as opposed to top-down management. You don’t want to be the one who’s embarrassed by breaking everything. And, on the flip side, you want to be the one who’s recognized as making the most contributions. That’s true in open source, and we think that it could be just as true in improving commercial proprietary software development.
Gardner: Great. Well, thanks. I think that wraps it up for our time allotment today. We’ve been discussing productivity in development and bringing more visibility and accountability in data, fact-based development, if you will.
To work through this discussion, we’ve been joined by Rob Cheng. He’s the Director of Developer Solutions at Borland Software. We’ve been discussing a product called Gauntlet that will be coming to market in full maturity in Q1 of 2007. Thank you for joining, Rob.
Cheng: Thank you very much, Dana.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You’ve been listening to a sponsored BriefingsDirect podcast. Thanks for listening.
Listen to the podcast here.
Podcast sponsor: Borland Software, Inc.
Transcript of Dana Gardner’s BriefingsDirect podcast on development process management. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Monday, January 22, 2007
Sunday, January 21, 2007
Transcript of BriefingsDirect SOA Insights Edition Vol. 7 Podcast On Measuring SOA ROI and How SaaS and SOA Intersect
Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition, recorded Jan. 5, 2007.
Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Dana Gardner at 603-528-2435.
Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Volume 7, a weekly discussion and dissection of Services Oriented Architecture (SOA)-related news and events, with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions, ZDNet blogger and Redmond Developer magazine columnist.
Our panel this week -- and that’s the week of Jan. 1, 2007 -- consists of Steve Garone, a former IDC group VP, founder of the AlignIT Group, and an independent industry analyst. Welcome, Steve.
Steve Garone: Hi, Dana, and happy New Year.
Gardner: Thanks. Also joining us again, Joe McKendrick, research consultant, columnist at Database Trends, and a blogger at ZDNet and ebizQ. Welcome back, Joe.
Joe McKendrick: Thanks, Dana, I’m glad to be back, and happy New Year as well.
Gardner: Also, Tony Baer, making a second appearance on this show, principal at onStrategies. Thanks for joining, Tony.
Tony Baer: Happy New Year, Dana.
Dana Gardner: Thank you, and making his SOA Insights debut, Jim Kobielus, he’s a principal analyst at Current Analysis. Hi, Jim.
Jim Kobielus: Hi, Dana.
Gardner: Also, welcome to our listeners. Our topics for today span the SOA spectrum of business and technology. We’re going to be discussing metrics of success, ROI, TCO, how do I know I’m doing? I guess we could call this the Ed Koch approach to SOA: "How am I doing?" Yet we can’t really judge how well SOA is doing, if we don’t have metrics.
We’re also going to work on the impact of SOA on outsourcing: outsourcing of IT, outsourcing of business process. Is this an accelerant? Is this a detractor? What’s the impact? We’re going to explore that. Also, the impact of SOA on services -- software as a service (SaaS) in particular; business applications coming across the wire.
So first off, let’s go back to the metrics topic. Someone out there, and I believe it was you, Tony, mentioned that at one point recently Verizon had come out and said that with a stable of 500 services that they were expecting to yield $20 million in savings over two years. Can you tell us where that information came from and how credible you think it is?
Baer: Good question. I actually spoke with the person who went by the title Executive Director of Strategic Systems at Verizon. Essentially, he reported to the CIO and he was in charge of their SOA program.
He gave a presentation on "SOA In Action." "In action" being two words there. I want to be very careful about that. He gave some metrics. I spoke with him about six or eight months ago, so if there is current data, I’m sure it’s different. But this has been going at Verizon for roughly a couple of years. He spoke of creating in excess of over 500, what he called "unique," services.
Now, I’m always a little skeptical when I hear the term "services," because what does that really mean? The definition can be very loose. It could be we have some functionality here and we’re somehow exposing it, or exposing it using standards, using official Web services standards.
Anyway, his point was that these 500 services, which were developed or reused by a community of over 7,000 developers, now account for -- when I say "now," this is as of six or eight months ago -- over 8.5 million transactions per day. He claims that within a couple of years it has saved the company $20 million. I would presume this is in development time, as opposed to having to create each of these things and reinvent the wheel, which is the traditional way of going about it.
My eyes tend to glaze over at such large numbers without having any greater detail, but the fact is, it’s always been a central tenet of reuse that if you can utilize something and not have to create it from scratch, you theoretically should save money and time. So, my sense is that I think the numbers are probably a bit high and a little loose, but I think they point to the potential that SOA can provide in delivering real savings.
Gardner: I wonder if the figures are high, especially on the number of services. I’ve talked to several companies that seem to think that even a smaller number of services could handle just about all of their back-office and front-office needs, perhaps not in vertical usage or industry usage. Then, for a company like Verizon, with the kind of IT budget they have, $20 million might actually not be that much.
Let’s go to Steve Garone. You’ve talked to a number of companies, and you were looking at ROI and business metrics, do these numbers make sense to you?
Garone: The numbers really aren’t out of the ordinary, but I think one of the challenges that organizations face in doing SOA and then trying to determine their ROI -- for want of a better way to put it -- isn't unique to SOA. It’s really a function of various other types of IT-related activities as well. How do you actually quantify what your ROI is, given the advantages of using an SOA approach? I’ve listed the main reasons why people would want to do SOA, in terms of the advantages, and they basically break down to four major areas.
First is what we just mentioned, which is the reuse of IT assets. That's fairly easy to quantify. As I’ve talked to customers and people who have implemented SOA, I find that this is one that they’re comfortable with and are fairly aggressive about putting numbers to, in terms of what they’ve actually saved. It’s not an easy task, because what actually gets saved when you reuse an asset? You certainly save them some development time and development cost. Do you also save maintenance cost, and how do you quantify that? Is it easier to upgrade assets, and to what extent is that dependent on the infrastructure you have in place?
A lot of factors go into it, so I see various levels of rigor. You see people claiming that they were able to reduce the expense associated with doing the application integration test they normally would have to do. That one is a real tough one, because just understanding what you would have had to do and breaking that down into, not only some specific cost, but also the relationship of the result to being able to do business better, faster with more customers and so on, is a real tough task to accomplish.
The other ones have to do with meeting compliance requirements, which we all know are growing by leaps and bounds every day.
Gardner: Regulatory compliances?
Garone: Yeah, and there’s an area where, again, I think the quantification of benefit might be a little bit easier because the requirements are fairly well identified. In fact, they are very well identified.
The fourth one, which I think is the hardest one, is the issue of how agile do you make your business. It’s a buzzword we hear all the time around SOA. That’s a real tough one because you have to ask such questions as: "If I didn’t do this, how many customers that I'm doing business with now, will I not be doing business with, and what affects that? How do I know what my reach was before versus now? How do I quantify how easy it was for a customer to get to me, and if they do, how comfortable they were given what I presented to them, doing business versus leaving because things were taking too long?"
Gardner: So, most of these are soft measurements, and they would take place over a long period of time, because it’s a journey, and you can’t just say, "Well, we went one quarter. What's our payback?"
Garone: That’s right.
Gardner: Now, let’s go over to Joe McKendrick. These are soft paybacks, and it also seems as if we’re comparing apples to oranges. If we look at mainframe, client-server, Web, and then SOA, which is such a different beast, you can’t really compare. It’s almost a leap of faith. Where do you see the right business metrics coming and getting people onboard in terms of dollars and cents here, Joe?
McKendrick: Well one thing I’ve said a lot on my blog is that the companies that are most likely implementing, or are implementing, SOA are probably the companies that don't really need SOA as much. The companies that really could use SOA in their operations are likely not to be the ones implementing, it’s kind of a paradox.
The reason that is the companies that have the management vision, the support to roll out and move an SOA project forward -- providing top management support, for example -- are likely to have lot of other initiatives on the table. They’re likely to be very forward-looking with their management. Their compensation structure, for example, is likely to be a bit more advanced. They’re likely to have other implementations -- a data warehouse, for example -- a very advanced data warehouse. Essentially what you have in these companies is an orchestra making a lot of beautiful music and SOA may be the horn section, but how do you measure the impact of SOA versus another type of implementation or another type of project?
Gardner: Okay. So the song, the piece of art that you’re creating with this orchestra rises up the charts, and if they're a hit, is it because of the woodwind. Is it because of percussion? Is it because of the conductor? We really don't know. We just know that it’s a hit. Right?
McKendrick: Exactly. There are a lot of case studies out there now. A couple of prominent ones are IBM and HP, who had cases of eating their own dog food. They’ve been able show some terrific significant impact from SOA or service deployment. IBM claims it’s reduced its application portfolio by 75%.
They have one quarter of the applications running their business versus back in 1998 when they really first started looking at this. HP claims it's seeing a payback of about $70 million as result of its SOA efforts. These are companies that tend to be forward thinking. They know that they’re being closely watched, and that their own operations are being closely watched.
In both cases, the benefit is coming from consolidation. They’re using SOA as a lever to help consolidate data centers, various far-flung parts of their enterprises. SOA is just part of that consolidation picture. There’s also virtualization going on, server consolidation. IBM, of course, is big with the mainframe, and they’re consolidating a lot of their operations on Linux running on the mainframe. It’s very much a mixed picture, and as I said, Dana, it’s a matter of being able to separate what’s making this beautiful music.
Gardner: I’ve spoken to HP and IBM as well, and they’re really undergoing IT transformation, probably business transformation, and there are many constituent parts to that, of which SOA is one. But SOA is one that has, I suppose, a lot of interdependencies and effects across many of these other activities, whether it’s server consolidation, application modernization, IT shared services, virtualization and what have you.
Let’s go over to Jim. Jim, if SOA is important, almost like a root system that cuts across a number of different trees that are growing, is it even worthwhile measuring it, or should people just be smart enough to recognize that this is the right thing to do?
Kobielus: That’s an interesting metaphor there -- SOA as a root system. My visual image of SOA is a very complex hyper-mesh. In other words, like a root system, where you have tendrils going hither and yon, the tendrils being simply interactions among services and client.
It’s very worthwhile to measure the ROI of SOA as a paradigm or an approach for enabling and for maximizing the sharing and reuse and interoperability of distributed resources across your network. You make an investment as an organization, as an enterprise, and in this approach you want to know whether you’re investing your funds and your resources wisely. When I think of SOA’s ROI, I think of two numbers, and those numbers are 100 and zero.
As we know, SOA focuses on how you maximize the sharing and reuse of services, of application functionality and resources. In other words, how do you enable a 100 percent reuse as a nirvana? We’ll never get there, but in any given organization, 100 percent reuse, service reuse, first and foremost is a consolidation topic. What that means is, if you do SOA right, you’re doing much more with much less.
You’re able to consolidate redundant silos of application functionality and data throughout the organization. You’re able to consolidate fewer software licenses and servers, with the associated translation and cost savings and capital on operating budgets, fewer redundant software components and so forth. The need for fewer programming groups, as we can consolidate that as well.
So 100 percent reuse is the nirvana. The zero comes in the sense that, if you’re doing SOA right, the marginal cost of billing the next application drops pretty close to zero. You’re able to reuse everything that’s already been built. You do not have to reinvent the wheel. So, basically, a 100 percent reuse means zero marginal cost of building the next application. Of course, as I said, you enable that vision through consolidation, both in software and hardware, and in programming teams, and so forth.
So, once again, getting back to your root-system-and-tree metaphor, SOA becomes this ubiquitous root system from which new sprigs can pop up, without needing to lay down their own root system. Rather they are simply branches on a huge underground system. In Northern Michigan, where I’m from, scientists have discovered the world’s largest organism as a mushroom or a fungus of some sort that spans 30, 40, or 50 square miles. They determined though DNA analysis that it's the exact same individual and has got the largest biomass in the world. In essence -- and it’s all underground pretty much. That’s what SOA is all about, essentially all the services in an SOA sort of share a common DNA.
Gardner: Well, there’s the message we need to take to the CEOs and the CFOs. Let’s make our IT like a fungus.
Kobielus: I think they probably already believe that!
Gardner: Let’s take this now to the outsourcing side. You mentioned between zero and a 100 percent in terms of reuse. That would be reuse for your internal resources, but what about reuse of external services? Things that are made available in a marketplace, because of the effects of economics and supply and demand in competition, could push the prices down while even enhancing the quality of services.
For those commoditized services that everyone uses, that really don’t differentiate anyway, why not go in that direction? What’s the relationship here, Steve Garone, between capturing ROI and lowering total cost of ownership, and starting to look toward outsourcing an SOA as being somehow a tag team?
Garone: Well, actually I think there is a fairly close relationship. Again, it’s not unusual that the drivers are really the same as they are in the case of other types of IT investments. Outsourcing can potentially drive costs down, increase quality, allow you to lighten up your IT staff, and so on. So, the same rules apply here. If you look at what’s going on, there are actually several companies now that are doing this. And, it’s not just commodity services. Look at companies like Salesforce.com, for example, which is essentially creating a services-based hosting environment, not only for their applications, but for applications and services that you actually produce in house.
Gardner: Incidentally, we should expect from Amazon.com similar activity around a retail and e-commerce sector.
Garone: I wouldn’t be surprised. So, the drivers are the same, and I think it’s going to happen. I think that if you’re talking about outsourcing services from the hosting standpoint, that’s certainly real and it’s certainly happening now. If you’re talking about outsourcing services from the development and IT support standpoint, meaning how do I go about building my SOA environment, there’s going to be a stage initially where there’s going to be a lot of that, for the same reasons people outsourced that kind of work before.
People look toward SOA and Web services to simplify their application integration environment. That may, in fact, be the case, but in the short term, people still really need to know how to do it and need, in some cases, to outsource for the expertise to get that done.
Gardner: It seems to me that in the not-too-distant future being able to access commodity-level services that you can incorporate into your SOA activities, off the wire, in a competitive environment, would be a really big economic benefit, to reduce your total costs. Maybe not grow your top line right away, but certainly cut your overall cost, which has been an important element of IT for certainly the last five years.
Let’s take this another step sideways. What about IT outsourcing? We had this phenomena a few years ago. Some of the largest entities on Wall Street -- and I’m thinking of JPMorgan Chase -- went to large companies like IBM and said, "Here, you take it. We don’t want to do IT anymore."
Then maybe a few years, even some months, later decided, "Well, maybe that’s not the right approach." If we can get into a marketplace of services, if we created an SOA, doesn't that really obviate the need for IT outsourcing? Don't you want to retain the competency of business process and composite application authoring, regardless of where those sources come from? Isn’t SOA and the outsourcing of services a better model than wholesale IT outsourcing? I’ll throw that to you, Tony.
Baer: I was just thinking back to what Steve was saying, and it kind of applies to this answer as well. If there is one benefit that SOA delivers, it’s that the value becomes the service rather than the plumbing. If you think about the way we've traditionally developed functionality or integrated systems, we’ve had to spend inordinate amounts of time in the plumbing and maintaining it. SOA theoretically, if it’s done right, standardizes the plumbing, makes everything declarative, so you take out the guess work. The result is that if you look at outsourcing, SOA separates the plumbing from the service. Therefore, what is probably ideal for outsourcing would be the plumbing, because that’s where the value is and that’s not where IT organizations should be spinning their wheels.
Gardner: So where is the value?
Baer: The value is in the -- and this will sound a little cliché-ish -- intellectual property, which is represented by the actual service. That’s the service that delivers the actual business logic. It’s the way a company does business. The way a company does business is not the way it puts together its SOA plumbing. That should be standard. That’s obviously an ideal scenario, but look at how this could impact the outsourcing market. If you follow SOA principles and comply with SOA standards, it allows for that kind of separation, and therefore it makes for an ideal opportunity for outsourcing services to say, "Hey, we’ll take care of your plumbing, and you take care of your services."
Gardner: So, you’re saying perhaps that I should look to commoditize services and find the cheapest best way to acquire them, either on premises, co-located, outsourced entirely, software as a service (SaaS). Then, for those differentiating services that I’m applying intellectual property and perhaps even patents to, then I could even host those somewhere off premises, and it’s bringing that together within my organization that makes me a winner in my market?
Baer: That’s one way of putting it, but another way is, "I create the service, and by the way I rely on you to create and maintain the plumbing." There are many different ways of slicing and dicing this.
Garone: To some extent it depends on what kind of vendor you’re dealing with. I’ll go back to the example I gave earlier of Salesforce.com, which will do some of both. They create a services-based infrastructure that you can use to not only host your IP-unique applications in services, but also to host the ones that they’re providing to you, and if you do it right, you can easily integrate with.
Gardner: Right.
Kobielus: On the issue of outsourcing, the only thing that the company should never consider outsourcing in terms of core competencies is the core competency of making money and delivering value. Everything else is fair game, and I think what everybody is saying here is that they agree with that. SOA -- or much of what we think of as SOA -- revolves round the plumbing, the protocols, the application servers, the middleware fabric, etc., that all is fair game for outsourcing.
Gardner: I like the terms that were used a moment ago. It can be "sliced and diced in a number of different ways." That’s the good news, but it’s also the bad news, because no one really has a model they can fall back on and say, "Ah, here’s the proper way to do this, when we apply outsourcing and software as a service to the SOA mix." Or have I got this wrong? Jim, can you think of any past instances in the history of IT we can look at to get some idea of the best way to slice and dice these many variables?
Kobielus: Going back to my sense of knowledge of IT history, I’m going to have to sleep on that one, Dana.
Gardner: Can anyone else out there think of any precedents that we can look to in terms of what’s the right balance of on-premises, commoditized, software as a service, SOA? It gets kind of messy.
McKendrick: I don’t know if there really is a precedent. This is all open for innovation, Dana. There are a couple of interesting examples, actually, if you look at the telecom industry. I like to use the term "loosely coupled business." SOA is based on loosely coupled components or services. A lot of telecoms actually don’t use their own internal resources to provide services to their external customers. They act as a broker.
Gardner: It shows sometimes.
McKendrick: Unfortunately, yes. I’ve heard a statistic that at Cisco, the networking giant, and 80 percent of its products and services come through outside services that it essentially brokers. I think that’s kind of the model SOA is helping us move toward -- this idea of a loosely coupled business that can pull together services from perhaps throughout the globe, package them, and offer them to an end user or customer that they identify as important to their market.
Gardner: So, maybe the metaphor has moved from a 30-square-mile fungus to a 30-square-mile amoeba.
Koblieus: Amoeba is a good metaphor here, because business models must continue or will always continue to evolve; therefore IT must continue to evolve to the next business model, the next IT sourcing model, whatever helps them to survive and make money.
Gardner: We came up against this when we talked to Jeff Pendleton. We thought about architecture and we looked at some of the architects, and we came to Frank Gehry, and we said, "Wow, the outside looks different and almost illogical, but inside, the stuff holding it together is very logically designed."
Then we moved from that to this notion of a sound stage. You would build a sound stage, or even a movie set, depending on whatever scene you were shooting. And, we wanted to have IT act like a flexible sound stage or a movie set. We would just build the set based on whatever the activity was, do what we needed to do, and then perhaps tear it down or reuse parts of it. So, we seem to be coming back to that, although we’re talking about not just our own services as an enterprise on premises, but increasingly a mixture of software as a service and outsourcing.
Anybody want to pick at that or expand on it?
Gardner: Well, we're in complete agreement, that’s great.
Garone: I’m not quite sure I understood it, to be honest with you.
Gardner: Do you remember that call we had about the sound stage show; I think you were on that call?
McKendrick: It’s an interesting analogy.
Gardner: MGM right?
Baer: Right. That actually is about the old movie studio metaphor, one studio and many different productions going on at the same time. Basically, the piece parts were all interchangeable, and piece parts we can call plumbing.
Gardner: So, we’re making five different movies on the lot, and we’re going to bring cameras and lighting equipment and backdrops and makeup artists, and just apply them in the same way, but with a different end outcome. Right?
Baer: I guess you can say the metaphor coming out of this discussion has evolved from fungus to amoeba to let’s bring back the studio system into IT.
Kobielus: Right, if you look at what SOA governance involves, if you examine a given service throughout its life cycle, it begins in the planning stage, it’s a gleam in some business person’s eye, and then it goes to the system engineers, the designers the business people. There’s a governance life cycle on those projects as well. That governance role structure in the movie industry has changed since the very start and it continues to evolve.
The notion of the classic studio system for Hollywood was there between the '20s and the '50s, and then it gradually gave way to more independent producers. Now, you’ve got all these indies everywhere. What I’m getting at is that, the governance of any given movie production project, how it’s done, the best practice for that particular industry, continues to evolve from generation to generation.
Likewise in the IT world. If you look at the life cycle of governance of the creation of any application or functionality, of the whole software is on the life cycle, that paradigm continues to evolve from generation to generation, with new platforms, new tools, and so forth. So, its one of those things where now you’ve got to look at the structure of the SOA governance environment.
You've got roles that are specific to the plan, specific to the development stage, roles that are specific to the deployment and optimization stages and so on. What I’m getting at, then, is that I think the movie studio analogy is very interesting, because it focuses on the core of governance as a workflow over a life cycle involving various roles, and those roles continue to munge into each other and get broken off from each other.
Garone: Dana, just jumping in here, now that I had a chance to think a little bit about your analogy, and just to follow up on that last thought. In essence, the movie studios as we know them now, don’t have the role that we used to know them as having. They used to actually make movies. My understanding is that the customer who is the independent producer, or maybe a larger producer who may bring on another independent producer under them, basically is outsourcing the physical plant requirements to make a movie to the studio.
Gardner: So it becomes an ecology rather than all-in-one.
Garone: Exactly. It’s an ecosystem.
Gardner: Is it the right direction, Steve? David Lean made "Dr. Zhivago" for probably $20 or $30 million. Now, it costs $200 million to make a pirate movie. Can we follow movies as the logical business example? It seems like they’re making less of a movie for more money?
Garone: I know I’m going off on a tangent here in terms of analogies, but the movie industry has almost gone the same way as Major League Baseball. You’ve got stars that get $10, $20, $30 millions a picture. Does it really matter all that much that there’s an independent producer outsourcing the physical plant requirements to a major studio? I don’t know the industry that well, but my gut feeling is that both parties feel that that’s the most efficient way to do it.
Baer: Actually, can I jump in there, because I’ve heard -- beyond the specifics -- of an entertainment industry where values have gotten hopelessly inflated. But look at the mechanism behind this. Before, you had monolithic studios, which had their own life cycles, their own software development life cycles.
Gardner: Exclusive stars that they’d sign up for years at a time.
Garone: Right, it was the Major League Baseball model.
Baer: Exactly, and it’s sort of analogous to the monolithic software marketing, except for the fact that all the piece parts underneath were more interchangeable.
I guess you can argue that if you are an SAP, your piece parts inside the infrastructure were designed to be a self-contained interchangeable ecosystem. But look at the way it has evolved now. Part of it is that the technology for producing films and videos has become much more accessible, and therefore you have now an independent film industry. So, that's analogous to the infrastructure that has become much cheaper, much more open, which is what SOA can do.
In turn, you look at the business model. It's become more interchangeable, so that a studio might collaborate. Let’s say that Universal might work with Sony -- collaborate on a picture -- because it’s so costly that neither one of them could do it themselves. Well, they have the infrastructure and the business process in place that enables that to happen. Meanwhile, an independent film like "Crash" comes out of nowhere last year and wins the Academy Award for best picture of the year, and it’s all possible because you now have this more accessible infrastructure. The moral of the story for SOA here, trying to bring this back on topic -- is that if you get the plumbing down right -- you don’t have to be an SAP to introduce a killer app anymore.
Kobielus: I’m going to take this movie industry metaphor out of the realm of metaphor into the actuality of, teams becoming very SOA-focused in terms of the actual production ... A mashup is reusing existing components of service -- content, -- whatever into new vehicles or new compositions. If you look at the content that’s being developed out there in the IT world, more of it's getting built through various types of mashups, which is very much an instantiation of the SOA paradigm into a different world -- not the software world so much as the normal cultural world that we all inhabit.
Gardner: It's interesting, because when we talk about movie production, it’s really content creation. It’s artistic. It’s creative, complex, and changeable. It has to fit a certain audience at a certain point in time. A movie that was made 30 years ago might not work today and vice versa. What’s also interesting here, looking through this lens of the movie industry, is that depending on the culture of the organization, it can adjust.
So, if you’re a big monolithic enterprise, you're like a command and control structure, you want to do it all your way. You’ve always done this. You’re sort of a traditional brick-and-mortar company. Then, you can perhaps approach IT like an MGM studio, and you can have your own infrastructure, all of your own developers. You can do things without much off-the-shelf or highly customized -- but you come out with a really great product if you’re willing to put that emphasis in. And, if you have such a great product that you dominate your industry, then the numbers will make sense. The investment you made in IT will pay off.
On the other hand, if you want to be a small company, a green-field software-as-a-service organization, you want to only be in an ecology play where you’re acquiring things, and you’re going to plug them in and then rip them out, reuse as much as you can. Then, you can be like a Miramax instead of an MGM. You’re going to do just independent films. You’re going to go in and spend a little bit of money, and maybe you’ll still come up with some great product that will be right fit for your market. The nice thing is, as we said earlier, you can slice and dice it either way.
Does this make sense -- that the culture of the company needs to be considered when we look at the way in which it approaches IT?
Kobielus: For sure. Some companies are not very partner friendly. Therefore, they’re not really the kinds of corporate cultures that lend themselves to outsourcing. First and foremost is managing the relationships. There are other companies that are very partner friendly, it's really quite often a culture issue.
Gardner: Right. I think we’re going to need to cut off our discussion there, but obviously we can take this further and I’m sure we will, but perhaps one of the takeaways that we’ve led to is that companies need to be introspective. They need to look at what kind of organization they are, what kind of organization they want to be, and get a sense of which IT approaches, particularly vis-Ã -vis SOA and software-as-a-service and outsourcing, make sense for them.
McKendrick: The question is: does management -- does the C level -- understand what SOA is about.
Gardner: I’m not sure we’ve even seen a lot of instances where companies would say, "Let’s take a look at our culture and define it, and then decide our IT approach as a result of that."
Garone: That’s true. They’re going to have to do that. Many of them have not, because they started out looking at SOA from an individual or a pilot project perspective. When it expands beyond there, that will change. It will have to.
Baer: Well, they look at SOA as a technology as opposed to a way of delivering their business services.
Garone: When the technology has been mastered, and has demonstrated its value, it will have to become more than that, and then you’ll see the cultural issues playing a bigger role. A lot of the IT managers and people that I've talked to who are involved in the implementations of SOA solutions, even on a pilot level, seem to understand the cultural issues. They’ve encountered them, and they’ve tried to deal with them. But you’re right. On the C level that hasn’t yet happened.
McKendrick: That goes back to the whole ROI argument we were having a bit earlier as well. The ROI is being seen in developer productivity, for example. If you want to generate hard numbers from an SOA project, developer productivity is the first place you look -- the ability to reuse applications or services without having to have the developers spend X-amount of hours reinventing the wheel.
Gardner: We need to have the equivalent of a Myers-Briggs test, where you actually can ask a series of questions, or do some research, and can define the temperament and the personality traits of a company. From there, you can back up and say, "Well, here’s what your IT should be."
You need to get into the DNA of how a company thinks and behaves organically in order to make the right fit with IT, because the companies that fail are companies where there IT culture and their business culture are at odds or don’t mesh. And they collapse. They can’t function.
Baer: No question about that, and the fact is that, Dana, what you were just saying could apply to almost any type of transformation, but I think this sounds very specific to SOA here. We’ve been providing a whole bunch of metaphors, dealing with other industries like entertainment business or the ability to partner, but if you think about the implications of SOA, it not only should erode barriers between you and your partners. It makes it technologically possible to erode the silos within your organization by going inside and looking at how many instances of the customer object we have? Do we really need 45 instances, when maybe five would do for a business of our breadth and diversity?
Gardner: Okay. Let’s leave it there and we’ll revisit this, but I think metaphors are essential in this period of SOA evolution, because there are so many variables, and because you can slice it and dice it in so many ways. If we can help companies look at ways in which they can reduce the number of variable, and help them get started and find some traction and therefore benefits. That’s an important thing to do.
Joining us in this discussion about the impact of outsourcing, software-as-a-service and how to measure success has been, Steve Garone, the former IDC group VP, founder of the AlignIT group and an independent analyst. Thanks for joining, Steve.
Garone: It’s been a pleasure, Dana.
Gardner: Joe McKendrick, research consultant, columnist at Database Trends, and blogger at ZDNet and ebizQ. Thanks, Joe.
McKendrick: Thanks, Dana, for the opportunity to join you guys.
Gardner: Tony Baer, principal at OnStrategies. Thanks, Tony.
Baer: Thanks, Dana, good way to kick off the New Year.
Gardner: Jim Kobielus, principal analyst at Current Analysis. Thanks for coming, and join us again sometimes, Jim.
Kobielus: Thanks, I’ll look forward to it.
Gardner: This is Dana Gardner, your producer, host and moderator here at BriefingsDirect SOA Insights Edition. Come and join us again next week. Thanks.
If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact me, Dana Gardner at 603-528-2435.
Listen to the podcast here.
Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 7. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Dana Gardner at 603-528-2435.
Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Volume 7, a weekly discussion and dissection of Services Oriented Architecture (SOA)-related news and events, with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions, ZDNet blogger and Redmond Developer magazine columnist.
Our panel this week -- and that’s the week of Jan. 1, 2007 -- consists of Steve Garone, a former IDC group VP, founder of the AlignIT Group, and an independent industry analyst. Welcome, Steve.
Steve Garone: Hi, Dana, and happy New Year.
Gardner: Thanks. Also joining us again, Joe McKendrick, research consultant, columnist at Database Trends, and a blogger at ZDNet and ebizQ. Welcome back, Joe.
Joe McKendrick: Thanks, Dana, I’m glad to be back, and happy New Year as well.
Gardner: Also, Tony Baer, making a second appearance on this show, principal at onStrategies. Thanks for joining, Tony.
Tony Baer: Happy New Year, Dana.
Dana Gardner: Thank you, and making his SOA Insights debut, Jim Kobielus, he’s a principal analyst at Current Analysis. Hi, Jim.
Jim Kobielus: Hi, Dana.
Gardner: Also, welcome to our listeners. Our topics for today span the SOA spectrum of business and technology. We’re going to be discussing metrics of success, ROI, TCO, how do I know I’m doing? I guess we could call this the Ed Koch approach to SOA: "How am I doing?" Yet we can’t really judge how well SOA is doing, if we don’t have metrics.
We’re also going to work on the impact of SOA on outsourcing: outsourcing of IT, outsourcing of business process. Is this an accelerant? Is this a detractor? What’s the impact? We’re going to explore that. Also, the impact of SOA on services -- software as a service (SaaS) in particular; business applications coming across the wire.
So first off, let’s go back to the metrics topic. Someone out there, and I believe it was you, Tony, mentioned that at one point recently Verizon had come out and said that with a stable of 500 services that they were expecting to yield $20 million in savings over two years. Can you tell us where that information came from and how credible you think it is?
Baer: Good question. I actually spoke with the person who went by the title Executive Director of Strategic Systems at Verizon. Essentially, he reported to the CIO and he was in charge of their SOA program.
He gave a presentation on "SOA In Action." "In action" being two words there. I want to be very careful about that. He gave some metrics. I spoke with him about six or eight months ago, so if there is current data, I’m sure it’s different. But this has been going at Verizon for roughly a couple of years. He spoke of creating in excess of over 500, what he called "unique," services.
Now, I’m always a little skeptical when I hear the term "services," because what does that really mean? The definition can be very loose. It could be we have some functionality here and we’re somehow exposing it, or exposing it using standards, using official Web services standards.
Anyway, his point was that these 500 services, which were developed or reused by a community of over 7,000 developers, now account for -- when I say "now," this is as of six or eight months ago -- over 8.5 million transactions per day. He claims that within a couple of years it has saved the company $20 million. I would presume this is in development time, as opposed to having to create each of these things and reinvent the wheel, which is the traditional way of going about it.
My eyes tend to glaze over at such large numbers without having any greater detail, but the fact is, it’s always been a central tenet of reuse that if you can utilize something and not have to create it from scratch, you theoretically should save money and time. So, my sense is that I think the numbers are probably a bit high and a little loose, but I think they point to the potential that SOA can provide in delivering real savings.
Gardner: I wonder if the figures are high, especially on the number of services. I’ve talked to several companies that seem to think that even a smaller number of services could handle just about all of their back-office and front-office needs, perhaps not in vertical usage or industry usage. Then, for a company like Verizon, with the kind of IT budget they have, $20 million might actually not be that much.
Let’s go to Steve Garone. You’ve talked to a number of companies, and you were looking at ROI and business metrics, do these numbers make sense to you?
Garone: The numbers really aren’t out of the ordinary, but I think one of the challenges that organizations face in doing SOA and then trying to determine their ROI -- for want of a better way to put it -- isn't unique to SOA. It’s really a function of various other types of IT-related activities as well. How do you actually quantify what your ROI is, given the advantages of using an SOA approach? I’ve listed the main reasons why people would want to do SOA, in terms of the advantages, and they basically break down to four major areas.
First is what we just mentioned, which is the reuse of IT assets. That's fairly easy to quantify. As I’ve talked to customers and people who have implemented SOA, I find that this is one that they’re comfortable with and are fairly aggressive about putting numbers to, in terms of what they’ve actually saved. It’s not an easy task, because what actually gets saved when you reuse an asset? You certainly save them some development time and development cost. Do you also save maintenance cost, and how do you quantify that? Is it easier to upgrade assets, and to what extent is that dependent on the infrastructure you have in place?
A lot of factors go into it, so I see various levels of rigor. You see people claiming that they were able to reduce the expense associated with doing the application integration test they normally would have to do. That one is a real tough one, because just understanding what you would have had to do and breaking that down into, not only some specific cost, but also the relationship of the result to being able to do business better, faster with more customers and so on, is a real tough task to accomplish.
The other ones have to do with meeting compliance requirements, which we all know are growing by leaps and bounds every day.
Gardner: Regulatory compliances?
Garone: Yeah, and there’s an area where, again, I think the quantification of benefit might be a little bit easier because the requirements are fairly well identified. In fact, they are very well identified.
The fourth one, which I think is the hardest one, is the issue of how agile do you make your business. It’s a buzzword we hear all the time around SOA. That’s a real tough one because you have to ask such questions as: "If I didn’t do this, how many customers that I'm doing business with now, will I not be doing business with, and what affects that? How do I know what my reach was before versus now? How do I quantify how easy it was for a customer to get to me, and if they do, how comfortable they were given what I presented to them, doing business versus leaving because things were taking too long?"
Gardner: So, most of these are soft measurements, and they would take place over a long period of time, because it’s a journey, and you can’t just say, "Well, we went one quarter. What's our payback?"
Garone: That’s right.
Gardner: Now, let’s go over to Joe McKendrick. These are soft paybacks, and it also seems as if we’re comparing apples to oranges. If we look at mainframe, client-server, Web, and then SOA, which is such a different beast, you can’t really compare. It’s almost a leap of faith. Where do you see the right business metrics coming and getting people onboard in terms of dollars and cents here, Joe?
McKendrick: Well one thing I’ve said a lot on my blog is that the companies that are most likely implementing, or are implementing, SOA are probably the companies that don't really need SOA as much. The companies that really could use SOA in their operations are likely not to be the ones implementing, it’s kind of a paradox.
The reason that is the companies that have the management vision, the support to roll out and move an SOA project forward -- providing top management support, for example -- are likely to have lot of other initiatives on the table. They’re likely to be very forward-looking with their management. Their compensation structure, for example, is likely to be a bit more advanced. They’re likely to have other implementations -- a data warehouse, for example -- a very advanced data warehouse. Essentially what you have in these companies is an orchestra making a lot of beautiful music and SOA may be the horn section, but how do you measure the impact of SOA versus another type of implementation or another type of project?
Gardner: Okay. So the song, the piece of art that you’re creating with this orchestra rises up the charts, and if they're a hit, is it because of the woodwind. Is it because of percussion? Is it because of the conductor? We really don't know. We just know that it’s a hit. Right?
McKendrick: Exactly. There are a lot of case studies out there now. A couple of prominent ones are IBM and HP, who had cases of eating their own dog food. They’ve been able show some terrific significant impact from SOA or service deployment. IBM claims it’s reduced its application portfolio by 75%.
They have one quarter of the applications running their business versus back in 1998 when they really first started looking at this. HP claims it's seeing a payback of about $70 million as result of its SOA efforts. These are companies that tend to be forward thinking. They know that they’re being closely watched, and that their own operations are being closely watched.
In both cases, the benefit is coming from consolidation. They’re using SOA as a lever to help consolidate data centers, various far-flung parts of their enterprises. SOA is just part of that consolidation picture. There’s also virtualization going on, server consolidation. IBM, of course, is big with the mainframe, and they’re consolidating a lot of their operations on Linux running on the mainframe. It’s very much a mixed picture, and as I said, Dana, it’s a matter of being able to separate what’s making this beautiful music.
Gardner: I’ve spoken to HP and IBM as well, and they’re really undergoing IT transformation, probably business transformation, and there are many constituent parts to that, of which SOA is one. But SOA is one that has, I suppose, a lot of interdependencies and effects across many of these other activities, whether it’s server consolidation, application modernization, IT shared services, virtualization and what have you.
Let’s go over to Jim. Jim, if SOA is important, almost like a root system that cuts across a number of different trees that are growing, is it even worthwhile measuring it, or should people just be smart enough to recognize that this is the right thing to do?
Kobielus: That’s an interesting metaphor there -- SOA as a root system. My visual image of SOA is a very complex hyper-mesh. In other words, like a root system, where you have tendrils going hither and yon, the tendrils being simply interactions among services and client.
It’s very worthwhile to measure the ROI of SOA as a paradigm or an approach for enabling and for maximizing the sharing and reuse and interoperability of distributed resources across your network. You make an investment as an organization, as an enterprise, and in this approach you want to know whether you’re investing your funds and your resources wisely. When I think of SOA’s ROI, I think of two numbers, and those numbers are 100 and zero.
As we know, SOA focuses on how you maximize the sharing and reuse of services, of application functionality and resources. In other words, how do you enable a 100 percent reuse as a nirvana? We’ll never get there, but in any given organization, 100 percent reuse, service reuse, first and foremost is a consolidation topic. What that means is, if you do SOA right, you’re doing much more with much less.
You’re able to consolidate redundant silos of application functionality and data throughout the organization. You’re able to consolidate fewer software licenses and servers, with the associated translation and cost savings and capital on operating budgets, fewer redundant software components and so forth. The need for fewer programming groups, as we can consolidate that as well.
So 100 percent reuse is the nirvana. The zero comes in the sense that, if you’re doing SOA right, the marginal cost of billing the next application drops pretty close to zero. You’re able to reuse everything that’s already been built. You do not have to reinvent the wheel. So, basically, a 100 percent reuse means zero marginal cost of building the next application. Of course, as I said, you enable that vision through consolidation, both in software and hardware, and in programming teams, and so forth.
So, once again, getting back to your root-system-and-tree metaphor, SOA becomes this ubiquitous root system from which new sprigs can pop up, without needing to lay down their own root system. Rather they are simply branches on a huge underground system. In Northern Michigan, where I’m from, scientists have discovered the world’s largest organism as a mushroom or a fungus of some sort that spans 30, 40, or 50 square miles. They determined though DNA analysis that it's the exact same individual and has got the largest biomass in the world. In essence -- and it’s all underground pretty much. That’s what SOA is all about, essentially all the services in an SOA sort of share a common DNA.
Gardner: Well, there’s the message we need to take to the CEOs and the CFOs. Let’s make our IT like a fungus.
Kobielus: I think they probably already believe that!
Gardner: Let’s take this now to the outsourcing side. You mentioned between zero and a 100 percent in terms of reuse. That would be reuse for your internal resources, but what about reuse of external services? Things that are made available in a marketplace, because of the effects of economics and supply and demand in competition, could push the prices down while even enhancing the quality of services.
For those commoditized services that everyone uses, that really don’t differentiate anyway, why not go in that direction? What’s the relationship here, Steve Garone, between capturing ROI and lowering total cost of ownership, and starting to look toward outsourcing an SOA as being somehow a tag team?
Garone: Well, actually I think there is a fairly close relationship. Again, it’s not unusual that the drivers are really the same as they are in the case of other types of IT investments. Outsourcing can potentially drive costs down, increase quality, allow you to lighten up your IT staff, and so on. So, the same rules apply here. If you look at what’s going on, there are actually several companies now that are doing this. And, it’s not just commodity services. Look at companies like Salesforce.com, for example, which is essentially creating a services-based hosting environment, not only for their applications, but for applications and services that you actually produce in house.
Gardner: Incidentally, we should expect from Amazon.com similar activity around a retail and e-commerce sector.
Garone: I wouldn’t be surprised. So, the drivers are the same, and I think it’s going to happen. I think that if you’re talking about outsourcing services from the hosting standpoint, that’s certainly real and it’s certainly happening now. If you’re talking about outsourcing services from the development and IT support standpoint, meaning how do I go about building my SOA environment, there’s going to be a stage initially where there’s going to be a lot of that, for the same reasons people outsourced that kind of work before.
People look toward SOA and Web services to simplify their application integration environment. That may, in fact, be the case, but in the short term, people still really need to know how to do it and need, in some cases, to outsource for the expertise to get that done.
Gardner: It seems to me that in the not-too-distant future being able to access commodity-level services that you can incorporate into your SOA activities, off the wire, in a competitive environment, would be a really big economic benefit, to reduce your total costs. Maybe not grow your top line right away, but certainly cut your overall cost, which has been an important element of IT for certainly the last five years.
Let’s take this another step sideways. What about IT outsourcing? We had this phenomena a few years ago. Some of the largest entities on Wall Street -- and I’m thinking of JPMorgan Chase -- went to large companies like IBM and said, "Here, you take it. We don’t want to do IT anymore."
Then maybe a few years, even some months, later decided, "Well, maybe that’s not the right approach." If we can get into a marketplace of services, if we created an SOA, doesn't that really obviate the need for IT outsourcing? Don't you want to retain the competency of business process and composite application authoring, regardless of where those sources come from? Isn’t SOA and the outsourcing of services a better model than wholesale IT outsourcing? I’ll throw that to you, Tony.
Baer: I was just thinking back to what Steve was saying, and it kind of applies to this answer as well. If there is one benefit that SOA delivers, it’s that the value becomes the service rather than the plumbing. If you think about the way we've traditionally developed functionality or integrated systems, we’ve had to spend inordinate amounts of time in the plumbing and maintaining it. SOA theoretically, if it’s done right, standardizes the plumbing, makes everything declarative, so you take out the guess work. The result is that if you look at outsourcing, SOA separates the plumbing from the service. Therefore, what is probably ideal for outsourcing would be the plumbing, because that’s where the value is and that’s not where IT organizations should be spinning their wheels.
Gardner: So where is the value?
Baer: The value is in the -- and this will sound a little cliché-ish -- intellectual property, which is represented by the actual service. That’s the service that delivers the actual business logic. It’s the way a company does business. The way a company does business is not the way it puts together its SOA plumbing. That should be standard. That’s obviously an ideal scenario, but look at how this could impact the outsourcing market. If you follow SOA principles and comply with SOA standards, it allows for that kind of separation, and therefore it makes for an ideal opportunity for outsourcing services to say, "Hey, we’ll take care of your plumbing, and you take care of your services."
Gardner: So, you’re saying perhaps that I should look to commoditize services and find the cheapest best way to acquire them, either on premises, co-located, outsourced entirely, software as a service (SaaS). Then, for those differentiating services that I’m applying intellectual property and perhaps even patents to, then I could even host those somewhere off premises, and it’s bringing that together within my organization that makes me a winner in my market?
Baer: That’s one way of putting it, but another way is, "I create the service, and by the way I rely on you to create and maintain the plumbing." There are many different ways of slicing and dicing this.
Garone: To some extent it depends on what kind of vendor you’re dealing with. I’ll go back to the example I gave earlier of Salesforce.com, which will do some of both. They create a services-based infrastructure that you can use to not only host your IP-unique applications in services, but also to host the ones that they’re providing to you, and if you do it right, you can easily integrate with.
Gardner: Right.
Kobielus: On the issue of outsourcing, the only thing that the company should never consider outsourcing in terms of core competencies is the core competency of making money and delivering value. Everything else is fair game, and I think what everybody is saying here is that they agree with that. SOA -- or much of what we think of as SOA -- revolves round the plumbing, the protocols, the application servers, the middleware fabric, etc., that all is fair game for outsourcing.
Gardner: I like the terms that were used a moment ago. It can be "sliced and diced in a number of different ways." That’s the good news, but it’s also the bad news, because no one really has a model they can fall back on and say, "Ah, here’s the proper way to do this, when we apply outsourcing and software as a service to the SOA mix." Or have I got this wrong? Jim, can you think of any past instances in the history of IT we can look at to get some idea of the best way to slice and dice these many variables?
Kobielus: Going back to my sense of knowledge of IT history, I’m going to have to sleep on that one, Dana.
Gardner: Can anyone else out there think of any precedents that we can look to in terms of what’s the right balance of on-premises, commoditized, software as a service, SOA? It gets kind of messy.
McKendrick: I don’t know if there really is a precedent. This is all open for innovation, Dana. There are a couple of interesting examples, actually, if you look at the telecom industry. I like to use the term "loosely coupled business." SOA is based on loosely coupled components or services. A lot of telecoms actually don’t use their own internal resources to provide services to their external customers. They act as a broker.
Gardner: It shows sometimes.
McKendrick: Unfortunately, yes. I’ve heard a statistic that at Cisco, the networking giant, and 80 percent of its products and services come through outside services that it essentially brokers. I think that’s kind of the model SOA is helping us move toward -- this idea of a loosely coupled business that can pull together services from perhaps throughout the globe, package them, and offer them to an end user or customer that they identify as important to their market.
Gardner: So, maybe the metaphor has moved from a 30-square-mile fungus to a 30-square-mile amoeba.
Koblieus: Amoeba is a good metaphor here, because business models must continue or will always continue to evolve; therefore IT must continue to evolve to the next business model, the next IT sourcing model, whatever helps them to survive and make money.
Gardner: We came up against this when we talked to Jeff Pendleton. We thought about architecture and we looked at some of the architects, and we came to Frank Gehry, and we said, "Wow, the outside looks different and almost illogical, but inside, the stuff holding it together is very logically designed."
Then we moved from that to this notion of a sound stage. You would build a sound stage, or even a movie set, depending on whatever scene you were shooting. And, we wanted to have IT act like a flexible sound stage or a movie set. We would just build the set based on whatever the activity was, do what we needed to do, and then perhaps tear it down or reuse parts of it. So, we seem to be coming back to that, although we’re talking about not just our own services as an enterprise on premises, but increasingly a mixture of software as a service and outsourcing.
Anybody want to pick at that or expand on it?
Gardner: Well, we're in complete agreement, that’s great.
Garone: I’m not quite sure I understood it, to be honest with you.
Gardner: Do you remember that call we had about the sound stage show; I think you were on that call?
McKendrick: It’s an interesting analogy.
Gardner: MGM right?
Baer: Right. That actually is about the old movie studio metaphor, one studio and many different productions going on at the same time. Basically, the piece parts were all interchangeable, and piece parts we can call plumbing.
Gardner: So, we’re making five different movies on the lot, and we’re going to bring cameras and lighting equipment and backdrops and makeup artists, and just apply them in the same way, but with a different end outcome. Right?
Baer: I guess you can say the metaphor coming out of this discussion has evolved from fungus to amoeba to let’s bring back the studio system into IT.
Kobielus: Right, if you look at what SOA governance involves, if you examine a given service throughout its life cycle, it begins in the planning stage, it’s a gleam in some business person’s eye, and then it goes to the system engineers, the designers the business people. There’s a governance life cycle on those projects as well. That governance role structure in the movie industry has changed since the very start and it continues to evolve.
The notion of the classic studio system for Hollywood was there between the '20s and the '50s, and then it gradually gave way to more independent producers. Now, you’ve got all these indies everywhere. What I’m getting at is that, the governance of any given movie production project, how it’s done, the best practice for that particular industry, continues to evolve from generation to generation.
Likewise in the IT world. If you look at the life cycle of governance of the creation of any application or functionality, of the whole software is on the life cycle, that paradigm continues to evolve from generation to generation, with new platforms, new tools, and so forth. So, its one of those things where now you’ve got to look at the structure of the SOA governance environment.
You've got roles that are specific to the plan, specific to the development stage, roles that are specific to the deployment and optimization stages and so on. What I’m getting at, then, is that I think the movie studio analogy is very interesting, because it focuses on the core of governance as a workflow over a life cycle involving various roles, and those roles continue to munge into each other and get broken off from each other.
Garone: Dana, just jumping in here, now that I had a chance to think a little bit about your analogy, and just to follow up on that last thought. In essence, the movie studios as we know them now, don’t have the role that we used to know them as having. They used to actually make movies. My understanding is that the customer who is the independent producer, or maybe a larger producer who may bring on another independent producer under them, basically is outsourcing the physical plant requirements to make a movie to the studio.
Gardner: So it becomes an ecology rather than all-in-one.
Garone: Exactly. It’s an ecosystem.
Gardner: Is it the right direction, Steve? David Lean made "Dr. Zhivago" for probably $20 or $30 million. Now, it costs $200 million to make a pirate movie. Can we follow movies as the logical business example? It seems like they’re making less of a movie for more money?
Garone: I know I’m going off on a tangent here in terms of analogies, but the movie industry has almost gone the same way as Major League Baseball. You’ve got stars that get $10, $20, $30 millions a picture. Does it really matter all that much that there’s an independent producer outsourcing the physical plant requirements to a major studio? I don’t know the industry that well, but my gut feeling is that both parties feel that that’s the most efficient way to do it.
Baer: Actually, can I jump in there, because I’ve heard -- beyond the specifics -- of an entertainment industry where values have gotten hopelessly inflated. But look at the mechanism behind this. Before, you had monolithic studios, which had their own life cycles, their own software development life cycles.
Gardner: Exclusive stars that they’d sign up for years at a time.
Garone: Right, it was the Major League Baseball model.
Baer: Exactly, and it’s sort of analogous to the monolithic software marketing, except for the fact that all the piece parts underneath were more interchangeable.
I guess you can argue that if you are an SAP, your piece parts inside the infrastructure were designed to be a self-contained interchangeable ecosystem. But look at the way it has evolved now. Part of it is that the technology for producing films and videos has become much more accessible, and therefore you have now an independent film industry. So, that's analogous to the infrastructure that has become much cheaper, much more open, which is what SOA can do.
In turn, you look at the business model. It's become more interchangeable, so that a studio might collaborate. Let’s say that Universal might work with Sony -- collaborate on a picture -- because it’s so costly that neither one of them could do it themselves. Well, they have the infrastructure and the business process in place that enables that to happen. Meanwhile, an independent film like "Crash" comes out of nowhere last year and wins the Academy Award for best picture of the year, and it’s all possible because you now have this more accessible infrastructure. The moral of the story for SOA here, trying to bring this back on topic -- is that if you get the plumbing down right -- you don’t have to be an SAP to introduce a killer app anymore.
Kobielus: I’m going to take this movie industry metaphor out of the realm of metaphor into the actuality of, teams becoming very SOA-focused in terms of the actual production ... A mashup is reusing existing components of service -- content, -- whatever into new vehicles or new compositions. If you look at the content that’s being developed out there in the IT world, more of it's getting built through various types of mashups, which is very much an instantiation of the SOA paradigm into a different world -- not the software world so much as the normal cultural world that we all inhabit.
Gardner: It's interesting, because when we talk about movie production, it’s really content creation. It’s artistic. It’s creative, complex, and changeable. It has to fit a certain audience at a certain point in time. A movie that was made 30 years ago might not work today and vice versa. What’s also interesting here, looking through this lens of the movie industry, is that depending on the culture of the organization, it can adjust.
So, if you’re a big monolithic enterprise, you're like a command and control structure, you want to do it all your way. You’ve always done this. You’re sort of a traditional brick-and-mortar company. Then, you can perhaps approach IT like an MGM studio, and you can have your own infrastructure, all of your own developers. You can do things without much off-the-shelf or highly customized -- but you come out with a really great product if you’re willing to put that emphasis in. And, if you have such a great product that you dominate your industry, then the numbers will make sense. The investment you made in IT will pay off.
On the other hand, if you want to be a small company, a green-field software-as-a-service organization, you want to only be in an ecology play where you’re acquiring things, and you’re going to plug them in and then rip them out, reuse as much as you can. Then, you can be like a Miramax instead of an MGM. You’re going to do just independent films. You’re going to go in and spend a little bit of money, and maybe you’ll still come up with some great product that will be right fit for your market. The nice thing is, as we said earlier, you can slice and dice it either way.
Does this make sense -- that the culture of the company needs to be considered when we look at the way in which it approaches IT?
Kobielus: For sure. Some companies are not very partner friendly. Therefore, they’re not really the kinds of corporate cultures that lend themselves to outsourcing. First and foremost is managing the relationships. There are other companies that are very partner friendly, it's really quite often a culture issue.
Gardner: Right. I think we’re going to need to cut off our discussion there, but obviously we can take this further and I’m sure we will, but perhaps one of the takeaways that we’ve led to is that companies need to be introspective. They need to look at what kind of organization they are, what kind of organization they want to be, and get a sense of which IT approaches, particularly vis-Ã -vis SOA and software-as-a-service and outsourcing, make sense for them.
McKendrick: The question is: does management -- does the C level -- understand what SOA is about.
Gardner: I’m not sure we’ve even seen a lot of instances where companies would say, "Let’s take a look at our culture and define it, and then decide our IT approach as a result of that."
Garone: That’s true. They’re going to have to do that. Many of them have not, because they started out looking at SOA from an individual or a pilot project perspective. When it expands beyond there, that will change. It will have to.
Baer: Well, they look at SOA as a technology as opposed to a way of delivering their business services.
Garone: When the technology has been mastered, and has demonstrated its value, it will have to become more than that, and then you’ll see the cultural issues playing a bigger role. A lot of the IT managers and people that I've talked to who are involved in the implementations of SOA solutions, even on a pilot level, seem to understand the cultural issues. They’ve encountered them, and they’ve tried to deal with them. But you’re right. On the C level that hasn’t yet happened.
McKendrick: That goes back to the whole ROI argument we were having a bit earlier as well. The ROI is being seen in developer productivity, for example. If you want to generate hard numbers from an SOA project, developer productivity is the first place you look -- the ability to reuse applications or services without having to have the developers spend X-amount of hours reinventing the wheel.
Gardner: We need to have the equivalent of a Myers-Briggs test, where you actually can ask a series of questions, or do some research, and can define the temperament and the personality traits of a company. From there, you can back up and say, "Well, here’s what your IT should be."
You need to get into the DNA of how a company thinks and behaves organically in order to make the right fit with IT, because the companies that fail are companies where there IT culture and their business culture are at odds or don’t mesh. And they collapse. They can’t function.
Baer: No question about that, and the fact is that, Dana, what you were just saying could apply to almost any type of transformation, but I think this sounds very specific to SOA here. We’ve been providing a whole bunch of metaphors, dealing with other industries like entertainment business or the ability to partner, but if you think about the implications of SOA, it not only should erode barriers between you and your partners. It makes it technologically possible to erode the silos within your organization by going inside and looking at how many instances of the customer object we have? Do we really need 45 instances, when maybe five would do for a business of our breadth and diversity?
Gardner: Okay. Let’s leave it there and we’ll revisit this, but I think metaphors are essential in this period of SOA evolution, because there are so many variables, and because you can slice it and dice it in so many ways. If we can help companies look at ways in which they can reduce the number of variable, and help them get started and find some traction and therefore benefits. That’s an important thing to do.
Joining us in this discussion about the impact of outsourcing, software-as-a-service and how to measure success has been, Steve Garone, the former IDC group VP, founder of the AlignIT group and an independent analyst. Thanks for joining, Steve.
Garone: It’s been a pleasure, Dana.
Gardner: Joe McKendrick, research consultant, columnist at Database Trends, and blogger at ZDNet and ebizQ. Thanks, Joe.
McKendrick: Thanks, Dana, for the opportunity to join you guys.
Gardner: Tony Baer, principal at OnStrategies. Thanks, Tony.
Baer: Thanks, Dana, good way to kick off the New Year.
Gardner: Jim Kobielus, principal analyst at Current Analysis. Thanks for coming, and join us again sometimes, Jim.
Kobielus: Thanks, I’ll look forward to it.
Gardner: This is Dana Gardner, your producer, host and moderator here at BriefingsDirect SOA Insights Edition. Come and join us again next week. Thanks.
If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact me, Dana Gardner at 603-528-2435.
Listen to the podcast here.
Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 7. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Sunday, January 14, 2007
Transcript of BriefingsDirect Podcast on the Future of Enterprise Search
Edited transcript of BriefingsDirect[TM] podcast on search advancements with host Dana Gardner, recorded Jan. 3, 2007.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today a sponsored BriefingsDirect podcast discussion on search, the role of search, how search relates to today’s mega trends of Web 2.0, Enterprise 2.0, Services Oriented Architecture (SOA) and even mobile lifestyles.
Joining us in this discussion are executives from FAST Search & Transfer, a global company based in Oslo, Norway. Joining us is John Markus Lervik, the CEO. Also, Bjorn Olstad, CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. Welcome to the show everyone.
John Markus Lervik: Good morning, Dana.
Gardner: While search has been around for many years, it seems like we’ve had somewhat of a philosophical shift in the last few years, in the sense that people are no longer thinking of information based on its origins, but really they’ve supplanted these data structures with how they can use it and how they can access it. That cuts across desktop, intranet, and Web; and it seems that access to information quickly and in the way that any individual wants it has really changed the notion of fleet and agile business.
In fact, business process is increasingly being driven by how people access content and then inject that into business goals. So, I want to first talk to you about business goals because many companies are very practical, of course. They’re looking at their top line and they’re looking at their bottom line. I want to throw this out perhaps to John Markus, how are companies using information discovery to further their business goals in a way now that’s perhaps different from ever before?
Lervik: Dana, I think that one way of looking at technology is always in the light of the business processes that they’re enabling. Many of our customers are thinking about ways in which they can achieve competitive advantage through productivity change, and they’re looking for something new. What we enable is for search to help facilitate communications between individuals.
We’ve all heard of Web 2.0 applied to the "Websphere," but in the enterprise, Enterprise 2.0 trends of social networking are also taking hold, allowing people to collaborate better and allowing individuals to streamline business processes. What's interesting for us in the future is search as an entry point into facilitating improved business processes, and business process improvement leads to improved business goals, as well as business-model innovations.
Gardner: Is there an impact here for applications themselves to start to be created based on what search can bring to the table rather than having search perhaps be an afterthought? What do you see as the future of search-driven applications?
Bjorn Olstad: This is Bjorn, and I'll have a go at that. The fundamental difference between search and such things as database, content management, and document management systems is that search starts with the user and then does reverse engineering -- what is necessary to realize this user's experience. It's starting with the content and then trying to deduce what should we do with this content. When you do that, you end up with a user-driven experience.
As you’re starting off, Dana, the user doesn’t care about the origin of the content. So, the ability to go into content sources and then chop that into information nuggets that can then be re-orchestrated and put into the context of what is useful for that specific individual user in what he’s trying to accomplish right now is a very powerful paradigm. It’s also a paradigm that can be used as the basis for building applications.
We see search more and more becoming the framework for the metaphor in how the user interface is being defined and also for how the application is actually defined by the source of the framework and then embedding functionalities in a source-stream framework.
Gardner: Is this really a shift of intent or are there some technologies that are newly arriving that empower this shift? Bjorn, can you tell us a little bit more about what’s happening technically that is allowing an expansion of the role of search?
Olstad: Let's go back a little bit. As you said, search used to be an afterthought. Typically, you had an application that was kind of hard-wired, done in HTML, or hard-coded down to the logic that usually resided on some type of database. So there was hard-wiring between the schemas and the data model and the database, and then the user interface.
Going forward, I think increasingly the user experience will be driven by algorithms and will be dynamic, so that you can actually optimize the user experience and the business efficiency. This is valid for both consumer-facing portals that you’ve seen on the Web but also on other applications internally. So then you get that framework and you say. "Let’s not hire system integrators to hard wire my content. Let the user decide, and when we deduce what the user likes, the algorithms can populate the user experience."
This will include both content and relevant services, and maybe a connection to people, so that you can find people to collaborate with. That becomes a framework, and search has matured from being merely access to documents to being this umbrella environment where you can provide access both to structured data and unstructured data and transform them into structure by putting additional metadata onto it. You can do the same with rich media, and search suddenly becomes that framework that can give you access to everything, as opposed to just the text that didn’t fit into the hard wire or the schema model.
Gardner: I suppose that with the increasing use of SOA, where applications are decomposed into services with the productivity coming from reuse as much as possible, and with governance providing an overlay for managing and organizing services, that search could have quite a powerful impact on how services are put together on the fly to create unique business processes. Do you see search moving in that direction as well?
Olstad: Absolutely. If you think about Web 1.0, Web 2.0 and Web 3.0, one way to interpret this is that Web 1.0 included these monolithic Web services. Then, in Web 2.0, it’s about the different services that make up atomic individual services and then having technologies to mash up and combine them to create new services on top of that, the same way as SOA. It seems nice at first because they are driven toward reuse, et cetera, but you move the complexity to the next level. How do you actually orchestrate? How do you put together these atomic content components and atomic service components? How do you actually do that and how to do that in a user-centric way, so you can gain the right user experience?
With search, it’s actually designed to do exactly that, to bring both components and services to atomic units and then bring the orchestration level that is able to recompile these content and service elements into what makes sense for the end user. Hence, it brings 3.0 integration levels on top of the atomic components that we have in Web 2.0.
Zia Zaman: This isn’t just our idea, if I can build upon that. I was talking to an information provider that provides information to financial services firms. What he asked for was the ability to atomize information, allowing the users the flexibility to consume it in their own way. So, it’s exactly what Bjorn was talking about. In Enterprise 2.0, we allow for the atomization. And then the re-organization around the business processes is what happens in the next phase.
Gardner: Zia, there is already quite a bit of difference of opinion on how to define Web 2.0, never mind Web 3.0, but let’s try to just get a handle on Web 3.0 for our audience. To me this is really getting towards the Semantic Web, where there is context on an automated basis, with much more rich metadata involved, and therefore the ability to apply algorithms more generally across the Internet. Is that your understanding at FAST about this Web 3.0 or how would you further that definition?
Zaman: Dana, I think you've got it right. Metadata is the key, but I'm going to pass this one over to Bjorn to talk a little bit more about Semantic Web.
Olstad: There are so many opinions about the Semantic Web. First of all, what I don’t like with the approach of Semantic Web is that you have to put in structure, and it’s not really useful until almost everybody has started to put structure into it.
So, how do you leap from that and actually make it useful before everybody has adopted this new scheme? Search can actually play a role, because search can auto-generate metadata and find ways to use that structure and to improve the discovery. Then, the allocation elements that the traditional Semantic Web talks about can be aimed at how to improve algorithms, as opposed to starting from scratch. In doing that, I think search has the opportunity to deliver on the premise of the Semantic Web, by applying algorithms as opposed to altering tools.
If you then move to Web 3.0, then using algorithms increasingly becomes a component of it. I think an immediate step, which is an opportunity that FAST is looking at right now, is how can you use search at this level to go from atomic services to the rich environment, where you effectively can develop and deploy services. That’s kind of the orchestration: how you put services and content components into meaningful context for the end users. That is what brings business value. It brings time to market value for companies that are trying either to develop Web 2.0 and Web 3.0 applications or move their current models to a more collaborative approach.
Gardner: I suppose one of the frustrations we’ve heard quite often from users is that they can find things easier on the World Wide Web, than on their own hard drive or their own enterprise network. We’ve come a long way to assuage those concerns, but it seems to me like perhaps we’re at a point where we can go and leapfrog the Web.
If we’re thinking about a Semantic Web, that’s something that requires a lot of standardization. It could take a number of years and a high level of complexity, but moving to the "Semantic Enterprise" seems something that’s quite a bit more attainable in the short term. Do you think, and I’ll throw this out to anyone, are we going to move to a point where we can apply these benefits to the Enterprise and then the Web will catch up to them or is the Web always going to be a bit easier to manage?
Olstad: Maybe I could start, and, Zia, you could give some business examples. This is an excellent question, Dana. First of all, just so that we’re on the same page about what the Web experience is all about, there are two types of queries. There is the type of query where you know what you’re looking for. So you search for some properties and what you’re actually looking for.
The other type of query is when you don’t know what's there, but you throw out a wide, open-ended query, and you want the data to talk to you. In Web search, there is no completeness requirement. Web search is defined when you type in one or two query terms, which is a nice starting point for your browsing. So you get a link to a Website where you can start. There is no completeness. There is no overview of the other elements that didn’t end on the first result page.
Web search has moved outside Web search and can be used inside the Enterprise for this semantic analysis. It can also identify facts inside documents, combine it with the information structure databases, and provide discovery. You can start to do analytics, so that you can relate concepts and enable people to make decisions and get completeness and overview. That is why you see a merger between business intelligence and search starting to appear.
Gardner: So, perhaps moving toward a business intelligence (BI) benefit vis-Ã -vis search, and of course the other tools that are available for BI, is it first a necessary step to the Semantic Enterprise?
Olstad: Yes, I think so. To give just one example of a company that is trying to use search in an innovative way, what about all communication internal in the company? What about instant messaging, email, Voice over IP (VOIP), Skype, et cetera, and how to improve the quality of that communication and also derive analytics from it.
For example, if I call you, Dana, I should call you as a conceptual entity and you should actually be the one who decides if that ends up as a Skype call or cell phone call. This company is doing that and using search to orchestrate all the communication, allowing you, as the recipient of the call, to configure how you want to be accessed. By having that system, you can also create analytics that understand what is really happening in the company: who are the experts, what’s happening, what are the trends, etc.
Zaman: I actually think that it could be a lot simpler than that. The expert location is something that Bjorn just touched upon, which may be one of the next killer apps in the enterprise. What we’re finding more and more when we talk to our customers is that information discovery isn’t just about connecting users to results. It’s about connecting users to answers, to analytics, and to services -- but most importantly, to people. Connecting people to other people, so that they can do their job better. To improve decision-making throughout the enterprise is a big part of what information discovery tools are about to enable.
One way of looking at it is, if I can find the expert in the organization or outside the organization who can help me best in the decision process that I'm currently in -- and I can find that expert in the shortest period of time, using insights about who this person might be by using metadata about this individual, by actually discovering this path that I didn’t know existed upfront -- then I have actually added value to the business. So, we believe that search as a matching service is really one of the killer applications.
Gardner: So, what we’re talking about here is bringing the tacit knowledge that’s available to any organization into play with the semantic knowledge that’s gathered across all modalities, all types of media and information, but allowing those to come together in real-time based on someone’s specific needs through a very simple query.
Zaman: That’s right. I often like to talk about Greek philosophers, and Socrates is one of my favorites. The reason is because of what he did with a seeker. He didn’t answer a question with an answer. Rather he asked another question, and that allowed the seeker to refine his or her question, until finally they got to what they were looking for. In many ways, technology -- whether it’s information discovery, search, and business intelligence, whatever it might be -- shouldn’t insult the intelligence of the seeker. Rather it should allow individuals to find the answer that they’re looking for, either through tacit information that’s stored in the enterprise, or semantic information, or structured information, whatever it might be. What we’re trying to do is mimic the type of dialogue that Socrates had.
Gardner: Sure, because people are quite good at peeling away the layers of an onion, as a metaphor, to find what it is they’re looking for. There is no beginning or end to this peeling, but we certainly want to get him or her close to where they can extract real value.
Zaman: You got it.
Gardner: All right. Now people have been talking about this for years. I deal with developers a lot. I think they understand a lot of these theories. They understand the value and the goals, but putting it into practice, putting it into execution seems to be where there is still a need for some work. When we go to developers, we try to encourage them to start exploiting these capabilities of changing the mindset of how an application, specifically the UI, is actually constructed to make the entry point into an application more amenable for search and for gathering of these layers within an organization’s assets and resources.
Are there tools? Are there APIs? Are there SDKs? Is there a mindset? How do we bring the developer into play here? I’ll throw this out to anyone.
Olstad: I think Gartner predicted that in 2012, 75 percent of the world's applications will use search as the dominant metaphor for how an application is being operated by the end-user. That is a very powerful statement. And it means that all those developers that you’re asking about, Dana, they have to adapt to that.
Simplicity in an application [means] that you have a powerful metaphor that you can use to deduce what you should expect the application to be doing. Search is exactly that. So, obviously, search is evolving to not only provide access to text, but to provide value to all types of elements.
Another change in search technology is that it handles content natively, and previously it did not. In Web search you type some queries and you get a URL link to the content. Now you can take XML or SQL databases, and you can get native content. You can search in native content with queries that emulate what you can do in the database or in an XML database. This means that you can build the same type of application with the freedom to access data in an ordinalistic way, and have a mechanism to prioritize and do exactly what Zia talked about with Socrates.
Technically, if you use an open-ended query like, "List the innovative people in my company," or something, you could get back kind of a menu of people that have been referred to in the documents or discussions where there is talk of innovation, and then get the facts related to these people. So it’s not coming back with an answer, but it is coming back with an analysis -- and giving you the opportunity to refine the query.
Gardner: Another thing with developers is that they’re interested in seeing examples. Are there folks out there doing this now who are really on the cutting-edge, who have been able to instantiate rather than theorize? Maybe, Zia, you could address this. Are there folks that are out there using FAST Search & Transfer technologies or other technologies who are actually realizing some of these benefits? And, who are they, and how are they doing it?
Zaman: All over the landscape of the economy, people are using search in interesting ways. I think that the message to the developer is: "Think about the app that you’re working on right now, and think about how the users are accessing either the application or the information."
Take a consumer perspective and you’ll definitely think that search plays an important part in whatever it is that you’re doing. So, get involved and talk to people about search. I think you will find that, even in your own sphere, there are people who are already working on search technologies. In one particular investment bank, 15 or 16 different applications of FAST Search have been put into place across the business to help with everything from getting access to research reports to improving customer service.
There are search policies at law firms or at pharmaceutical companies or retail or e-commerce searches for Christmas shopping. Or just B2B searches, so that one individual can find another professional in the shortest amount of time in the right region. All these applications are about the user consuming information in a way that is most effective for them -- and search technologies are enabling each of these examples. Maybe John or Bjorn, you want to talk about a European example and I can talk about an American example?
Olstad: Sure. Let me start off with our broad business goals, and there are some that have nice benefits for the businesses. We have many customers that don’t have a choice because if you look at media, entertainment, and telcos, there are so many disruptive changes going on. You have print transforming to electronic. You have advertising on the Internet, where search-based advertising is now bypassing traditional banner advertising. You have telcos, where telephone calls may soon be free because there are alternative ways to monetize it. And you already have Skype, Voice over IP, etc.
So, if what a telco provider, for example, is providing today is going to be free, then the name of the game is going to be who is able to create applications and to have service delivery platforms that use that customer base to derive new services. They’re not very well equipped for that. The key enabling-technology that actually helps with this has got to be the user-driven applications. And they see search from a monetization perspective and from the perspective of building the end-user experience. So for lot of customers that we work with from media, entrainment, and telcos -- it’s not that it's "nice to have," but it's "How do we survive?"
Zaman: An example of that in the back office is AutoTrader.com, which is a phenomenal example of how search was used to fundamentally rethink the way in which records were served up. The database administrator at AutoTrader says that they tripled their performance at half the cost, using a highly scalable search platform, which was very easy to deploy and a lot more dynamic than their traditional database architecture. So, even in the back office for applications as diverse as AutoTrader, we are seeing search being used to really disrupt the way in which you think about information delivery.
Gardner: It also seems that the younger generation, as they come into play here, are probably already automatically thinking "search." The people who have grown up with the Web around them start almost any activity with search. So, this is becoming imbued into the society and into the behavior and culture?
At the same time, these folks are also quite used to mobile devices and small handheld devices. Can you explore for me a little bit of how search plays into young people in mobile devices? Perhaps we’re seeing some activities in early-adopter environments, South Korea for example, where search has to be the only way in a small-form factor that vast amounts of information could be managed. Help me understand the impact of use and mobile search?
Olstad: That’s a very interesting question, Dana. As you said, most exploration starts with a search on the Web. You start with a search on a Web search engine, and then you come to the site of this company. Then that company has some options. Should we let the user continue with the search experience or should we force him to go back and understand the text and how we have orchestrated all our content? Then, he is not allowed to do anything meaningful before he has completed that education. That’s slowing down the progress of the users.
Moving to mobile, if we go back to 2001-2002, we were trying to educate mobile operators that they needed to use search. That was a difficult discussion, a difficult sell. In Europe and Asia, at least, it was very popular to download ringtones and wallpapers and things like that. The operators felt that it was sufficient to have a chart with the most popular items and have them available on the static page, allowing people to choose from that.
When they moved to search, what they experienced was that a good way to measure the UI or the usability for the user is how many operations it took to actually perform the action. One of our customers is Vodafone. They measured how many operations young people had to do, from coming into the portal until they could complete a transaction, which in this case was to download a ringtone or music, etc. That used to be 3.5-operations, or something like that, when it was based on the charts and navigation mechanisms. When they substituted search, it went down to 1.5, which means that almost always you go directly to something that you can download. And, once in a while, you have to do one iteration before you get it. That drove revenue and traffic tremendously.
Zaman: You’re right. Generation Y and the younger generation are very used to unfettered access to their data, their content. What does that mean in the Enterprise? This is something that I hear increasingly on Wall Street and in the City of London. I know how to deal with the managing director, who is in his 40s. It’s very easy. For their relationship with their data, they ask someone junior.
But for that younger investment banker who’s just grown up with unfettered access to their personal data, they keep asking why they can’t get access to enterprise data with the same facility. The information technology people who are serving this younger generation have to change. They don’t quite understand that search and other technologies, including Web 2.0 technologies, are forcing them to rethink the way in which they have access to their information, because that’s the way they can be creative and allow for them to get the data they need in order to make the right decisions. I think it’s having a cultural effect, even in more staid industries.
Olstad: Then, if you go to Japan or to Asia, you have also this competition between the telco providers, where the name of the game now is how do you add video services, and with extremely high bandwidth on the phones, how to do that? You see multimedia screening, where you are serving video quality down to the cell phones. So there is a tremendous interest in how to capitalize on new user experience to combine rich media assets, and then use this to build the community stuff you have seen in YouTube in the U.S. This is happening on the mobile side at a tremendous speed in Asia.
Gardner: Very interesting. So we’re really still in the early stages of search. It sounds like there is going to be a lot to discuss at your forthcoming FASTforward ’07 event. Perhaps as a little tease for our audience, is there anything you can tell us about this event in terms of some additional technologies, products or approaches? I will throw this out to anyone.
Olstad: If I could start from the technology side. There will be a lot of cool introductions and some new technology. We will look at how to interface search with repositories, from structured data to rich media and to traditional text. We think there is a game-changing opportunity in doing that and also to enable the Semantic Web in a more powerful way, as we talked about.
We also really believe that, as we have touched on here, that search is not just a feature, but it will become the framework for building search-driven, user-driven applications, to actually make that a reality and make it possible to build those applications. We are introducing new components to do that.
We see that search is driving the connective role in Web 2.0 in the evolution towards Web 3.0 by being a service-delivery platform. So we are introducing a Web 2.0 platform that uses search as the main orchestration -- both for monetization purposes and for understanding both content and the user. We'll be tracking the user and understanding his preferences, building personalized experiences, and creating this platform where you can take the atomic components of content and services and effectively roll that out as user-centric services. We’re building a service delivery platform around search for doing that and you will see applications that already have been built on top of that platform.
Zaman: I think it’s going to be a very hands-on conference in San Diego. There will be customers. There will be developers. There will be case studies, demos, and a lot of learning going on, interacting and sharing of ideas. Sure, there will be the predictions and the analysts and the advice that goes along with it. But really I think that the value of this search conference will be in the exchange of innovative ideas that takes place, not just from FAST to the rest of the community -- but also from the community to other members within the community. I think it's going to be great.
Gardner: Thanks. John Markus, just to return back to the business value, we’ve been discussing this at a fairly technical level -- developers and business intelligence folks -- but for CEOs, for the leadership at many enterprises, what are some of the salient messages that you think they need to understand in order to better appreciate search and why it makes a good deal of sense to invest in it?
Lervik: In general for most information-intensive companies -- media companies, financial services companies, telecom companies and pharmaceutical companies -- search is becoming both a key part of the infrastructure to enable these companies to make decisions, but also -- more importantly -- to drive and increase business online.
Because we see more and more of these companies relying on effectively connecting customers or end-users to their products, to their contents, and to their services. That’s where search comes in as a mission-critical application, or at least to enable these mission critical applications. At FASTforward, there will be a great number of speakers who will talk about how these have effectively implemented a solution that drive and increase business.
Gardner: I’m afraid we’re about out of time. We’ve been discussing search, the role of search, Enterprise 2.0, Web 3.0. Clearly the role and impact of search is increasing rapidly and will increase further over the coming years.
I’d like to thank our guests and sponsor, FAST Search & Transfer. We’ve been talking with John Markus Lervik, the CEO. Also, Bjorn Olstad, the CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for joining us for this BriefingsDirect podcast.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Transcript of Dana Gardner’s BriefingsDirect podcast on the future of enterprise search. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today a sponsored BriefingsDirect podcast discussion on search, the role of search, how search relates to today’s mega trends of Web 2.0, Enterprise 2.0, Services Oriented Architecture (SOA) and even mobile lifestyles.
Joining us in this discussion are executives from FAST Search & Transfer, a global company based in Oslo, Norway. Joining us is John Markus Lervik, the CEO. Also, Bjorn Olstad, CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. Welcome to the show everyone.
John Markus Lervik: Good morning, Dana.
Gardner: While search has been around for many years, it seems like we’ve had somewhat of a philosophical shift in the last few years, in the sense that people are no longer thinking of information based on its origins, but really they’ve supplanted these data structures with how they can use it and how they can access it. That cuts across desktop, intranet, and Web; and it seems that access to information quickly and in the way that any individual wants it has really changed the notion of fleet and agile business.
In fact, business process is increasingly being driven by how people access content and then inject that into business goals. So, I want to first talk to you about business goals because many companies are very practical, of course. They’re looking at their top line and they’re looking at their bottom line. I want to throw this out perhaps to John Markus, how are companies using information discovery to further their business goals in a way now that’s perhaps different from ever before?
Lervik: Dana, I think that one way of looking at technology is always in the light of the business processes that they’re enabling. Many of our customers are thinking about ways in which they can achieve competitive advantage through productivity change, and they’re looking for something new. What we enable is for search to help facilitate communications between individuals.
We’ve all heard of Web 2.0 applied to the "Websphere," but in the enterprise, Enterprise 2.0 trends of social networking are also taking hold, allowing people to collaborate better and allowing individuals to streamline business processes. What's interesting for us in the future is search as an entry point into facilitating improved business processes, and business process improvement leads to improved business goals, as well as business-model innovations.
Gardner: Is there an impact here for applications themselves to start to be created based on what search can bring to the table rather than having search perhaps be an afterthought? What do you see as the future of search-driven applications?
Bjorn Olstad: This is Bjorn, and I'll have a go at that. The fundamental difference between search and such things as database, content management, and document management systems is that search starts with the user and then does reverse engineering -- what is necessary to realize this user's experience. It's starting with the content and then trying to deduce what should we do with this content. When you do that, you end up with a user-driven experience.
As you’re starting off, Dana, the user doesn’t care about the origin of the content. So, the ability to go into content sources and then chop that into information nuggets that can then be re-orchestrated and put into the context of what is useful for that specific individual user in what he’s trying to accomplish right now is a very powerful paradigm. It’s also a paradigm that can be used as the basis for building applications.
We see search more and more becoming the framework for the metaphor in how the user interface is being defined and also for how the application is actually defined by the source of the framework and then embedding functionalities in a source-stream framework.
Gardner: Is this really a shift of intent or are there some technologies that are newly arriving that empower this shift? Bjorn, can you tell us a little bit more about what’s happening technically that is allowing an expansion of the role of search?
Olstad: Let's go back a little bit. As you said, search used to be an afterthought. Typically, you had an application that was kind of hard-wired, done in HTML, or hard-coded down to the logic that usually resided on some type of database. So there was hard-wiring between the schemas and the data model and the database, and then the user interface.
Going forward, I think increasingly the user experience will be driven by algorithms and will be dynamic, so that you can actually optimize the user experience and the business efficiency. This is valid for both consumer-facing portals that you’ve seen on the Web but also on other applications internally. So then you get that framework and you say. "Let’s not hire system integrators to hard wire my content. Let the user decide, and when we deduce what the user likes, the algorithms can populate the user experience."
This will include both content and relevant services, and maybe a connection to people, so that you can find people to collaborate with. That becomes a framework, and search has matured from being merely access to documents to being this umbrella environment where you can provide access both to structured data and unstructured data and transform them into structure by putting additional metadata onto it. You can do the same with rich media, and search suddenly becomes that framework that can give you access to everything, as opposed to just the text that didn’t fit into the hard wire or the schema model.
Gardner: I suppose that with the increasing use of SOA, where applications are decomposed into services with the productivity coming from reuse as much as possible, and with governance providing an overlay for managing and organizing services, that search could have quite a powerful impact on how services are put together on the fly to create unique business processes. Do you see search moving in that direction as well?
Olstad: Absolutely. If you think about Web 1.0, Web 2.0 and Web 3.0, one way to interpret this is that Web 1.0 included these monolithic Web services. Then, in Web 2.0, it’s about the different services that make up atomic individual services and then having technologies to mash up and combine them to create new services on top of that, the same way as SOA. It seems nice at first because they are driven toward reuse, et cetera, but you move the complexity to the next level. How do you actually orchestrate? How do you put together these atomic content components and atomic service components? How do you actually do that and how to do that in a user-centric way, so you can gain the right user experience?
With search, it’s actually designed to do exactly that, to bring both components and services to atomic units and then bring the orchestration level that is able to recompile these content and service elements into what makes sense for the end user. Hence, it brings 3.0 integration levels on top of the atomic components that we have in Web 2.0.
Zia Zaman: This isn’t just our idea, if I can build upon that. I was talking to an information provider that provides information to financial services firms. What he asked for was the ability to atomize information, allowing the users the flexibility to consume it in their own way. So, it’s exactly what Bjorn was talking about. In Enterprise 2.0, we allow for the atomization. And then the re-organization around the business processes is what happens in the next phase.
Gardner: Zia, there is already quite a bit of difference of opinion on how to define Web 2.0, never mind Web 3.0, but let’s try to just get a handle on Web 3.0 for our audience. To me this is really getting towards the Semantic Web, where there is context on an automated basis, with much more rich metadata involved, and therefore the ability to apply algorithms more generally across the Internet. Is that your understanding at FAST about this Web 3.0 or how would you further that definition?
Zaman: Dana, I think you've got it right. Metadata is the key, but I'm going to pass this one over to Bjorn to talk a little bit more about Semantic Web.
Olstad: There are so many opinions about the Semantic Web. First of all, what I don’t like with the approach of Semantic Web is that you have to put in structure, and it’s not really useful until almost everybody has started to put structure into it.
So, how do you leap from that and actually make it useful before everybody has adopted this new scheme? Search can actually play a role, because search can auto-generate metadata and find ways to use that structure and to improve the discovery. Then, the allocation elements that the traditional Semantic Web talks about can be aimed at how to improve algorithms, as opposed to starting from scratch. In doing that, I think search has the opportunity to deliver on the premise of the Semantic Web, by applying algorithms as opposed to altering tools.
If you then move to Web 3.0, then using algorithms increasingly becomes a component of it. I think an immediate step, which is an opportunity that FAST is looking at right now, is how can you use search at this level to go from atomic services to the rich environment, where you effectively can develop and deploy services. That’s kind of the orchestration: how you put services and content components into meaningful context for the end users. That is what brings business value. It brings time to market value for companies that are trying either to develop Web 2.0 and Web 3.0 applications or move their current models to a more collaborative approach.
Gardner: I suppose one of the frustrations we’ve heard quite often from users is that they can find things easier on the World Wide Web, than on their own hard drive or their own enterprise network. We’ve come a long way to assuage those concerns, but it seems to me like perhaps we’re at a point where we can go and leapfrog the Web.
If we’re thinking about a Semantic Web, that’s something that requires a lot of standardization. It could take a number of years and a high level of complexity, but moving to the "Semantic Enterprise" seems something that’s quite a bit more attainable in the short term. Do you think, and I’ll throw this out to anyone, are we going to move to a point where we can apply these benefits to the Enterprise and then the Web will catch up to them or is the Web always going to be a bit easier to manage?
Olstad: Maybe I could start, and, Zia, you could give some business examples. This is an excellent question, Dana. First of all, just so that we’re on the same page about what the Web experience is all about, there are two types of queries. There is the type of query where you know what you’re looking for. So you search for some properties and what you’re actually looking for.
The other type of query is when you don’t know what's there, but you throw out a wide, open-ended query, and you want the data to talk to you. In Web search, there is no completeness requirement. Web search is defined when you type in one or two query terms, which is a nice starting point for your browsing. So you get a link to a Website where you can start. There is no completeness. There is no overview of the other elements that didn’t end on the first result page.
Web search has moved outside Web search and can be used inside the Enterprise for this semantic analysis. It can also identify facts inside documents, combine it with the information structure databases, and provide discovery. You can start to do analytics, so that you can relate concepts and enable people to make decisions and get completeness and overview. That is why you see a merger between business intelligence and search starting to appear.
Gardner: So, perhaps moving toward a business intelligence (BI) benefit vis-Ã -vis search, and of course the other tools that are available for BI, is it first a necessary step to the Semantic Enterprise?
Olstad: Yes, I think so. To give just one example of a company that is trying to use search in an innovative way, what about all communication internal in the company? What about instant messaging, email, Voice over IP (VOIP), Skype, et cetera, and how to improve the quality of that communication and also derive analytics from it.
For example, if I call you, Dana, I should call you as a conceptual entity and you should actually be the one who decides if that ends up as a Skype call or cell phone call. This company is doing that and using search to orchestrate all the communication, allowing you, as the recipient of the call, to configure how you want to be accessed. By having that system, you can also create analytics that understand what is really happening in the company: who are the experts, what’s happening, what are the trends, etc.
Zaman: I actually think that it could be a lot simpler than that. The expert location is something that Bjorn just touched upon, which may be one of the next killer apps in the enterprise. What we’re finding more and more when we talk to our customers is that information discovery isn’t just about connecting users to results. It’s about connecting users to answers, to analytics, and to services -- but most importantly, to people. Connecting people to other people, so that they can do their job better. To improve decision-making throughout the enterprise is a big part of what information discovery tools are about to enable.
One way of looking at it is, if I can find the expert in the organization or outside the organization who can help me best in the decision process that I'm currently in -- and I can find that expert in the shortest period of time, using insights about who this person might be by using metadata about this individual, by actually discovering this path that I didn’t know existed upfront -- then I have actually added value to the business. So, we believe that search as a matching service is really one of the killer applications.
Gardner: So, what we’re talking about here is bringing the tacit knowledge that’s available to any organization into play with the semantic knowledge that’s gathered across all modalities, all types of media and information, but allowing those to come together in real-time based on someone’s specific needs through a very simple query.
Zaman: That’s right. I often like to talk about Greek philosophers, and Socrates is one of my favorites. The reason is because of what he did with a seeker. He didn’t answer a question with an answer. Rather he asked another question, and that allowed the seeker to refine his or her question, until finally they got to what they were looking for. In many ways, technology -- whether it’s information discovery, search, and business intelligence, whatever it might be -- shouldn’t insult the intelligence of the seeker. Rather it should allow individuals to find the answer that they’re looking for, either through tacit information that’s stored in the enterprise, or semantic information, or structured information, whatever it might be. What we’re trying to do is mimic the type of dialogue that Socrates had.
Gardner: Sure, because people are quite good at peeling away the layers of an onion, as a metaphor, to find what it is they’re looking for. There is no beginning or end to this peeling, but we certainly want to get him or her close to where they can extract real value.
Zaman: You got it.
Gardner: All right. Now people have been talking about this for years. I deal with developers a lot. I think they understand a lot of these theories. They understand the value and the goals, but putting it into practice, putting it into execution seems to be where there is still a need for some work. When we go to developers, we try to encourage them to start exploiting these capabilities of changing the mindset of how an application, specifically the UI, is actually constructed to make the entry point into an application more amenable for search and for gathering of these layers within an organization’s assets and resources.
Are there tools? Are there APIs? Are there SDKs? Is there a mindset? How do we bring the developer into play here? I’ll throw this out to anyone.
Olstad: I think Gartner predicted that in 2012, 75 percent of the world's applications will use search as the dominant metaphor for how an application is being operated by the end-user. That is a very powerful statement. And it means that all those developers that you’re asking about, Dana, they have to adapt to that.
Simplicity in an application [means] that you have a powerful metaphor that you can use to deduce what you should expect the application to be doing. Search is exactly that. So, obviously, search is evolving to not only provide access to text, but to provide value to all types of elements.
Another change in search technology is that it handles content natively, and previously it did not. In Web search you type some queries and you get a URL link to the content. Now you can take XML or SQL databases, and you can get native content. You can search in native content with queries that emulate what you can do in the database or in an XML database. This means that you can build the same type of application with the freedom to access data in an ordinalistic way, and have a mechanism to prioritize and do exactly what Zia talked about with Socrates.
Technically, if you use an open-ended query like, "List the innovative people in my company," or something, you could get back kind of a menu of people that have been referred to in the documents or discussions where there is talk of innovation, and then get the facts related to these people. So it’s not coming back with an answer, but it is coming back with an analysis -- and giving you the opportunity to refine the query.
Gardner: Another thing with developers is that they’re interested in seeing examples. Are there folks out there doing this now who are really on the cutting-edge, who have been able to instantiate rather than theorize? Maybe, Zia, you could address this. Are there folks that are out there using FAST Search & Transfer technologies or other technologies who are actually realizing some of these benefits? And, who are they, and how are they doing it?
Zaman: All over the landscape of the economy, people are using search in interesting ways. I think that the message to the developer is: "Think about the app that you’re working on right now, and think about how the users are accessing either the application or the information."
Take a consumer perspective and you’ll definitely think that search plays an important part in whatever it is that you’re doing. So, get involved and talk to people about search. I think you will find that, even in your own sphere, there are people who are already working on search technologies. In one particular investment bank, 15 or 16 different applications of FAST Search have been put into place across the business to help with everything from getting access to research reports to improving customer service.
There are search policies at law firms or at pharmaceutical companies or retail or e-commerce searches for Christmas shopping. Or just B2B searches, so that one individual can find another professional in the shortest amount of time in the right region. All these applications are about the user consuming information in a way that is most effective for them -- and search technologies are enabling each of these examples. Maybe John or Bjorn, you want to talk about a European example and I can talk about an American example?
Olstad: Sure. Let me start off with our broad business goals, and there are some that have nice benefits for the businesses. We have many customers that don’t have a choice because if you look at media, entertainment, and telcos, there are so many disruptive changes going on. You have print transforming to electronic. You have advertising on the Internet, where search-based advertising is now bypassing traditional banner advertising. You have telcos, where telephone calls may soon be free because there are alternative ways to monetize it. And you already have Skype, Voice over IP, etc.
So, if what a telco provider, for example, is providing today is going to be free, then the name of the game is going to be who is able to create applications and to have service delivery platforms that use that customer base to derive new services. They’re not very well equipped for that. The key enabling-technology that actually helps with this has got to be the user-driven applications. And they see search from a monetization perspective and from the perspective of building the end-user experience. So for lot of customers that we work with from media, entrainment, and telcos -- it’s not that it's "nice to have," but it's "How do we survive?"
Zaman: An example of that in the back office is AutoTrader.com, which is a phenomenal example of how search was used to fundamentally rethink the way in which records were served up. The database administrator at AutoTrader says that they tripled their performance at half the cost, using a highly scalable search platform, which was very easy to deploy and a lot more dynamic than their traditional database architecture. So, even in the back office for applications as diverse as AutoTrader, we are seeing search being used to really disrupt the way in which you think about information delivery.
Gardner: It also seems that the younger generation, as they come into play here, are probably already automatically thinking "search." The people who have grown up with the Web around them start almost any activity with search. So, this is becoming imbued into the society and into the behavior and culture?
At the same time, these folks are also quite used to mobile devices and small handheld devices. Can you explore for me a little bit of how search plays into young people in mobile devices? Perhaps we’re seeing some activities in early-adopter environments, South Korea for example, where search has to be the only way in a small-form factor that vast amounts of information could be managed. Help me understand the impact of use and mobile search?
Olstad: That’s a very interesting question, Dana. As you said, most exploration starts with a search on the Web. You start with a search on a Web search engine, and then you come to the site of this company. Then that company has some options. Should we let the user continue with the search experience or should we force him to go back and understand the text and how we have orchestrated all our content? Then, he is not allowed to do anything meaningful before he has completed that education. That’s slowing down the progress of the users.
Moving to mobile, if we go back to 2001-2002, we were trying to educate mobile operators that they needed to use search. That was a difficult discussion, a difficult sell. In Europe and Asia, at least, it was very popular to download ringtones and wallpapers and things like that. The operators felt that it was sufficient to have a chart with the most popular items and have them available on the static page, allowing people to choose from that.
When they moved to search, what they experienced was that a good way to measure the UI or the usability for the user is how many operations it took to actually perform the action. One of our customers is Vodafone. They measured how many operations young people had to do, from coming into the portal until they could complete a transaction, which in this case was to download a ringtone or music, etc. That used to be 3.5-operations, or something like that, when it was based on the charts and navigation mechanisms. When they substituted search, it went down to 1.5, which means that almost always you go directly to something that you can download. And, once in a while, you have to do one iteration before you get it. That drove revenue and traffic tremendously.
Zaman: You’re right. Generation Y and the younger generation are very used to unfettered access to their data, their content. What does that mean in the Enterprise? This is something that I hear increasingly on Wall Street and in the City of London. I know how to deal with the managing director, who is in his 40s. It’s very easy. For their relationship with their data, they ask someone junior.
But for that younger investment banker who’s just grown up with unfettered access to their personal data, they keep asking why they can’t get access to enterprise data with the same facility. The information technology people who are serving this younger generation have to change. They don’t quite understand that search and other technologies, including Web 2.0 technologies, are forcing them to rethink the way in which they have access to their information, because that’s the way they can be creative and allow for them to get the data they need in order to make the right decisions. I think it’s having a cultural effect, even in more staid industries.
Olstad: Then, if you go to Japan or to Asia, you have also this competition between the telco providers, where the name of the game now is how do you add video services, and with extremely high bandwidth on the phones, how to do that? You see multimedia screening, where you are serving video quality down to the cell phones. So there is a tremendous interest in how to capitalize on new user experience to combine rich media assets, and then use this to build the community stuff you have seen in YouTube in the U.S. This is happening on the mobile side at a tremendous speed in Asia.
Gardner: Very interesting. So we’re really still in the early stages of search. It sounds like there is going to be a lot to discuss at your forthcoming FASTforward ’07 event. Perhaps as a little tease for our audience, is there anything you can tell us about this event in terms of some additional technologies, products or approaches? I will throw this out to anyone.
Olstad: If I could start from the technology side. There will be a lot of cool introductions and some new technology. We will look at how to interface search with repositories, from structured data to rich media and to traditional text. We think there is a game-changing opportunity in doing that and also to enable the Semantic Web in a more powerful way, as we talked about.
We also really believe that, as we have touched on here, that search is not just a feature, but it will become the framework for building search-driven, user-driven applications, to actually make that a reality and make it possible to build those applications. We are introducing new components to do that.
We see that search is driving the connective role in Web 2.0 in the evolution towards Web 3.0 by being a service-delivery platform. So we are introducing a Web 2.0 platform that uses search as the main orchestration -- both for monetization purposes and for understanding both content and the user. We'll be tracking the user and understanding his preferences, building personalized experiences, and creating this platform where you can take the atomic components of content and services and effectively roll that out as user-centric services. We’re building a service delivery platform around search for doing that and you will see applications that already have been built on top of that platform.
Zaman: I think it’s going to be a very hands-on conference in San Diego. There will be customers. There will be developers. There will be case studies, demos, and a lot of learning going on, interacting and sharing of ideas. Sure, there will be the predictions and the analysts and the advice that goes along with it. But really I think that the value of this search conference will be in the exchange of innovative ideas that takes place, not just from FAST to the rest of the community -- but also from the community to other members within the community. I think it's going to be great.
Gardner: Thanks. John Markus, just to return back to the business value, we’ve been discussing this at a fairly technical level -- developers and business intelligence folks -- but for CEOs, for the leadership at many enterprises, what are some of the salient messages that you think they need to understand in order to better appreciate search and why it makes a good deal of sense to invest in it?
Lervik: In general for most information-intensive companies -- media companies, financial services companies, telecom companies and pharmaceutical companies -- search is becoming both a key part of the infrastructure to enable these companies to make decisions, but also -- more importantly -- to drive and increase business online.
Because we see more and more of these companies relying on effectively connecting customers or end-users to their products, to their contents, and to their services. That’s where search comes in as a mission-critical application, or at least to enable these mission critical applications. At FASTforward, there will be a great number of speakers who will talk about how these have effectively implemented a solution that drive and increase business.
Gardner: I’m afraid we’re about out of time. We’ve been discussing search, the role of search, Enterprise 2.0, Web 3.0. Clearly the role and impact of search is increasing rapidly and will increase further over the coming years.
I’d like to thank our guests and sponsor, FAST Search & Transfer. We’ve been talking with John Markus Lervik, the CEO. Also, Bjorn Olstad, the CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for joining us for this BriefingsDirect podcast.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Transcript of Dana Gardner’s BriefingsDirect podcast on the future of enterprise search. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Subscribe to:
Posts (Atom)
