Integration Infrastructure Approaches Must Adjust to New World of SaaS and Shared Services

Edited transcript of BriefingsDirect[TM] podcast with Cape Clear's Annrai O'Toole and analyst Phil Wainewright, recorded Sept. 13, 2007.

Listen to the podcast here. Sponsor: Cape Clear Software.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today, a sponsored podcast discussion on integration infrastructure for modern software-as-a-service (SaaS), and on-demand applications providers.

The support and requirements for integration among these organizations, these hosters of many different types of services and applications, require a different integration approach. They're dealing with complexity, massive scale, and a cost structure that’s very lean. They have a business model that’s often based on ongoing subscriptions, and, therefore, they have a margin issue to maintain. They also have a wide variability of users and service-level agreements (SLA) to meet.

Reuse is becoming an important issue, as are patterns of automation. We're going to take a look at the market for integration in these modern service provider organizations. We’re going to look at the characteristics of the IT infrastructure support that meets these new requirements. And, we’re going to offer some solutions and approaches that satisfy these needs provided by Cape Clear Software.

Here to help us weed through this issue, we have Phil Wainewright. He's an independent consultant, director of Procullux Ventures, and also a fellow ZDNet blogger, primarily on this SaaS environment. We're also joined by Annrai O’Toole, CEO of Cape Clear Software. Welcome to you both.

Phil Wainewright: Thanks.

Annrai O'Toole: Great to be here.

Gardner: Phil, let’s start with you. It seems that SaaS and the new ecology of providers that are welling up around it require a different approach to integration, and this amounts to supporting the shared-services capability. Not only are these service providers going to require this capability, but also enterprises, as they become hosts themselves. Can you describe the different approaches to integration that we're now going to need in order for this business model to prosper?

Wainewright: Part of the issue that people face is that integration has crept up on them. In the early days of SaaS, it was very much about adopting point solutions. One of the reasons Salesforce.com has been so successful in the CRM space is that what Salesforce.com did with automation software didn’t have to affect the rest of the business, because they could just do it in isolation. IT wasn't really worried about it. So, they said, "Okay, we’ll just go to do that and it won’t affect the rest of the IT infrastructure" Now, we're getting more sophisticated about SaaS, because it's being taken on board in a whole range of areas within the enterprise, and people want to do integration.

There are two forms of integration coming to the fore. The first is where data needs to be exchanged with legacy applications within the enterprise. The second form of integration that we see -- not at the moment, but it’s increasingly going to be an issue -- is where people want to integrate between different services coming in from the cloud. It’s a topic that’s familiar when we talk about mashups, fairly simple integrations of services that are done at the browser level. In the enterprise space, people tend to talk about composite applications, and it seems to be more difficult when you are dealing with a range of data sources that have to be combined.

Gardner: It seems to me that now we are elevating to an integration of integration types. Does that make sense?

Wainewright: People have realized that if you're doing integration to each separate service that's out there, then you're creating the same point-to-point spaghetti that people were trying to get away from by moving to this new IT paradigm. People are starting to think that there's a better way of doing this. If there's a better way of delivering the software, then there ought to be a better way of integrating it together as well.

Therefore, they realize that if we share the integration, rather than building it from scratch each time, we can bring into the integration field some of the benefits that we see with the shared-services architecture or SaaS.

Gardner: How is this different from the way that application service providers (ASPs) of an earlier era went? It seems that they had a stack underneath each and every application. Now, we're hearing more about virtualization and the efficient use of infrastructure and platform elements. Help us compare and contrast the ASP model to where SaaS providers are headed now.

Wainewright: In the ASP model, you took existing conventional applications -- I like to call them unreconstructed applications -- and just hosted them in a datacenter. I've always said that it’s more than just a relocation exercise. You need to change the way the software is architected to take advantage of the fact that it's now in the Web and is in a shared-services environment.

In 1999 or 2000, one company came out with hosted webMethods integration as an offering. The problem was that it wasn't scalable. This is the problem with the ASP model. You were hosting a customized infrastructure for each individual customer.

Although there was some sharing of the infrastructure that you could do with virtualization -- and Oracle’s On-Demand is an example that’s done a lot of work with virtualization -- you still end up with every individual customer having an individual application tailored to his or her specific requirements. You get no economies of scale from doing that.

So, the new generation of SaaS providers, are really talking about a shared infrastructure, where the application is configured and tailored to the needs of individual customers. In a way, they’re segmented off from the way the infrastructure works underneath.

Gardner: Annrai, at Cape Clear Software, you’re targeting these providers and recognizing that they’re going to be moving to service-oriented architecture (SOA) relatively quickly, based on their requirements of their business and demands of complexity. How are you seeing these providers, as you are encountering them in the field, managing this discrepancy between commonality on the infrastructure, but high variability on the application and delivery to the end-users?

O'Toole: As Phil just said, taking unique integrations and hosting them isn’t very useful, because the person who is doing the work just ends up doing tons and tons of different customizations for every single customer. In the integration world, the solutions to these problems are reasonably well understood. The difficulty has always been about delivering them.

So, we’ve been working pretty closely with a number of prominent SaaS companies, most publicly Workday and some of our large enterprise customers, in figuring out how to solve integration problems for these on-demand and hosted providers.

What come out of it are two issues. First, Salesforce.com, really pioneered multi-tenanting as a concept to reduce the cost of hosting an application for lots of different users. We've got to do the same thing with integration, and we’ve been working with our customers and updating our enterprise service bus (ESB) to support multi-tenanting at the ESB level. You can have integration and have it segmented, so that it can be utilized by different customers at the same time.

In a large enterprise, this allows you have a shared-service model as well. You can build integrations in a datacenter and then segment them, so that different people can use them simultaneously.

There is also another aspect to the problem that needs to be solved. When you build an integration, you always end up having to customize it in some way for different customers. Customers will have different data formats. They’ll want to access it slightly differently. Some people will want to talk to it over SOAP. Some won't, and they’ll want to use something like REST. Or they might be going backwards and are only able to send it FTP drops, or something like that.

Multi-tenanting is one solution to the problem. The other is what we call multi-channel, which is the ability to have an integration, and make it available with different security policies, different transports, and different transformations going in and out.

A combination of multi-tenanting and multi-channeling allows you to build integrations once, make them accessible to different users, and make them accessible in different ways for each of those different customers. It gives you the scalability and reuse you need to make this model viable.

Gardner: Phil, it sounds as if we're really redefining integration and that we’re putting a higher abstraction, infrastructure-as-a-service, integration-as-a-service approach and model around it. Is this unprecedented, or is there anything in the past that we can look to, within even a closed environment, or a siloed environment, that relates to this. Or, are we into something entirely new when it comes to integration?

Wainewright: We're trying something that hasn’t really been tried with conviction in the past. In fairness, there’s still a lot of skepticism out there about whether we can really share integrations in the way that Annrai has just described. There’s a certain amount of agreement that we need to reuse integrations, but people haven’t necessarily brought into the whole shared services model.

People are particularly doubtful whether you can do all of the nuances of integration that you need to do in a purely declarative model, whereby you can simply have a description of the integrations that are required and have a need to work, and then that gets abstracted by the architecture into operation.

Annrai could probably elaborate on how it’s become possible to get the level of detail into this more loosely coupled architecture to enable people to share integrations, without giving up the individual requirements that they each have.

O'Toole: To pick up on that, I would be the first to share Phil’s skepticism and to offer that we may not have all the answers. However, one point worth bearing in mind here is that this problem is going to get solved, because the economic reality of it suggests that we must solve this. One, the payoff for getting it right is huge. Second, the whole model of SaaS won’t be successful, unless we skin the integration problem. We don’t want the world to be limited to just having Salesforce.com with its siloed application.

We want SaaS to be the generic solution for everybody. That’s the way the industry is going, and that can only happen by solving this problem. So, we’re having a good stab at it, and I'll just briefly address some of the things that I think enable us to do it now, as opposed to in the past.

First, there is a standardization that’s taken place. A set of standards has been created around SOA, giving us the interoperability platform that makes it possible in a way that was never possible before. Second is an acceptance of this shared-services, hosted model.

Years ago, people would have laughed at you and said, "I’m going to trust all my customer data to a provider in the cloud?" But, they’re doing it happily because of the economics of it. The whole trend towards trusting people with outsourced offerings means that the people will be more likely to trust integrations out there, because a lot of the technology to do this has been around for quite sometime.

Gardner: To Phil’s point, Annrai, how do you begin to accomplish this? Is this a matter of creating forms, templates, and models and then applying that to the ESB, where the ESB executes on that model and then they can be reused? If not, please explain? Then second, how much automation are we talking about? Are we going from 5 percent or 10 percent reuse, and then working up more -- or can we actually bite off more of a total integration as a pattern?

O'Toole: That’s a very good question. It's a movement to a more declarative style of integration. Certainly, what we’re doing is saying that a lot of the issues in integration are very common across different types of integration. For example, a lot of the issues currently in integration revolve around data transformation. Other issues revolve around the complexities of dealing with transports. For example, information can come in on email, but I need to be able to send it out to someone else on FTP.

The third set of issues is around handling errors in a meaningful way, so that when things go wrong, you’ve got a nice model of propagating errors back to people. The next set of things is security policies. Again, in integration you’re touching different things with different security models. With what we’ve done, you can visualize a whole bunch of pre-build security policies, transports, and transformations, and you’ve got this palette of things that you can then assemble together.

Someone still has to write the core business logic of the integration. That fact hasn’t changed. You still need that. Once I have the core business logic of the integration built, then all of the things around it I can put on in a visual and declarative manner around that integration, such as, "I want that integration to be accessed with this security policy, over this transport and with this transformation to get the data in and out of it."

I can take a single integration and create hundreds of different access routes into it, all with different security policies, different transformations, etc., and can segment them so that none of those things interferes with another during that multi-tenanted mode.

That technique allows people to reuse that integration for hundreds, if not thousands, of different customers and users in a totally segmented way. So, that’s at the core of what we are doing.

Gardner: What’s a ballpark figure for percentage of reuse, versus project-by-project or instance-by-instance customization?

O'Toole: There is no hard and fast data, so we rely on heuristics and rule of thumb. If we look at what our own consultants and systems engineers would do for customers, 70 or 80 percent of our time is spent around configuring different transports, and working at security. These are the things that are totally unglamorous, but really nasty gotchas in integration. For a long time, as people focused on making the core creation of the integration easier, they missed those other issues around this, and they’ve always been left to one side.

You might say, "It’s not hard to write a few lines of Java code to handle a different security policy." Well, it isn’t, but it is, if you've got to do a thousand different times for a thousand different people. It becomes tedious, and then you've got to manage all those things. So, it really is attacking some of the kind of nasty little gotchas on which people spend the vast majority of their time.

Gardner: Phil, you mentioned declarative, and Annrai mentioned graphical and visual approaches to this. Is it too farfetched to consider that non-programmers and non-technicians will be getting involved with this? Are we elevating an integration function to folks that are architects and/or business process analyst types?

Wainewright: A lot of the integration that people need to do is for business reason. Therefore, it should be in the hands of business people. For a long time, the answer back from the technologists has been, "Well, sorry guys, we can’t let you go there, because it’s too complicated." If a lot of the infrastructure can be automated and done in a way that, as Annrai says, takes care of the complexity, then it becomes easier to offer out certain aspects of the integration to business people, but I think the technology guys are still going to have to be there.

It’s a good idea for business people to be able to link up different business processes, or to say, "I want to aggregate this data with that data, so I can analyze it, as well as have a dashboard that tells me what’s actually going on in my business," but that has to operate within the constraints provided by the technology guys who actually make it all happen reliably.

I was listening to what Annrai was saying. He made a couple of very interesting statements, which are worth going back to, because a lot of people who approach integrations in terms of SaaS, are really talking about on-premises integration that links into the SaaS stuff in the cloud.

Annrai is talking about taking the integration off premises, putting it in the cloud as well, and then sharing a lot of the capability. In the SaaS market space, large amounts of integration reuse is happening at the moment. It's simply a matter of installing the same data connector in different customers, and perhaps reusing the expertise, but it’s not in any sense a shared service.

You're basically duplicating all of that infrastructure. If you can put the integration infrastructure in the cloud, then you start to get the benefits of shared services and you can get above the 50 percent level of sharing. But, if you’re going to do that, you’ve got to have the confidence that the architecture, can actually support all of that and isn’t going to fall over with certainly a thousand integrations happening all at once, and mustn’t conflict with each other. Cape Clear has actually managed to package that into a product that people can buy and install. I think that’s pretty impressive.

Gardner: Annrai, it still brings up these issues about integration on the hosting organization side, where they’re going to be integrating across resources, infrastructure components, application types, and data streams. They’re going to need to do that to manage the complexity and scale of their endeavor and their business, but is there a certain level of integration where there is more permeability between the enterprise, the on-premises, and what’s available through the cloud or the provider?

Are you talking about both, or either/or? Does one to have to come first, and how do you see this shaking out in terms of integration and these different directions?

O'Toole: The way I’d phrase this is that we’re seeing a little bit of both. We’re not wildly being drawn into one end of the spectrum or the other at this time. What we're seeing from companies like Workday is a requirement for them to more or less exclusively host integrations for all their customers, and all the various back ends that they talk to.

For example, they talk to things like ADP and benefits providers and so on, and they are hosting those integrations on their sites, because they don’t want to go to ADP -- well, they can’t go to ADP -- and make ADP to change and so on.

Gardner: And, ADP is a payroll service provider.

O'Toole: It's the largest payroll service provider in the world, I believe.

We’re seeing things like that, but in enterprises you’re seeing this big move to virtualization and shared services. They’re saying, "Why are we having development teams build integration in all these branch offices at all these locations around the world? It’s extremely wasteful. It's a lot of skill that we've got to push out, and there are a lot of things that go wrong with these. Can't we consolidate all of those into a centralized data center? We’ll host those integrations for those individual business units or those at departments, but we'll do it here. We’ve got all the expertise in one place."

Those guys are delighted, because at the individual local level they don’t maintain all the costs and all the complexity of dealing with all the issues. It’s hosted out in their internal cloud. We haven't seen enough data points on that, but this hosted integration model can work. We’ve got it working for pure entities in SaaS companies like Workday, and we’ve got it working for a number of large enterprises. There is enough evidence for us to believe that this is really going to be the way forward for everybody in the industry.

Gardner: At Cape Clear Software you're not just dealing with this as an abstraction. You have product ties. I understand that you’re going to be introducing a new iteration to the Cape Clear platform, Version 7.5, in late September. Can you tell us a little bit about how you’ve taken on the need in the market, what you foresee as possible and doable, and what you’re actually delivering?

O'Toole: The generic need we've seen in the market is what we like to call on-demand integration. Before the advent of all the things we’ve just talked about -- shared services and SaaS -- the only way people did integration was with big patches, what became known as enterprise application integration (EAI), and they were essentially hand customized, on-premises integrations that were different for every customer.

We think that the demand we’ve seen in the market is a move away from on-premises integration into on-demand integration, because, for the reasons we spoke of, people want to be able to integrate with SaaS. They want to run shared-services models in their own organizations.

We think there is a new movement from EAI into on-demand integration and we have been targeting several features in our ESB, specifically around this multi-tenanting and multi-channel stuff, and that’s all getting rolled up into a release called Cape Clear 7.5 that goes out on September 25. We're pretty pleased with this. All our guys worked hard, and I think it’s the best product we’ve ever done.

Gardner: Give us the short list of the new functionality?

O'Toole: Essentially I think it boils down into three major buckets. There’s a new graphical editor that we're calling the SOA Assembly Editor, and this is an Eclipse-based editor that allows you to graphically clip together the type of things I was describing: transports, security and so on. That’s a whole new editor that no one has ever seen in our product before, except the people who have been involved in our tech preview.

Gardner: Let me pause you there. Is this something that you foresee being useful and applicable to non-techies or at least folks that can cross the chasm between the business issues and the technologies? Who’s going to typical author with this tool?

O'Toole: It’s not an either/or. It’s certainly not something that the business users can use exclusively by themselves, but it's not a tool exclusively for developers. Our grand vision around this business/technical user thing is that you start to create sets of tools that can be shared across people. You can have high-level business folks describing business process and BPMN, and they can be imported into other sets of tools that the technologists use.

Then, there are some issues around how this thing is going to be accessed? Our business analysts do care about that, because they say, "These sets of customers must be able to access it this way." They might not know the details of the technology involved, but they do know how those people want to be able to use the service. So, as I say, it’s a little bit of both. Ultimately, people are really going to get value out of it, but, at the end of day, they’re going to be more on the technology side, because it’s really going to smooth up their job of automating what was an incredibly laborious process.

Gardner: Okay. That was bucket one.

O'Toole: The second bucket of the features is all around multi-tenanting. These are core additions into the ESB to allow segmentation of integrations, data, and reporting. You can set how you want to identify inbound customers, clients, or businesses and then segment use of those integrations on the reporting and management of those integrations, based on those identities.

The third bucket is all of the stuff we’ve seen customers need in terms of running and maintaining large enterprise class Business Process Execution Language (BPEL) deployments. Obviously a lot of our product has been built around the notion of BPEL, as a kind of core metaphor for how people build integrations and manage business possesses. So, we are totally committed to BPEL.

In working with our customers on that, in terms of very large-scale deployments, there's a whole set of issues around how people maintain and manage. So, as our products are evolving, they're evolving away from just being a kind of BPEL engine into a full BPEL management system, where we are giving people all the tools to monitor transactions, and repair transactions when they fail. This is largely for business reasons, because if something goes wrong in the business information, they've got to be able to rebuild that last business information and get that business transaction back on track.

Gardner: It occurred to me earlier, when you were talking, that we are really not only recasting integration, but we are moving beyond implementation of integration into management of integration, and then a step further beyond that into the management of change and integration. Does that sound fair?

O'Toole: One of the nice thing about BPEL is they are long running processes, but the challenge that presents is how do I manage the change in long running processes. I can’t go in and wipe the database and say, "I didn’t like that schema that I had underlying that business process." You need to be able to evolve that. So there's initial support for some of those concepts in this version as well.

Gardner: I like to bounce it off of Phil. Do you think that we're well into integration management and that the execution is important, but it’s the higher value to a shared services environment that's in this change-management capability?

Wainewright: Well, yes. This is part of a big movement that we are seeing in the way that we approach IT from a batch process of having a requirement, building to it, delivering it, and then bringing it into operation, to a much more iterative, ongoing, continuous process of delivering something that is constantly adjusting to the business needs of the organization.

So yes, IT is really catching up with reality. We’re now reaching the level of sophistication in IT infrastructure that actually allows us to manage change as a continuous succession of events, rather than having to wait for an upgrade to Windows or new version rolled out, or whatever it is, to actually move to the next stage in our business automation.

I think it should be welcomed and it’s inevitable. Perhaps it’s frustrating that it always seems to take so long to make it happen. The vision always seems to be several years ahead of the reality, but it's definitely the transformation that we are seeing.

Gardner: Great! I think we’ll leave it there. We’ve been discussing redefining integration as a reusable function for providers of services and hosted applications, and the notion of a shared services environment, which applies to both large enterprises as well as hosted.

We’ve also been discussing the marketplace and this movement towards management and agility in a SOA environment. We’ve talked about how Cape Clear Software is delivering Cape Clear 7.5 to help grease the skids towards this integration functionality. Phil Wainewright, an independent consultant, director of Procullux Ventures and ZDNet SaaS blogger, has joined us in this discussion. Thank you, Phil.

Wainewright: It’s been a pleasure.

Gardner: Also we have been joined by Annrai O'Toole, the CEO of Cape Clear Software. Thank you Annrai.

O'Toole: Thank you guys, it’s been very interesting.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You’ve been listening to a sponsored BriefingsDirect Podcast. Thanks for listening.

Listen to the podcast here. Sponsor: Cape Clear Software.

Transcript of BriefingsDirect Podcast on SaaS with Cape Clear's Annrai O'Toole and analyst Phil Wainewright. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

Probabilistic Analysis Predicts IT Systems Problems Before Costly Applications Outages

Edited transcript of BriefingsDirect[TM] podcast on probabilistic IT systems analysis and management, recorded Aug. 16, 2007.

Listen to the podcast here. Sponsor: Integrien Corp.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today our sponsored podcast focuses on the operational integrity of data centers, the high cost of IT operations, and the extremely high cost of application downtime and performance degradation.

Rather than losing control to ever-increasing complexity, and gaining less and less insight into the root causes of problematic applications and services, enterprises and on-demand application providers alike need to predict how systems will behave under a variety of conditions.

By adding real-time analytics to their systems management practices, operators can fully determine the normal state of how systems should be performing. Then, by measuring the characteristics of systems under many conditions over time, datacenter administrators and systems operators gain the ability to predict and prevent threats to the performance of their applications and services.

As a result they can stay ahead of complexity, and contain the costs of ongoing high-performance applications delivery.

This ability to maintain operational integrity through predictive analytics means IT operators can significantly reduce costs while delivering high levels of service.

Here to help us understand how to manage complexity by leveraging probabilistic systems management and remediation, we are joined by Mazda Marvasti, the CTO of Integrien Corp. Welcome to the show, Mazda.

Mazda Marvasti: Thank you, Dana.

Gardner: Why don’t we take a look at the problem set? Most people are now aware that their IT budgets are strained just by ongoing costs. Whether they are in a medium-sized company, large enterprise, or service-hosting environment, some 70 percent to 80 percent of budgets are going to ongoing operations.

That leaves very little left over for discretionary spending. If you have constant change or a dynamic environment, you're left without much resources to tap in order to meet a dynamic market shift. Can you explain how we got to this position? Why are we spending so much just to keep our heads above water in IT?

Marvasti: When we started in the IT environment, if you remember the mainframe days, it was pretty well defined. You had couple of big boxes. They ran a couple of large applications. It was well understood. You could collect some data from it, so you knew what was going on within it.

We graduated to the client-server era, where we had more flexibility in terms of deployment -- but with that came increasing complexity. Then we moved ahead to n-tier Web applications, and we had yet another increase in complexity. A lot of products came in to try to alleviate that complexity for deep-data collection. And management systems grew out, covering an entire enterprise for data collection, but the complexity was still there.

Now, with service-oriented architecture (SOA) and virtualization moving into application-development and data-center automation, there is a tremendous amount of complexity in the operations arena. You can’t have the people who used to have the "tribal knowledge" in their head determining where the problems are coming from or what the issues are.

The problems and the complexity have gone beyond the capability of people just sitting there in front of screens of data, trying to make sense out of it. So, as we gained efficiency from application development, we need consistency of performance and availability, but all of this added to the complexity of managing the data center.

That’s how the evolution of the data center went from being totally deterministic, meaning that you knew every variable, could measure it, and had very specific rules telling you if certain things happened, and what they were and what they meant -- all the way to a non-deterministic era, which we are in right now.

Now, you can't possibly know all the variables, and the rules that you come up with today may be invalid tomorrow, all just because of change that has gone on in your environment. So, you cannot use the same techniques that you used 10 or 15 years ago to manage your operations today. Yet that’s what the current tools are doing. They are just more of the same, and that’s not meeting the requirements of the operations center anymore.

Gardner: At the same time, we are seeing that a company’s applications are increasingly the primary way that it reaches out to its sell side, to customers -- as well as its buy side, to its supply chain, its partners, and ecology. So applications are growing more important. The environment is growing more complex, and the ability to know what’s going on is completely out of hand.

Marvasti: That’s right. You used to know exactly where your application was, what systems it touched, and what it was doing. Now, because of the demand of the customers and the demands of the business to develop applications more rapidly, you’ve gone into an SOA era or an n-tier application era, where you have a lot of reusability of components for faster development and better quality of applications -- but also a lot more complexity in the operations arena.

What that has led to is that you no longer even know in a deterministic fashion where your applications might be touching or into what arenas they might be going. There's no sense of, "This is it. These are the bounds of my application." Now it’s getting cloudier, especially with SOA coming in.

Gardner: We’ve seen some attempts in the conventional management space to keep up with this. We’ve been generating more agents, putting in more sniffers, applying different kinds of management. And yet we still seem to be suffering the problems. What do you think is a next step in terms of coming to grips with this -- perhaps on an holistic basis -- so that we can get as much of the picture as possible?

Marvasti: The "business service" is the thing that the organization offers to its customers. It runs through their data center, IT operations, and the business center. It goes across multiple technology layers and stacks. So having data collection at a specific stack or for a specific technology silo, in and of itself, is insufficient to tell you the problems with the business service, which is what you are ultimately trying to get to. You really need to do a holistic analysis of the data from all of the silos that the business service runs through.

You may have some networking silos, where people are using specific tools to do network management -- and that’s perfectly fine. But then the business service may go through some Web tier, application tier, database tier, or storage -- and then all of those devices may be virtualized. There may be some calls to a SOA.

There are deep-dive tools to collect data and report upon specifics of what maybe going on within silos, but you really need to do an analysis across all the silos to tell you where the problems of the business service may be coming from. The interesting thing is that there is a lot of information locked into these metrics. Once correlated across the silos, they paint a pretty good picture as to the impending problem or the root cause of what a problem may be.

By looking at individual metrics collected via silos you don’t get as full a picture as if you were to correlate that individual metric with another metric in another silo. That paints a much larger picture as to what may be going on within your business service.

Gardner: So if we want to gather insights and even predictability into the business service level -- that higher abstraction of what is productive -- we need to go in and mine this data in this context. But it seems to me that it’s in many different formats. From that "Tower of Babel" how do you create a unified view? Or you are creating metadata? What’s the secret sauce that gets you from raw data to analysis?

Marvasti: One misperception is that, "I need to have every piece of metric that I collect go into a magical box that then tells me everything I need to know." In fact, you don’t need to have every piece of metrics. There is much information locked between the correlation of the metrics. We’ve seen at our customers that a gap in monitoring in one silo can often be compensated by data collection in other silos.

So, if you have a monitoring system already -- IBM Tivoli, as an example -- and you are collecting operating-system metrics, you may have one or two other application-specific metrics that you are also collecting. That may be enough to tell you everything that is going on within your business service. You don't need to go to the nth degree of data collection and harmonization of that data into one data repository to get a clear picture.

Even starting with what you’ve got now, without having to go very deep, what we’ve seen in our customers is that it actually lights up a pretty good volume of information in terms of what may be going on across the silos. They couldn't achieve that by just looking at individual metrics.

Gardner: It’s a matter of getting to the right information that’s going to tell you the most across the context of a service?

Marvasti: To a certain degree, but a lot of times you don’t even know what the right metrics are. Basically I go to our customers and say, "What do you have?" Let’s just start with that, and then the application will determine whether you actually have gaps in your monitoring or whether these metrics that you are collecting are the right ones to solve those specific problems.

If not, we can figure out where the gaps may be. A lot of times, customers don’t even know what the right metrics are. And that’s one of the mental shifts of thinking deterministically versus probabilistically.

Deterministically is, "What are the right metrics that I need to collect to be able to identify this problem?" In fact, what we’ve found out is that a particular problem in a business service can be modeled by a group or a set of metric event conditions that are seemingly unrelated to that problem, but are pretty good indicators of the occurrence of that problem.

When we start with what they have, we often point out that there is a lot more information within that data set. They don’t really need to ask, "Do I have the right metrics or not?"

Gardner: Once you’ve established a pretty good sense of the right metrics and the right data, then I suppose you need to apply the right analysis and tools. Maybe this would be a good time for you to explain about the heritage of Integrien, how it came about, and how you get from this deterministic to more probabilistic or probability-oriented approach?

Marvasti: I’ve been working on these types of problems for the past 18 years. Since graduate school, I’ve been analyzing data extraction of information from disparate data. I went to work for Ford and General Motors -- really large environments. Back then, it was client-servers and how those environments were being managed. I could see the impending complexity, because I saw the level of pressure that there was on application developers to develop more reusable code and to develop faster with higher quality.

All that led to the Web application era. Back then, I was the CTO of a company called LowerMyBills.com here in the Los Angeles area. One problem I had was that I had a few people with the tribal knowledge to manage and run the systems, but that was very scary to me. I couldn't rely on these people to be able to have a continuous business going on.

So I started looking at management systems, because I thought it was probably a solved problem. I looked at a lot of management tools out there, and saw that it was mainly centered on data collection, manual rule writing, and better way of presenting the same data over and over.

I didn’t see any way of doing a deep analysis of the data to bring out insights. That’s when I and my business partner Al Eisaian, who is our CEO, formed a company to specifically attack this problem. That was in 2001. We spent a couple of years developing the product, got our first set of customers in 2003, and really started proving the model.

One of the interesting things is that if you have a small environment, your tendency is to think that it's small enough that you can manage it, and that actually may be true. You develop some specific technical knowledge about your systems and you can move from there. But in the larger environments where there is so much change happening in the environment it becomes impossible to manage it that way.

A product like ours almost becomes a necessity, because we’ve transitioned from people knowing in their heads what to do, to not being able to comprehend all of the things happening in the data center. The technology we developed was meant to address this problem of not being able to make sense of the data coming through, so that you could make an intelligent decision about problems occurring in the environment.

Gardner: Clearly a tacit knowledge approach is not sufficient, and just throwing more people at it is not going to solve the problem. What’s the next step? How do we get to a position where we can gather and then analyze data in such a way that we get to that Holy Grail, which is predictive, rather than reactive, response.

Marvasti: Obviously, the first step is collecting the data. Without the data, you can’t really do much. A lot of investment has already gone into data collection mechanisms, be it agent-based or agent-less. So there is data being collected right now.

The missing piece is the utilization of that data and the extraction of information from that data. Right now, as you said at the beginning of your introduction, a lot of cost is going toward keeping the lights on at the operations center. That’s typically people cost, where people are deployed 24/7, looking at monitors, looking at failures, and then trying to do postmortem on the problem.

This does require a little bit of mind shift from deterministic to probabilistic. The reason for that is that a lot of things have been built to make the operations center do a really good job of cleaning up after an accident, but not a lot of thought has been put into place of what to do if you're forewarned of an accident, before it actually happens.

Gardner: How do I intercede? How do I do something?

Marvasti: How do I intercede? What do I do? What does it mean? For example, one of the outputs from our product is a predictive alert that says, "With 80 percent confidence, this particular problem will occur within the next 15 minutes." Well, nothing has happened yet, so what does my run book say I should do? The run book is missing that information. The run book only has the information on how to clean it up after an accident happens.

That’s the missing piece in the operations arena. Part of the challenge for our company is getting the operations folks to start thinking in a different fashion. You can do it a little at a time. It doesn’t have to be a complete shift in one fell swoop, but it does require that change in mentality. Now that I am actually forewarned about something, how do I prevent it, as opposed to cleaning up after it happens.

Gardner: When we talk about operational efficiency, are we talking about one or two percent here and there? Is this a rounding error? Are we talking about some really wasteful practices that we can address? What’s the typical return on investment that you are pursuing?

Marvasti: It’s not one or two percent. We're talking about a completely different way of managing operations. After a problem occurs, you typically have a lot of people on a bridge call, and then you go through a process of elimination to determine where the problem is coming from, or what might have caused it. Once the specific technology silo has been determined, then they go to the experts for that particular silo to figure out what’s going on. That actually has a lot of time and manpower associated with it.

What we're talking about is being proactive, so that you know something is about to happen, and we can tell you to a certain probability where it’s going to be. Now you have a list of low-hanging fruits to go after, as opposed to just covering everybody in the operations center, trying to get the problem fixed.

The first order of business is, "Okay, this problem is about to occur, and this is where it may occur. So, that’s the guy I’m going to engage first." Basically, you have a way of following down from the most probable to the least probable, and not involving all the people that typically get involved in a bridge call to try to resolve the issues.

One gain is the reduction in mean time to identify where the problem is coming from. The other one is not having all of those people on these calls. This reduces the man-hours associated with root-cause determination and source identification of the problem. In different environments you're going to see different percentages, but in one environment that I saw first hand, one of the largest health-care organizations, it is like 20-30 percent of cost, just associated with people being on bridge calls, on a continuous basis.

Gardner: Now, this notion of "management forensics," can you explain that a little bit?

Marvasti: One of the larger problems in IT is actually getting to the root cause of problems. What do you know? How do you know what the root cause is? Often times, something happens and the necessity of getting the business service back up forces people to reboot the servers and worry later about figuring out what happened. But, when you do that, you lose a lot of information that would have been very helpful in determining what the root cause was.

The forensic side of it is: The data is collected already, so we already know what it is. If you have the state when a problem occurred, that’s a captured environment in the data base that you can always go back to.

What we offer is the ability to walk back in time, without having the server down, while you are doing your investigation. You can bring the server back up, come back to our product, and then walk back in time to see exactly what were the leading indicators to the problems you experienced. Using those leading indicators, you can get to the root causes very quickly. That eliminates the guess work of where to start, reduces the time to get to the root cause, and maybe even prevent it.

Sometimes you only have so much time to work on something. If you can’t solve it by that time, you move on, and then the problem occurs again. That's the forensic side.

Gardner: We talked earlier about this notion of capturing the normal state, and now you've got this opportunity to capture an abnormal state. You can compare and contrast. Is that something that you use on an ongoing basis to come up with these probabilities? Or is the probability analysis something different?

Marvasti: No, that’s very much part and parcel of it. What we do is look to see what is the normal operating state of an environment. Then it is the abnormalities from that normal that become your trigger points of potential issues. Those are your first indicators that there might be a problem growing. We also do a cross-event analysis. That’s another probability analysis that we do. We look at patterns of events, as opposed to a single event, indicating a potential problem. One thing we've found is that events in seemingly unrelated silos are very good indicators of a potential problem that may brew some place else.

Doing that kind of analysis, looking at what’s normal, then abnormal becomes your first indicator. Then, doing a cross-event analysis to see what patterns indicate a particular problem becomes total normal to problem-prevention scenario.

Gardner: There has to be a cause-and-effect. As much as we would like to imagine ghosts in the machines, that’s really not the case. It's simply a matter of tracking it down.

Marvasti: Exactly. The interesting thing is that you may be measuring a specific metric that is a clear indicator of a problem, but it is oftentimes some other metric on another machine that gets to be out of normal first, before the cause of the problem surfaces in the machine in question. So early indicators to a problem become events that occur some place else, and that’s really important to capture.

When I was talking about the cross-silo analysis, that’s the information that it brings out. It gives you lot more "heads-up" time to a potential problem than if you were just looking at a specific silo.

Gardner: Of course, each data center is unique, each company has its own history and legacy, and its IT department has evolved on its own terms. But is there any general crossover analysis? That is to say, is there a way of having an aggregate view of things and predicting things based on some assumptions, because of the particular systems that are in use? Or, is it site by site on a detailed level?

Marvasti: Every customer that I have seen is totally different. We developed our applications specifically to be learning-based, not rules-based. And by rules I mean not having any preconceived notion of what an environment may look like. Because if you have that, and the environment doesn’t look like that, you're going to be sending a lot of false positives -- which we definitely did not want to do.

Ours is a purely learning-based system. That means that we install our product, it starts gathering the metrics, and then it starts learning what your systems look like and behave like. Then based on that behavior it starts formulating the out-of-normal conditions that can lead to problems. That becomes unique to the customer environment. That is an advantage, because when you get something, it actually adapts itself to an environment.

For example, it learns your change management patterns. If you have a change windows occurring, it learns that change window. It knows that those change windows occur without anybody having to enter anything into the application. When you are doing wholesale upgrade of devices, it knows that change is coming about, because it has learned your patterns.

The downside of that is that it does take two to three weeks of gathering your data and learning what has been happening for it to become useful. The good side of it is that you get something that completely maps to your business, as opposed to having to map your business through a product. The downside is that it takes two or three weeks of learning time, before it starts producing some results for you.

Gardner: The name of your product set is Alive, is that correct?

Marvasti: That’s correct.

Gardner: I understand you are going to have a release coming out later this year, Alive 6.0?

Marvasti: That’s correct.

Gardner: I don’t expect you to pre-release, but perhaps you can give us some sense of the direction that the major new offerings within the product set will take. What they are directed toward? Can you give us a sneak peek on that?

Marvasti: Basically, we have three pillars that the product is based on. First is usability. That's a particular pet peeve of mine. I didn't find any of the applications out there very usable. We have spent a lot of time working with customers and working with different operations groups, trying to make sure that our product is actually usable for the people that we are designing for.

The second piece is interoperability. The majority of the organizations that we go to already have a whole bunch of systems, whether it be data collection systems, event management systems, or configuration management databases, etc. Our product absolutely needs to leverage those investments -- and they are leveragable. But even those investments in their silos don’t produce as much benefit to the customer as a product like ours going in there and utilizing all of that data that they have in there, and bringing out the information that’s locked within it.

The third piece is analytics. What we have in the product coming out is scalability to 100,000 servers. We've kind of gone wild on the scalability side, because we are designing for the future. Nobody that I know of right now has that kind of a scale, except maybe Google, but theirs is basically the same thing replicated thousands of times over, which is different than the enterprises we deal with, like banks or health-care organizations.

A single four-processor Xeon box, with Alive installed on it, can run real-time analytics for up to 100,000 devices. That’s the level of scale we're talking about. In terms of analytics, we've got three new pieces coming out, and basically every event we send out is a predictive event. It’s going to tell you this event occurred, and then this other set of events have a certain probability within a certain timeframe to occur.

Not only that, but then we can match it to what we call our "finger printing." Our finger printing is a pattern-matching technology that allows us to look at patterns of events and formulate a particular problem. It indicates particular problems and those become the predictive alerts to other problems.

What’s coming out in the product is really a lot of usability, reporting capabilities, and easier configurations. Tens of thousands of devices can be configured very quickly. We have interoperability -- Tivoli, OpenView, Hyperic -- and an open API that allows you to connect to our product and pump in any kind of data, even if it’s business data.

Our technology is context agnostic. What that means is that it does not have any understanding of applications, databases, etc. You can even put in business-type data and have it correlated with your IT data, and extract information that way as well.

Gardner: You mentioned usability. Who are the typical users and buyers of a product like Integrien Alive? Who is your target audience?

Marvasti: The typical user would be at the operations center. The interesting thing is that we have seen a lot of different users come in after the product is deployed. I've seen database administrators use our product, because they like to see what is normal behavior of their databases. So they run the analytics under database type metrics and get information that way.

I've seen application folks who want to have more visibility in terms of how this particular application is impacting the database. They become users. But the majority of users are going to be at the operations center -- people doing day-to-day event management and who are responsible for reducing the mean time to identify where the problems come from.

The typical buyers are directors of IT operations or VP of IT operations. We are really on the operation side, as opposed to the application development side.

Gardner: Do you suppose that in the future, when we get more deeply into SOA and virtualization, that some of the analysis that is derived through Integrien Alive becomes something that’s fed into a business dashboard, or something that’s used in governance around how services are going to be provisioned, or how service level agreements are going to be met?

Can we extrapolate as to how the dynamics of the data center and then the job of IT itself changes, on how your value might shift as well?

Marvasti: That link between IT and the business is starting to occur. I definitely believe that our product can play a major part in illuminating what in the business side gets impacted by IT. Because we are completely data agnostic, you can put in IT-type data, business-type data, or customer data -- and have all of it be correlated.

You then have one big holistic view as to what may impact what. ... If this happens, what else might happen? If I want to increase this, what are the other parameters that may be impacted?

So, you know what you want to play from the business side in terms of growth. Having that, we project how IT needs to change in order to support that growth. The information is there within the data and the very fact that we are completely data agnostic allows us to do that kind of a multi-function analysis within an enterprise.

Gardner: It sounds like you can move from an operational efficiency value to a business efficiency value pretty quickly?

Marvasti: Absolutely. Our initial target is the operations arena, because of the tremendous amount of inefficiencies there. But as we move into the future, that’s something we are going to look into.

Gardner: We mentioned Alive 6.0. Do you have a ball-park figure on when that’s due? Is it Q4 of 2007?

Marvasti: We are going to come out with it in 2007, and it will be available in Q4.

Gardner: Well, I think that covers it, and we are just about out of time. I want to thank Mazda Marvasti, the CTO of Integrien, for helping us understand more about the notion of management forensics and probabilistic- rather than deterministic-based analysis.

We have been seeking to understand better how to address high costs, and inefficiencies in data centers, as well as managing application performance -- perhaps in quite a different way than many companies have been accustomed to. Is there anything else you would like to add before we end, Mazda?

Marvasti: No, I appreciate your time, Dana, and thank you for your good questions.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You have been listening to a sponsored BriefingsDirect podcast. Thanks, and come back next time.

Listen to the podcast here. Sponsor: Integrien Corp.

Transcript of BriefingsDirect podcast on systems management efficiencies and analytics. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

Transcript of B2B Search Trends Podcast Based on Enquiro's Survey Findings

Edited transcript of BriefingsDirect[TM] podcast on B2B search usage, trends and future direction with host Dana Gardner, recorded June 26, 2007.

Listen to the podcast here. Watch the video here. Sponsor: ZoomInfo.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to a sponsored BriefingsDirect podcast. Today, a discussion about online search and its role in business-to-business (B2B) activities, particularly the research and acquisitions process for people who are in businesses trying to find goods and services. They seem to be using search more than ever.

We're going to look at a survey conducted last March, a fairly recent look at B2B users, their relationship to search, and how search is shifting based on the findings. Joining us to discuss this we have the originator of the survey, President and CEO of Enquiro Search Solutions, Gord Hotchkiss. Welcome to the show, Gord.

Gord Hotchkiss: Thank you.

Gardner: Also joining us to add some analysis and perspective on where search for B2B activities is going is Bryan Burdick, COO of ZoomInfo. Welcome, Bryan.

Bryan Burdick: Dana, thanks for having me.

Gardner: It seems that people, whether they're buying consumer goods, small business supplies -- anything from Gorilla Glue to guided missiles -- are using search at some point in this process. Some people start and end with search. They actually buy the products through a search process. I want to start by understanding a little bit about the survey itself and the fact that it’s the second survey that’s been done on this B2B search activity, the first being in 2004.

So, let’s go to Enquiro with Gord. Tell us a little bit about the history and some of the major bullet points of this particular survey?

Hotchkiss: As you said, we did the original survey in 2004 and, at the time, there wasn't a lot of research out there about search in general, even on the consumer side. There was virtually nothing on the B2B side. We knew search was important, just from what our clients were saying and the results we had had, but we hadn’t done anything extensive enough outside our client base to start to quantify how important it was.

The first survey was an attempt to do that. It certainly proved that search was important. We found that online activity, in particular that connected with search activity, was consistent in a large percentage of purchases. In 2007, we added more insight to the methodology. We wanted to understand the different roles that are typical in B2B purchases -- economic buyers versus technical buyers versus user buyers. We also wanted to get more understanding of the different phases of the buying cycle.

We structured the survey so that we could slice the data based on those parameters and get more insight into those areas. As far as the main takeaways from the study, obviously online activity is more important than ever. In fact, we asked respondents to indicate from a list of over 30 influencers what was most important to them in making the purchase decision. Online factors, such as interaction with the vendor Website and interaction with the search engine were right up there with the traditional winner, word of mouth. What we see is a real link between those and looking for objective information and specific detail.

A lot of that search activity happens on specific properties, and we’ll be diving into that a little bit later in the podcast. We did notice an evolution of behavior as you move through the funnel, and the nature of the interactions with the different online resources changes how you navigate to them and how you go to different sites for information. But, online research was consistent through the entire process, from awareness right through to purchase.

There’s a lot of back and forth. Offline factors influence online activity and vice-versa. So, we saw a merging of the online and the offline worlds in making these decisions and trying to come to what’s the right decision for your company or what’s the right product or service.

Gardner: Tell us a little bit about Enquiro. You are a marketing, consulting, and research firm. Is that correct?

Hotchkiss: Yes. We work with clients in putting together their search campaigns in the B2B space, but we also have an active research arm. So, we're continually doing research primarily on the usability and qualitative analysis side, but we do survey work as well. The purpose of that is to provide more insight into how consumers use search and how businesses use search to make buying decisions.

Gardner: Three years between these surveys is probably not a lot for many businesses, but it’s a huge amount of time in the search industry. What would you say was the biggest difference in your results and findings over this three-year period?

Hotchkiss: Surprisingly, we didn’t notice huge trend differences in the three-year period. If anything, we saw increased reliance on online factors and probably just more activity online and more interactions with sites, but it was the continuation of a trend we saw in 2004. We didn’t see any big shifts. We just found increased reliance on online to do that research.

When we say "increased reliance," we're probably talking 10 percentage points up over the three years. So, if 65 percent of the people were doing it in 2004, 75 percent of the people are doing it now. That’s primarily where we saw the trends going.

Gardner: And, you say that that you're also seeing search applied to this process in different ways and different facets. For example, word of mouth would tip someone off to go look for something, and the first way that they look for it is by using search.

Hotchkiss: Yes. When we looked at the different phases of the buying cycle, it starts with awareness. You become aware that you need something. There was a high percentage of people -- in the high 60-percent range -- who said, "Once I become aware that I need something, the first place I'm going to go is the search engine to start looking for it."

A lot of that traffic is going to end up on Google. It was the overwhelming choice among general search engines for B2B buyers.

But, as you move through the process, you start doing what we call a "landscape search." The first search is to get the lay of the land to figure out the information sites that have the information you are looking for. Who are the main vendors playing in this space? Where are the best bets to go and get more information to help make this purchase decision?

So, those first searches tend to be fairly generic -- shorter key phrases -- just to get the lay of the land to figure out where to go. As you progress, search tends to become more of a navigational shortcut, and we’ve seen this activity increase over the last two to three years. Increasingly, we're using search engines to get us from point A to point B online.

As you get into the buying process, you’re familiar with the vendor site. You’ve been on the site. You’ve checked different product information pages. As you come back to that research process, you say, "I really want to find that product spec sheet that I saw on this vendor site." A lot of that navigation to those specific pages happens through a search engine. So, there are multiple touch points through the process.

Gardner: Now, you did this search in March, and you surveyed 1,086 people, North Americans, mostly women -- 63 percent -- average age 43 years old, with 67 percent of them having at least attended university.

Hotchkiss: Right.

Gardner: Can you tell us little bit more about these people? Are these people that you acquired through strictly business activities? How did you know that they were in a purchasing mode?

Hotchkiss: When we structured the study, we used our sampling partner, Survey Sampling International, for access to their B2B decision-maker panel. In two different parts of the survey, we asked, “Are you currently considering a purchase in excess of a thousand dollars?” That was a qualifying question. If they answered yes, they got to continue the survey.

That’s how we determined what role they were playing in this purchase that was happening right now. What were they considering purchasing? What was influencing them? We wanted to use a purchase process they were in the middle of, because it would obviously be fresh in their minds and they could really tell us what they were going through, as far as what influenced them.

We also wanted to get a retroactive view of a successful transaction. So, in the second part of the survey, we asked them to recall a transaction they had made in the past 12 months. We wanted to see whether that initial search led to a successful purchase down the road, and, at the end of the road, how the different factors influenced them. So, we actually approached them from a couple of different angles.

Gardner: Now, 85 percent of these people say they're using online search for some aspect of this purchasing process. It strikes me that this involves trillions of dollars worth of goods. These are big companies and, in some cases, buying lots of goods at over a hundred thousand dollars a whack. Do you concur that we're talking about trillions of dollars of B2B goods now being impacted significantly by the search process?

Hotchkiss: Absolutely. The importance of this is maybe the most mind-numbing fact to contemplate. Traditionally, the B2B space has been a little slow to move into the search arena. Traditionally, in the search arena, the big advertisers tend to be travel or financial products.

B2B is just starting to understand how integral search is to all this activity. When you think of the nature of the B2B purchase, risk avoidance is a huge issue. You want to make sure that whatever decisions you make are well-researched and well-considered purchases. That naturally leads to a lot of online interaction.

Gardner: I suppose if I am making a $100,000 purchases, and I make a mistake, I am not going to be around for long. Right?

Hotchkiss: Exactly. The other thing is that we don’t tend to be as emotionally involved with the B2B purchase. Things like branding play different roles than when you're doing consumer purchases. The brand affinity is something you might have if it’s an area where you don’t have a lot of experience. It may be a new need that’s coming on the horizon for your business. You are really starting from Square One, and that’s the perfect place for search to plug in and be the solution when you start researching those purchases.

Gardner: Right. These folks are looking for practical approaches and real information. Let’s go to Bryan Burdick at ZoomInfo.

This is growing quickly as an overall trend, but ZoomInfo, which is focused on business search, is growing much more rapidly. What’s driving your growth at ZoomInfo, and how does that relate to this B2B search activity?

Burdick: The business information search is a primary factor driving our growth. Our company right now is growing on two fronts. One is our traditional paid-search model, where we have subscription services focused on people information that is targeted at salespeople and recruiters as a source for candidates and prospects.

The more rapidly growing piece of our business is the advertising-driven business information search engine, which I think is a really interesting trend related to the concept you guys were just talking about. Not only does the B2B advertiser spend lots of money today trying to reach out, but the B2B searcher has new tools, services, and capabilities that provide a richer, better, more efficient search than they’ve had through the traditional search engines.

Gardner: By far, the largest player in this is Google with, according to the Enquiro survey, 78 percent use by this group of respondents. Way down the line was Yahoo!, and then far below that was MSN. It strikes me that Google is a general search engine, and yet we are asking for very specific business information.

Bryan, do you expect that we are going to see some sort of a specialization or a cleaving between general search and more vertical specialized search?

Burdick: Absolutely, and, in fact, that’s really ZoomInfo’s mission -- to do for the business-information search world what the Expedia or Travelocity did for travel search. When you think about it, you can actually go to Google and find an airplane ticket, but why would you?

It’s so much more efficient to go to one of these vertical search engines that are looking at the databases, looking at the data, and indexing it in a much more efficient way. You're starting to see that in a lot of other verticals. Travel has been the quickest to adopt that, but everything from business information, like ZoomInfo, to podcasts with Podzinger, and other types of vertical searches, have been focusing on a niche and organizing the content more efficiently.

Hotchkiss: One thing we found in the survey is that there's a natural evolution through the process. Although you might start on Google as you are trying to find those information sites, quite often it’s the verticals that people work into as they are starting to look for specific, more granular information on the companies they're thinking of doing business with. That’s where ZoomInfo and other vertical players fit a need.

Gardner: I suppose another thing that seems the same from 2004 to 2007 is the importance of a supplier’s Website. According to your survey findings, people are very interested in word of mouth. They use the search engine to move from that point to gather more information, but they're very quickly interested in solid, simple, straightforward, text-based information from the suppliers themselves. I suppose that underscores the need for sellers to have a very strong Website.

Hotchkiss: That was a really strong finding, and not really surprising. It made sense, but I think how important the straightforward information was to the people doing the research was somewhat surprising. It’s one of those things where you get findings in research and then afterwards, when you apply common sense to it, you say, "Yeah, that just makes sense."

But, remember these searchers are out to gather information for an organizational buying model. They are gathering information that will, in a lot of cases, be passed on to other individuals to help them make the decision as a group.

You don’t necessarily want to sit through a linear presentation of information, like an online video, or even a podcast, if your intent is just to pickup specific data and pass it along. Now, if you are the user, and this is that something you are going to be using, you may be a lot more open to a linear how-to demo. But, it’s important to match the content on your site to the types of buyers and individuals who are gathering the information.

The takeaway we got from this was to make sure you're covering the basics first. Make sure that you're getting the clear concise product information out. In a lot of cases, you know you are going to be compared to the competition. Why not enable some of that to happen on your site and make the buyer's job easier by arming them with the necessary equipment.

The number one thing that came across as desirable was pricing information, which is really tough for B2B marketers to put on the site, because in a lot of cases these are complex solutions. But, what the buyers are looking for is qualifying it in a budget range.

Is this a $10,000 purchase, a $100,000 purchase or a million dollar purchase? I need to know that to qualify, so I can move on. Please give me that information. It can be ranges. It doesn’t have to be specific, but I need to be able to qualify it.

Burdick: Dana, I would add that some of the typical mistakes that a B2B marketer will make from a search-engine marketing perspective is jumping too quickly or focusing too much on the actual advertising piece. They need to do that, but sometimes they forget about the search engine optimization. Very often, they leave that up to the technical team, which may optimize the search or the site in ways that aren't optimal from a marketing perspective.

Then, as Gord was saying, they get the user, or the potential customer, in there and the customer gets lost on their own site, searching for the type of information that they're looking for. It’s not like a consumer model, where the consumer already knows, in most cases that they are looking for a DVD player or whatever it is. They may even have a model number, and they're looking for the best price online. It’s much more of an information-gathering process in the B2B case.

Gardner: Perhaps the takeaway here would be that people want to get just the facts upfront and they want a ballpark figure, but they also want to be able to use search to get to that information fairly quickly.

So, if you’re going to optimize your site, your key information can’t be 18 pages deep into the search process, but you also have to consider that factual information needs to be as accessible as your main branding page.

Burdick: Absolutely.

Hotchkiss: One more point on that. A lot of B2B marketers like to capture as much information about a lead as early in the process as possible, so it can be handed over to the sales department, which can close the sale.

But, what happens in a B-to-B purchase is that the first touch point with your vendor Website is typically not the decision maker, the ultimate decision maker? It tends to be somebody who is gathering information to help arm the company to make that eventual purchase decision.

So, if you push to establish contact with that person, they're not ready to establish contact with the vendor, because they don’t have the buying power. Even if you do push to get it, your salespeople are spending a lot of time following up on leads that aren’t qualified buyers. They have to retrench down the road and try to re-establish connections with the person who has the economic power.

So, it’s really a "date," and, in a lot of cases, it’s a long series of dates. If you push too fast you are just going to push the prospect away.

Gardner: You don’t want to propose on the first date.

Hotchkiss: Yeah.

Gardner: On the other hand, one of the findings from the survey was that 50 percent convert to a sale online. So, that means that when the research, winnowing, triage, and the comparative shopping are over, the economic buyer, who is empowered to make a decision, will go back online and consummate the deal. Does that make sense?

Hotchkiss: Yeah. Here's some further insight into that, because we saw that number surprising when we did the overall data. When we pulled the data apart, we found that a lot of those purchases tended to be things like computer systems, where they might have been buying through a Dell or someone like that.

Gardner: The direct model.

Hotchkiss: We thought that was a really high online conversion rate. As we looked at the data a little more, we saw that in a lot of cases it was smaller software purchases or system-based hardware purchases. That made a little more sense, as we went down that road. There was a fairly strong manufacturing component, where people were buying parts and different things. In those cases, a lot of those purchases are made through an electronic marketplace. We're seeing that as an increasing trend too, e-commerce-enabled market places.

Gardner: I suppose it's also logical that when the price or the total purchase price is less than $50,000 or closer to $10,000, they’ll be more likely to do that online confirmation and make a purchase. To me, this says that small- to medium-sized businesses selling small types of goods should be very focused on search and B2B online activities. Does that follow?

Hotchkiss: Everybody needs to be focused on search. I can’t see an exception. You mentioned the percentage that said they would go online. We segmented out the group that didn’t indicate they go online to see what was unique about them.

The only thing unique about them was their age. They tended to be older buyers and tended to be with smaller organizations, where the CEO was more actively involved in the purchase decision. That was really the only variants we saw. If it’s a generational thing, then obviously that percentage is going to get smaller every year.

Burdick: Dana, could I ask Gord a follow-up question to that?

Gardner: Of course.

Burdick: I'm curious whether, as you dug into the data, you saw any differences between online follow-throughs to purchase on hard goods versus services. I'd think that people buy computers online, but if they're buying services from a B2B company, that tends to be offline.

Hotchkiss: When we were looking at influencing factors, B2B services was the one where word of mouth edged out online factors by a significant margin. When you're trying to retain a service, word of mouth is still very influential. In pretty much every other category, online was right up there with word of mouth, in some cases edging it out as an influencing factor.

Gardner: Okay, another number from this was that 95 percent said they use search to find what they want at some point or another, but 37 percent were still seeking other sources. There seems to be a recognition that search is very powerful, that it’s a tool that shouldn’t be ignored under any circumstances, but that it's not getting the entire job done. Bryan, I wonder if you could respond to that. What else needs to happen in order for these people to get what they need?

Burdick: The short answer is they just need to come to ZoomInfo.com. Seriously, I don’t think it’s a matter of needing more information, but, in some cases, finding better information. When you think about the traditional search engines -- the Googles and the Yahoos! of the world -- there are so many consumer-oriented search transactions on a day-to-day basis that they have optimized their experience with the consumer in mind.

Search engines like ZoomInfo and some of the other business-information search engines are taking a different approach and optimizing the search experience, and therefore the relevance of the results, with the business-information searcher in mind. You can much more efficiently and quickly get to the information you're looking for.

The simple example that I like to use is that if you search for "enterprise routers" on Google, you are going to get 32 million results that will include everything from Enterprise Rent-A-Car to the latest episodes of Star Trek. Search for that on a business search engine like ZoomInfo and you'll get 150 companies that sell enterprise routers or manufacture enterprise routers. It just becomes a much more efficient process.

Gardner: Well, even the alternatives cited in this survey are still very general. There was Business.com and Thomas Register. This is every good under the sun. It might as well be a general search. KnowledgeStorm was also mentioned, but it's very specific to IT. It seems like there’s a whole category that needs to be filled here around vertical business search.

Burdick: The original survey that Enquiro did for us in 2004 was a key factor in our decision to move to more of a search-engine model, because what we see happening is the same kind of evolution that happened with the big search engines way back. It’s happening now in the vertical-search categories, where search engines started out as directories or, if you think back to Overture in their early days, totally paid listings.

Eventually those two forces came together to provide a best-of-both-worlds situation, where you’ve got not only great relevance on the results, but also great relevance on the targeted ads, and now that’s starting to happen in the verticals as well.

So, you’re starting out with some of the business-directory sites or the business paid-listing sites, because those are easy to do. As the technology gets more sophisticated and you can provide more relevant results for the business information seeker, you are delivering the value that the information seeker is looking for. Plus, you can target the ads better and provide an overall better experience.

Gardner: Let's go back to Gord on that. The survey found a larger percentage of people looked at the organic results on the left, but they were pretty much limited to the top four. A smaller percentage, about 12 or 13 percent, said that they look over to the right-hand side for the ads. That 13 percent might sound small, but compared to a click-through rate in a Web advertising model of usually less than 1 or 2 percent, 13 percent is still a pretty large chunk of people. What’s your impression of the impact of the advertising model in search for B2B activities?

Hotchkiss: Those percentages, by the way, aren’t that different from what we've seen in consumer-based research. Those breaks between organic and sponsored tend to remain fairly consistent across a number of different channels. One thing that’s just golden about search is it will connect a motivated and engaged user with the content they are looking for.

If that content is provided by a relevant ad, then they're open to that. In fact depending on where they are in the buying cycle, they may even be biased toward that, because they are ready to get information from somebody who is trying to provide them what they need to decide whether this is the thing they need to buy.

There's a totally different interaction when you're on a search engine actively engaged in a task and actively looking for information. You're far more receptive to messaging at that point. You're actively looking for it. And this seems to be slowly breaking into the consciousness of most advertisers. They're getting it, but they're getting it slowly. Anyone who moves into the search space, if they do search in a smart way and they capitalize on the potential of it, is just amazed by the return they get on this.

Gardner: Let’s beg a little more from these results than was intended. I started to see in the findings some patterns about typical processes, about how people would go about this activity -- the awareness, the research, the purchasing, and so forth. It seemed to me that there was, on one hand, a pattern of word of mouth that led to a search, that led to a Website, that then led to a refined search based on what they found, which then led to a hand-off to a purchasing activity by maybe a different department or individual.

There also seem to be offline influences, perhaps trades shows, perhaps traditional sales calls and activities, but that also took into consideration word of mouth that then identified what to search on, and so on. Am I reading too much into this, Gord, or were there patterns of process around the use of search in the purchasing activity?

Hotchkiss: It goes back to what I said before. Search tends to be the thing that connects you, as you move through the buying process, and it’s used in different ways and places as you progress through that. As far as identifiable patterns and usage behavior, if you did an end-up study of a large enough dataset, patterns would emerge. They always do emerge, but I'm not sure we would be confident enough diving into this particular dataset to try to tease that out of it.

What we did see is that B2B buying decisions are tremendously complex when they are compared to an individual consumer buying decision, where you have one person going through all the phases. You have multiple individuals influenced by different factors going back and forth.

What is consistent in that is whatever is influencing you -- whether it’s online or offline, a discussion with a colleague or a recommendation by a paid consultant -- there tends to be a mirror activity, in which whatever happens offline generates some kind of online activity that typically is initiated through a search engine. Then, you pull that information back, and it dances back and forth between the online and the offline world.

Gardner: So, there’s a barrier here in some senses. I'm sure most companies, especially the larger ones, have a standard operating procedure about how purchases will happen. It will be form x, y, z, and it will go through process review a, b, c, and then we’ll have to get signatures from individual g, b, h. How can we bridge this value that people see in search, and somehow bake that into a procedural process inside of an enterprise. Or are we asking too much here?

Hotchkiss: One of the interesting things, being both a researcher and a vendor, is we get to see both sides of it. We have access to more information than ever before, and buyers out there are armed with better information before they ever initiate contact with a vendor. They are gathering a lot of information and then they are trying to cram it into an existing buying process, whether that’s an RFP process or whatever.

So, like most things with the Internet, the traditional systems are being challenged by this new access to information that we never had before.

Over the next two to three years, what we're going to see is organizational buying processes being streamlined and being able to incorporate the fact that you have better informed buyers than you may have had before. The whole RFP process was to eliminate risk. The reason was make sure that you are considering alternatives and to make sure that you are almost forced to gather the information you need to make a dispassionate judgment about what was the best choice.

Now, in a lot of cases, the decision is already 80 percent made before the RFP process ever begins. Somebody has researched a vendor, has a very strong feeling about the vendor but now has to try to fit that into the established procedure, and they say, "Okay, now we've got to go out and find two or three more alternatives."

Heaven help the other two or three alternatives that are getting involved in that process. They have to go through the whole dance, but usually the preferred vendor on the front end gets the business on the back end. The other two or three players are just used as negotiating chips to try to get the price down. It’s interesting to watch how information from the Internet is changing virtually everything out there. This is no exception.

Gardner: This might be a little bit out in the future, but the role of search could morph into the role of auction and brokering activities. Does that make sense?

Hotchkiss: Yes, and for players in the space -- I suppose ZoomInfo as well -- if we can streamline the marketplace, if we can take this access to the information and make the buyer’s job easier, that’s a tremendous saving. I would hate to think of the number of man-hours that are invested internally in an organization for a fairly major purchase decision. How much more efficient you can make that process by simply empowering them with the information that they are going out to look for anyway?

Gardner: Bryan, you said that the Enquiro survey in 2004 made an impact on ZoomInfo in terms of its direction. Have the findings from 2007 had a similar influence? What new directions might you be heading in?

Burdick: As Gord had said earlier, the findings in 2007 reinforced what they had already learned three years ago. So, in one sense, it confirmed our own strategic direction as well. We re-launched the ZoomInfo.com site back on April 1, and moved into more of a traditional search-engine model, where all of the content, all of the search capabilities on ZoomInfo.com are free and subsidized by advertising.

We’ve seen that piece of our business take off like crazy in the last couple of months. Search traffic has grown dramatically. We’re up to close to 5 million unique visitors a month doing about 16 or 17 million searches a month on our site. All that is really driven by this need, this desire, among B2B buyers to find a more efficient process to get at this type of buying information.

Gardner: What advice would you have for those folks that are on the selling side of this? What should they do to position themselves in order to take better advantage of what’s occurring on the buyer side, particularly, in their use of search?

Burdick: There are a couple of landmines or traps to avoid. The first is to try to avoid competing with the consumer brands, either on the traditional engines or with the same types of keywords. If you are buying the same keyword as a consumer brand is buying on one of the traditional engines, you are typically going to get drowned out by the noise.

The other thing is to make sure that your own search marketing is coordinated with channel partners. I’ve seen lots of examples where the manufacturer is buying a set of key words and their value-added resellers are buying the same keywords. They end up bidding up the same keywords just to attract the same eyeballs. At the end of the day, the manufacturer is going to direct them to one of the resellers anyway.

Gardner: So that leads to confusion, rather than streamlining that particular process.

Burdick: Exactly. The other key thing, which we already touched on, is leveraging not only on the marketing side, but the search-engine optimization of your site in general, and optimizing the search-for-information experience that the buyer has once they get to your site.

Hotchkiss: One thing I would add on a more fundamental note is to make sure that the perspective you are using to evaluate your search strategy is the customer’s perspective and not your internal one. One of the common pitfalls we see is companies get into a bad case of "internal think." They look at everything from their internal perspective, and they are not shifting the looking glass 180 degrees and looking at it from their prospect’s perspective.

It’s amazing how enlightening it can be, when you start looking at what types of sites they are going to. You need to catch their attention, and know what kind of messaging you have to present and what kind of onsite experience you have to present, once you are successful in capturing the click. If you can force yourself to see from that perspective, you're going to do more to improve the effectiveness of your campaign.

Gardner: Well, thanks very much. We’ve been discussing the recent survey by Enquiro Search Solutions. It was Enquiro’s 2007 B2B search survey, and to help understand it better we’ve been joined by Gord Hotchkiss, the president and CEO of Enquiro. Thanks, Gord. Is there anything meaningful in the survey that we didn’t cover and that you think we should?

Hotchkiss: The original release covers the high-level findings. What we're working on now are three follow-up white papers that will be available on the site through the next three to four months. We're going to take each of the three buying roles that a lot of people within the survey fell within -- the economic buyer, the technical buyer, and the user buyer.

We're going to do insight from that particular buyer on how to search more effectively, plod through the process a little bit more, and how those hand-offs happen from role to role. I imagine there are going to be new insights out of that. We're taking different slices of the data. So, for those of you who are interested in that, keep checking out on Enquiro.com and we’ll ping you as those white papers become available.

Gardner: Terrific. And we’ve also been joined by Bryan Burdick, the chief operating officer at ZoomInfo. Is there anything that you’d like to cap the discussion with, Bryan?

Burdick: Only that I think from a vertical business information search perspective, that we’re really in the first inning here. A lot of interesting trends and enhancements are going to be coming down the road. One in particular that may have an influence in the next year or two is the community aspect within the search.

Gord, talked earlier about how there are multiple people that influence the B2B buying decision. I think that you’ll start to see a marriage of, not only B2B search, but also online community and a factoring into that whole process. Then, who knows where we’ll go from there? But I appreciate you having us on.

Gardner: I suppose that this notion of word of mouth being so important in shaping people’s direction that you might use search to find the word of mouth.

Burdick: Right, the word of community.

Gardner: There it is. Okay, well thank you, Bryan. This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. Thanks for joining.

Listen to the podcast here. Watch the video here. Sponsor: ZoomInfo.

Transcript of Dana Gardner’s BriefingsDirect podcast on B2B search usage, trends and future direction. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.