Transcript of BriefingsDirect Podcast on the Future of Enterprise Search

Edited transcript of BriefingsDirect[TM] podcast on search advancements with host Dana Gardner, recorded Jan. 3, 2007.

Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today a sponsored BriefingsDirect podcast discussion on search, the role of search, how search relates to today’s mega trends of Web 2.0, Enterprise 2.0, Services Oriented Architecture (SOA) and even mobile lifestyles.

Joining us in this discussion are executives from FAST Search & Transfer, a global company based in Oslo, Norway. Joining us is John Markus Lervik, the CEO. Also, Bjorn Olstad, CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. Welcome to the show everyone.

John Markus Lervik: Good morning, Dana.

Gardner: While search has been around for many years, it seems like we’ve had somewhat of a philosophical shift in the last few years, in the sense that people are no longer thinking of information based on its origins, but really they’ve supplanted these data structures with how they can use it and how they can access it. That cuts across desktop, intranet, and Web; and it seems that access to information quickly and in the way that any individual wants it has really changed the notion of fleet and agile business.

In fact, business process is increasingly being driven by how people access content and then inject that into business goals. So, I want to first talk to you about business goals because many companies are very practical, of course. They’re looking at their top line and they’re looking at their bottom line. I want to throw this out perhaps to John Markus, how are companies using information discovery to further their business goals in a way now that’s perhaps different from ever before?

Lervik: Dana, I think that one way of looking at technology is always in the light of the business processes that they’re enabling. Many of our customers are thinking about ways in which they can achieve competitive advantage through productivity change, and they’re looking for something new. What we enable is for search to help facilitate communications between individuals.

We’ve all heard of Web 2.0 applied to the "Websphere," but in the enterprise, Enterprise 2.0 trends of social networking are also taking hold, allowing people to collaborate better and allowing individuals to streamline business processes. What's interesting for us in the future is search as an entry point into facilitating improved business processes, and business process improvement leads to improved business goals, as well as business-model innovations.

Gardner: Is there an impact here for applications themselves to start to be created based on what search can bring to the table rather than having search perhaps be an afterthought? What do you see as the future of search-driven applications?

Bjorn Olstad: This is Bjorn, and I'll have a go at that. The fundamental difference between search and such things as database, content management, and document management systems is that search starts with the user and then does reverse engineering -- what is necessary to realize this user's experience. It's starting with the content and then trying to deduce what should we do with this content. When you do that, you end up with a user-driven experience.

As you’re starting off, Dana, the user doesn’t care about the origin of the content. So, the ability to go into content sources and then chop that into information nuggets that can then be re-orchestrated and put into the context of what is useful for that specific individual user in what he’s trying to accomplish right now is a very powerful paradigm. It’s also a paradigm that can be used as the basis for building applications.

We see search more and more becoming the framework for the metaphor in how the user interface is being defined and also for how the application is actually defined by the source of the framework and then embedding functionalities in a source-stream framework.

Gardner: Is this really a shift of intent or are there some technologies that are newly arriving that empower this shift? Bjorn, can you tell us a little bit more about what’s happening technically that is allowing an expansion of the role of search?

Olstad: Let's go back a little bit. As you said, search used to be an afterthought. Typically, you had an application that was kind of hard-wired, done in HTML, or hard-coded down to the logic that usually resided on some type of database. So there was hard-wiring between the schemas and the data model and the database, and then the user interface.

Going forward, I think increasingly the user experience will be driven by algorithms and will be dynamic, so that you can actually optimize the user experience and the business efficiency. This is valid for both consumer-facing portals that you’ve seen on the Web but also on other applications internally. So then you get that framework and you say. "Let’s not hire system integrators to hard wire my content. Let the user decide, and when we deduce what the user likes, the algorithms can populate the user experience."

This will include both content and relevant services, and maybe a connection to people, so that you can find people to collaborate with. That becomes a framework, and search has matured from being merely access to documents to being this umbrella environment where you can provide access both to structured data and unstructured data and transform them into structure by putting additional metadata onto it. You can do the same with rich media, and search suddenly becomes that framework that can give you access to everything, as opposed to just the text that didn’t fit into the hard wire or the schema model.

Gardner: I suppose that with the increasing use of SOA, where applications are decomposed into services with the productivity coming from reuse as much as possible, and with governance providing an overlay for managing and organizing services, that search could have quite a powerful impact on how services are put together on the fly to create unique business processes. Do you see search moving in that direction as well?

Olstad: Absolutely. If you think about Web 1.0, Web 2.0 and Web 3.0, one way to interpret this is that Web 1.0 included these monolithic Web services. Then, in Web 2.0, it’s about the different services that make up atomic individual services and then having technologies to mash up and combine them to create new services on top of that, the same way as SOA. It seems nice at first because they are driven toward reuse, et cetera, but you move the complexity to the next level. How do you actually orchestrate? How do you put together these atomic content components and atomic service components? How do you actually do that and how to do that in a user-centric way, so you can gain the right user experience?

With search, it’s actually designed to do exactly that, to bring both components and services to atomic units and then bring the orchestration level that is able to recompile these content and service elements into what makes sense for the end user. Hence, it brings 3.0 integration levels on top of the atomic components that we have in Web 2.0.

Zia Zaman: This isn’t just our idea, if I can build upon that. I was talking to an information provider that provides information to financial services firms. What he asked for was the ability to atomize information, allowing the users the flexibility to consume it in their own way. So, it’s exactly what Bjorn was talking about. In Enterprise 2.0, we allow for the atomization. And then the re-organization around the business processes is what happens in the next phase.

Gardner: Zia, there is already quite a bit of difference of opinion on how to define Web 2.0, never mind Web 3.0, but let’s try to just get a handle on Web 3.0 for our audience. To me this is really getting towards the Semantic Web, where there is context on an automated basis, with much more rich metadata involved, and therefore the ability to apply algorithms more generally across the Internet. Is that your understanding at FAST about this Web 3.0 or how would you further that definition?

Zaman: Dana, I think you've got it right. Metadata is the key, but I'm going to pass this one over to Bjorn to talk a little bit more about Semantic Web.

Olstad: There are so many opinions about the Semantic Web. First of all, what I don’t like with the approach of Semantic Web is that you have to put in structure, and it’s not really useful until almost everybody has started to put structure into it.

So, how do you leap from that and actually make it useful before everybody has adopted this new scheme? Search can actually play a role, because search can auto-generate metadata and find ways to use that structure and to improve the discovery. Then, the allocation elements that the traditional Semantic Web talks about can be aimed at how to improve algorithms, as opposed to starting from scratch. In doing that, I think search has the opportunity to deliver on the premise of the Semantic Web, by applying algorithms as opposed to altering tools.

If you then move to Web 3.0, then using algorithms increasingly becomes a component of it. I think an immediate step, which is an opportunity that FAST is looking at right now, is how can you use search at this level to go from atomic services to the rich environment, where you effectively can develop and deploy services. That’s kind of the orchestration: how you put services and content components into meaningful context for the end users. That is what brings business value. It brings time to market value for companies that are trying either to develop Web 2.0 and Web 3.0 applications or move their current models to a more collaborative approach.

Gardner: I suppose one of the frustrations we’ve heard quite often from users is that they can find things easier on the World Wide Web, than on their own hard drive or their own enterprise network. We’ve come a long way to assuage those concerns, but it seems to me like perhaps we’re at a point where we can go and leapfrog the Web.

If we’re thinking about a Semantic Web, that’s something that requires a lot of standardization. It could take a number of years and a high level of complexity, but moving to the "Semantic Enterprise" seems something that’s quite a bit more attainable in the short term. Do you think, and I’ll throw this out to anyone, are we going to move to a point where we can apply these benefits to the Enterprise and then the Web will catch up to them or is the Web always going to be a bit easier to manage?

Olstad: Maybe I could start, and, Zia, you could give some business examples. This is an excellent question, Dana. First of all, just so that we’re on the same page about what the Web experience is all about, there are two types of queries. There is the type of query where you know what you’re looking for. So you search for some properties and what you’re actually looking for.

The other type of query is when you don’t know what's there, but you throw out a wide, open-ended query, and you want the data to talk to you. In Web search, there is no completeness requirement. Web search is defined when you type in one or two query terms, which is a nice starting point for your browsing. So you get a link to a Website where you can start. There is no completeness. There is no overview of the other elements that didn’t end on the first result page.

Web search has moved outside Web search and can be used inside the Enterprise for this semantic analysis. It can also identify facts inside documents, combine it with the information structure databases, and provide discovery. You can start to do analytics, so that you can relate concepts and enable people to make decisions and get completeness and overview. That is why you see a merger between business intelligence and search starting to appear.

Gardner: So, perhaps moving toward a business intelligence (BI) benefit vis-à-vis search, and of course the other tools that are available for BI, is it first a necessary step to the Semantic Enterprise?

Olstad: Yes, I think so. To give just one example of a company that is trying to use search in an innovative way, what about all communication internal in the company? What about instant messaging, email, Voice over IP (VOIP), Skype, et cetera, and how to improve the quality of that communication and also derive analytics from it.

For example, if I call you, Dana, I should call you as a conceptual entity and you should actually be the one who decides if that ends up as a Skype call or cell phone call. This company is doing that and using search to orchestrate all the communication, allowing you, as the recipient of the call, to configure how you want to be accessed. By having that system, you can also create analytics that understand what is really happening in the company: who are the experts, what’s happening, what are the trends, etc.

Zaman: I actually think that it could be a lot simpler than that. The expert location is something that Bjorn just touched upon, which may be one of the next killer apps in the enterprise. What we’re finding more and more when we talk to our customers is that information discovery isn’t just about connecting users to results. It’s about connecting users to answers, to analytics, and to services -- but most importantly, to people. Connecting people to other people, so that they can do their job better. To improve decision-making throughout the enterprise is a big part of what information discovery tools are about to enable.

One way of looking at it is, if I can find the expert in the organization or outside the organization who can help me best in the decision process that I'm currently in -- and I can find that expert in the shortest period of time, using insights about who this person might be by using metadata about this individual, by actually discovering this path that I didn’t know existed upfront -- then I have actually added value to the business. So, we believe that search as a matching service is really one of the killer applications.

Gardner: So, what we’re talking about here is bringing the tacit knowledge that’s available to any organization into play with the semantic knowledge that’s gathered across all modalities, all types of media and information, but allowing those to come together in real-time based on someone’s specific needs through a very simple query.

Zaman: That’s right. I often like to talk about Greek philosophers, and Socrates is one of my favorites. The reason is because of what he did with a seeker. He didn’t answer a question with an answer. Rather he asked another question, and that allowed the seeker to refine his or her question, until finally they got to what they were looking for. In many ways, technology -- whether it’s information discovery, search, and business intelligence, whatever it might be -- shouldn’t insult the intelligence of the seeker. Rather it should allow individuals to find the answer that they’re looking for, either through tacit information that’s stored in the enterprise, or semantic information, or structured information, whatever it might be. What we’re trying to do is mimic the type of dialogue that Socrates had.

Gardner: Sure, because people are quite good at peeling away the layers of an onion, as a metaphor, to find what it is they’re looking for. There is no beginning or end to this peeling, but we certainly want to get him or her close to where they can extract real value.

Zaman: You got it.

Gardner: All right. Now people have been talking about this for years. I deal with developers a lot. I think they understand a lot of these theories. They understand the value and the goals, but putting it into practice, putting it into execution seems to be where there is still a need for some work. When we go to developers, we try to encourage them to start exploiting these capabilities of changing the mindset of how an application, specifically the UI, is actually constructed to make the entry point into an application more amenable for search and for gathering of these layers within an organization’s assets and resources.

Are there tools? Are there APIs? Are there SDKs? Is there a mindset? How do we bring the developer into play here? I’ll throw this out to anyone.

Olstad: I think Gartner predicted that in 2012, 75 percent of the world's applications will use search as the dominant metaphor for how an application is being operated by the end-user. That is a very powerful statement. And it means that all those developers that you’re asking about, Dana, they have to adapt to that.

Simplicity in an application [means] that you have a powerful metaphor that you can use to deduce what you should expect the application to be doing. Search is exactly that. So, obviously, search is evolving to not only provide access to text, but to provide value to all types of elements.

Another change in search technology is that it handles content natively, and previously it did not. In Web search you type some queries and you get a URL link to the content. Now you can take XML or SQL databases, and you can get native content. You can search in native content with queries that emulate what you can do in the database or in an XML database. This means that you can build the same type of application with the freedom to access data in an ordinalistic way, and have a mechanism to prioritize and do exactly what Zia talked about with Socrates.

Technically, if you use an open-ended query like, "List the innovative people in my company," or something, you could get back kind of a menu of people that have been referred to in the documents or discussions where there is talk of innovation, and then get the facts related to these people. So it’s not coming back with an answer, but it is coming back with an analysis -- and giving you the opportunity to refine the query.

Gardner: Another thing with developers is that they’re interested in seeing examples. Are there folks out there doing this now who are really on the cutting-edge, who have been able to instantiate rather than theorize? Maybe, Zia, you could address this. Are there folks that are out there using FAST Search & Transfer technologies or other technologies who are actually realizing some of these benefits? And, who are they, and how are they doing it?

Zaman: All over the landscape of the economy, people are using search in interesting ways. I think that the message to the developer is: "Think about the app that you’re working on right now, and think about how the users are accessing either the application or the information."

Take a consumer perspective and you’ll definitely think that search plays an important part in whatever it is that you’re doing. So, get involved and talk to people about search. I think you will find that, even in your own sphere, there are people who are already working on search technologies. In one particular investment bank, 15 or 16 different applications of FAST Search have been put into place across the business to help with everything from getting access to research reports to improving customer service.

There are search policies at law firms or at pharmaceutical companies or retail or e-commerce searches for Christmas shopping. Or just B2B searches, so that one individual can find another professional in the shortest amount of time in the right region. All these applications are about the user consuming information in a way that is most effective for them -- and search technologies are enabling each of these examples. Maybe John or Bjorn, you want to talk about a European example and I can talk about an American example?

Olstad: Sure. Let me start off with our broad business goals, and there are some that have nice benefits for the businesses. We have many customers that don’t have a choice because if you look at media, entertainment, and telcos, there are so many disruptive changes going on. You have print transforming to electronic. You have advertising on the Internet, where search-based advertising is now bypassing traditional banner advertising. You have telcos, where telephone calls may soon be free because there are alternative ways to monetize it. And you already have Skype, Voice over IP, etc.

So, if what a telco provider, for example, is providing today is going to be free, then the name of the game is going to be who is able to create applications and to have service delivery platforms that use that customer base to derive new services. They’re not very well equipped for that. The key enabling-technology that actually helps with this has got to be the user-driven applications. And they see search from a monetization perspective and from the perspective of building the end-user experience. So for lot of customers that we work with from media, entrainment, and telcos -- it’s not that it's "nice to have," but it's "How do we survive?"

Zaman: An example of that in the back office is AutoTrader.com, which is a phenomenal example of how search was used to fundamentally rethink the way in which records were served up. The database administrator at AutoTrader says that they tripled their performance at half the cost, using a highly scalable search platform, which was very easy to deploy and a lot more dynamic than their traditional database architecture. So, even in the back office for applications as diverse as AutoTrader, we are seeing search being used to really disrupt the way in which you think about information delivery.

Gardner: It also seems that the younger generation, as they come into play here, are probably already automatically thinking "search." The people who have grown up with the Web around them start almost any activity with search. So, this is becoming imbued into the society and into the behavior and culture?

At the same time, these folks are also quite used to mobile devices and small handheld devices. Can you explore for me a little bit of how search plays into young people in mobile devices? Perhaps we’re seeing some activities in early-adopter environments, South Korea for example, where search has to be the only way in a small-form factor that vast amounts of information could be managed. Help me understand the impact of use and mobile search?

Olstad: That’s a very interesting question, Dana. As you said, most exploration starts with a search on the Web. You start with a search on a Web search engine, and then you come to the site of this company. Then that company has some options. Should we let the user continue with the search experience or should we force him to go back and understand the text and how we have orchestrated all our content? Then, he is not allowed to do anything meaningful before he has completed that education. That’s slowing down the progress of the users.

Moving to mobile, if we go back to 2001-2002, we were trying to educate mobile operators that they needed to use search. That was a difficult discussion, a difficult sell. In Europe and Asia, at least, it was very popular to download ringtones and wallpapers and things like that. The operators felt that it was sufficient to have a chart with the most popular items and have them available on the static page, allowing people to choose from that.

When they moved to search, what they experienced was that a good way to measure the UI or the usability for the user is how many operations it took to actually perform the action. One of our customers is Vodafone. They measured how many operations young people had to do, from coming into the portal until they could complete a transaction, which in this case was to download a ringtone or music, etc. That used to be 3.5-operations, or something like that, when it was based on the charts and navigation mechanisms. When they substituted search, it went down to 1.5, which means that almost always you go directly to something that you can download. And, once in a while, you have to do one iteration before you get it. That drove revenue and traffic tremendously.

Zaman: You’re right. Generation Y and the younger generation are very used to unfettered access to their data, their content. What does that mean in the Enterprise? This is something that I hear increasingly on Wall Street and in the City of London. I know how to deal with the managing director, who is in his 40s. It’s very easy. For their relationship with their data, they ask someone junior.

But for that younger investment banker who’s just grown up with unfettered access to their personal data, they keep asking why they can’t get access to enterprise data with the same facility. The information technology people who are serving this younger generation have to change. They don’t quite understand that search and other technologies, including Web 2.0 technologies, are forcing them to rethink the way in which they have access to their information, because that’s the way they can be creative and allow for them to get the data they need in order to make the right decisions. I think it’s having a cultural effect, even in more staid industries.

Olstad: Then, if you go to Japan or to Asia, you have also this competition between the telco providers, where the name of the game now is how do you add video services, and with extremely high bandwidth on the phones, how to do that? You see multimedia screening, where you are serving video quality down to the cell phones. So there is a tremendous interest in how to capitalize on new user experience to combine rich media assets, and then use this to build the community stuff you have seen in YouTube in the U.S. This is happening on the mobile side at a tremendous speed in Asia.

Gardner: Very interesting. So we’re really still in the early stages of search. It sounds like there is going to be a lot to discuss at your forthcoming FASTforward ’07 event. Perhaps as a little tease for our audience, is there anything you can tell us about this event in terms of some additional technologies, products or approaches? I will throw this out to anyone.

Olstad: If I could start from the technology side. There will be a lot of cool introductions and some new technology. We will look at how to interface search with repositories, from structured data to rich media and to traditional text. We think there is a game-changing opportunity in doing that and also to enable the Semantic Web in a more powerful way, as we talked about.

We also really believe that, as we have touched on here, that search is not just a feature, but it will become the framework for building search-driven, user-driven applications, to actually make that a reality and make it possible to build those applications. We are introducing new components to do that.

We see that search is driving the connective role in Web 2.0 in the evolution towards Web 3.0 by being a service-delivery platform. So we are introducing a Web 2.0 platform that uses search as the main orchestration -- both for monetization purposes and for understanding both content and the user. We'll be tracking the user and understanding his preferences, building personalized experiences, and creating this platform where you can take the atomic components of content and services and effectively roll that out as user-centric services. We’re building a service delivery platform around search for doing that and you will see applications that already have been built on top of that platform.

Zaman: I think it’s going to be a very hands-on conference in San Diego. There will be customers. There will be developers. There will be case studies, demos, and a lot of learning going on, interacting and sharing of ideas. Sure, there will be the predictions and the analysts and the advice that goes along with it. But really I think that the value of this search conference will be in the exchange of innovative ideas that takes place, not just from FAST to the rest of the community -- but also from the community to other members within the community. I think it's going to be great.

Gardner: Thanks. John Markus, just to return back to the business value, we’ve been discussing this at a fairly technical level -- developers and business intelligence folks -- but for CEOs, for the leadership at many enterprises, what are some of the salient messages that you think they need to understand in order to better appreciate search and why it makes a good deal of sense to invest in it?

Lervik: In general for most information-intensive companies -- media companies, financial services companies, telecom companies and pharmaceutical companies -- search is becoming both a key part of the infrastructure to enable these companies to make decisions, but also -- more importantly -- to drive and increase business online.

Because we see more and more of these companies relying on effectively connecting customers or end-users to their products, to their contents, and to their services. That’s where search comes in as a mission-critical application, or at least to enable these mission critical applications. At FASTforward, there will be a great number of speakers who will talk about how these have effectively implemented a solution that drive and increase business.

Gardner: I’m afraid we’re about out of time. We’ve been discussing search, the role of search, Enterprise 2.0, Web 3.0. Clearly the role and impact of search is increasing rapidly and will increase further over the coming years.

I’d like to thank our guests and sponsor, FAST Search & Transfer. We’ve been talking with John Markus Lervik, the CEO. Also, Bjorn Olstad, the CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for joining us for this BriefingsDirect podcast.

Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.

Transcript of Dana Gardner’s BriefingsDirect podcast on the future of enterprise search. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.