Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Learn more. Sponsor: Kapow Technologies.
See popular event speaker Howard Dresner's latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.
Today, we present a sponsored podcast discussion on the future of business intelligence (BI) --
on bringing more information from more sources into an analytic process, and thereby getting more actionable intelligence out.The explosion of information from across the Web, from mobile devices, inside of social networks, and from the extended business processes that organizations are now employing all provide an opportunity, but they also provide a challenge.
This information can play a critical role in allowing organizations to gather and refine analytics into new market strategies, better buying decisions, and to be the first into new business development opportunities. The challenge is in getting at these Web data services and bringing them into play with existing BI tools and traditional data sets.
This is the first in a series of podcasts, looking at the future of BI and how Web data services can be brought to bear on better business outcomes.
So, what are Web data services and how can they be acquired? Furthermore, what is the future of BI when these extended data sources are made into strong components of the forecasts and analytics that enterprises need to survive the recession and also to best exploit the growth that follows?
Here to help us explain the benefits of Web data services and BI is Howard Dresner, president and founder of Dresner Advisory Services. Welcome to the show, Howard.
Howard Dresner: Thanks, Dana. It's great to be here today.
Gardner: We're also joined by Ron Yu, vice president of marketing at Kapow Technologies. Thanks for joining, Ron.
Ron Yu: Hi, Dana. Great to be with you today.
Gardner: Howard, let me start with you. We've certainly heard a lot about BI over the past several years. There's a very strong trend and lots of investments are being made. How does this, in fact, help companies during the downturn that we are unfortunately still in and then prepare for an upside?
Empowering end users
Dresner: BI is really about empowering end users, as well as their respective organizations,
with insight, the ability to develop perspective. In a downturn, what better time is there to have some understanding of some of the forces that are driving the business?Of course, it's always useful to have the benefit of insight and perspective, even in good times. But, it tends to go from being more outward-focused during good times, focused on markets and acquiring customers and so forth, to being more introspective or internally focused during the bad times, understanding efficiencies and how one can be more productive.
So, BI always has merit and in a downturn it's even more relevant, because we are really less tolerant of being able to make mistakes. We have to execute with even greater precision, and that's really what BI helps us do.
Gardner: Well, if we're looking either internally at our situation or externally at our opportunities, the more information we have at our disposal the stronger our analytical return.
It is a moving target, because the world continues to evolve. There are lots of information sources.
Dresner: Certainly, one would hope so. If you're trying to develop perspective, bringing as much relevant data or information to bear is a valuable thing to do. A lot of organizations focus just on lots of information. I think that you need to focus on the right information to help the organization and individuals carry out the mission of that organization.
Gardner: And that crucial definition of "right information" has changed or is a moving target. How do you keep track of what's the right stuff?
Dresner: It is a moving target, because the world continues to evolve. There are lots of information sources. When I first started covering this beat 20 years ago, the available information was largely just internal stores, corporate stores, or databases of information. Now, a lot of the information that ought to be used, and in many cases, is being used, is not just internal information, but is external as well.
There are syndicated sources, but also the entire World Wide Web, where we can learn about our customers and our competitors, as well as a whole host of sources that ought to considered, if we want to be effective in pursuing new markets or even serving our existing customers.
Gardner: Ron Yu, we've certainly seen an increase in business processes that are now developed from components beyond just a packaged application set. We've seen a mixture of Web, mobile, and other end points being brought to bear on how people interact with their businesses and these processes.
Give me a sense on the extended scope of BI and how do we get at what is now part and parcel with the extended enterprises.
The right data
Yu: I fully agree with Howard. It's all about the right data and, given the current global and
market conditions, enterprises have cut really deep -- from the line of business, but also into the IT organizations. However, they're still challenged with ways to drive more efficiencies, while also trying to innovate.The challenges that are being presented are monumental where traditional BI methods and tools are really providing powerful analytical capabilities. At the same time, they're increasingly constrained by limited access to not only relevant data, but how to get timely access to data.
What we see are pockets of departmental use cases, where marketing departments and product managers are starting to look outside in public data sources to bring in valuable information, so they can find out how the products and services are doing in the market.
Gardner: Howard, we began this discussion with a lofty goal of defining the future of BI. I wonder if you think that the innovation to come from BI activities is a function of the analytics engine or the tools, or is it a function of getting at more, but relevant, information and bringing that to bear.
Dresner: It's an interesting question. One of the things that I focus upon in my second book, which is about to be published next month, is performance-directed culture and the underpinning or the substrate of a performance-directed culture. I won't go into great detail right now, but it has to do with common trust in the information and the availability and currency of the information, as a way to help the organization align with the mission.
The future of BI is not just about the tools and technology. It's great to have tools and technology. I certainly am a fan of technology, being somewhat of a gadget fiend, but that's not going to solve your organization's problems and it's not going to help them align with the mission.
What is going to help them align with the mission is making sure that they have timely, relevant, and complete information, as well as the proper culture to help them support the mission of the enterprise.
Having all the gadgetry is great. Certainly, making the tools more intuitive is a useful and worthwhile thing to do, but it's only as good as the underlying content and insight to support those end users. The future is about focusing on the information and those insights that can empower the individuals, their respective departments, and the enterprise to stay aligned with the mission of that organization.
Other trends afoot
Gardner: The trend and interest in BI is not isolated. There are other complementary, or at least coincidental, mega-trends afoot. One of them, from my perspective, is this whole notion of community, rather than just company, individual, or monolithic thinking. We are expanding into ecosystems.
Cloud computing is becoming a popular notion nowadays. People are thinking about how to cross org
anizational boundaries, how to access resources, perhaps faster better cheaper, from across organizational boundaries.This also brings in this opportunity to start melding, mashing up, and comparing and contrasting data sets across these organizational boundaries. Is there a mega-trend that, from your perspective, Howard, we need to start thinking about BI as a data set-joined function?
Dresner: I fall back on Tom Malone's work, The Future of Work, his book from 2004, where he talks about organizations. Because of the reduced cost of communications, organizations will start to move, and are moving, towards looser bonds, democratized structures, and even market-based structures -- and he cites a number of examples in his book.
The way that you hold together an organization, this loosely bound organization, is through the notion of BI and performance management, which means we certainly have to compare, I wouldn't say data per se, but certainly various measures. We have to share data. We have to combine data and exchange data to get the job done -- whatever that job is. As needs be, we can break those bonds and form new bonds to get the job done.
For every application that IT or line of business develops, it just creates another data silo and another information silo. You have another place that information is disconnected from others.
This doesn’t mean that the future of business is a bunch of small micro-organizations coming together. It really applies to any organization that wants to be agile and entrepreneurial in nature. The underlying foundation of that has to be data and BI in order to function.
Gardner: So, it's about how these organizations relate to one another. Ron, from your perspective, what are some of the essential problems that need to be solved on allowing companies to better understand themselves and then to have this permeability at a process level, a content data, and BI level with other players?
Yu: The term I'd like to use is really about inclusive BI. Inclusive BI essentially includes new and external data sources for departmental applications, but that's only the beginning. Inclusive BI is a completely new mindset. For every application that IT or line of business develops, it just creates another data silo and another information silo. You have another place that information is disconnected from others.
Critical decision-making requires, as Howard was saying earlier, that all business information is easily leveraged whenever it's needed. But today, each application is separate and not joined. This makes the line of business and decision- making very difficult, and it's not in real time.
An easier way
As this dynamic business environment continues to grow, it’s completely infeasible for IT to update their existing data warehouses or to build a new data mart. That can't be the solution. There has to be an easier way to access and extract data exactly where it resides, without having to move data back and forth from data bases, data marts, and data warehouses, which effectively becomes snapshot.
When line of business is working with these data snapshots, by definition it's out of date. Catalytic CIOs and forward looking information architects understand this dilemma and, given that most enterprises are already Web-enabled, they are turning to Web data services to build bridges across all these data silos.
Gardner: Another trend we mentioned, the permeability of the organization, is this involvement -- people being participants in the social networks, having a great deal of publishing going on, putting content out there that can be very valuable to a company. End users seem to want to tell companies what they want, if the companies are willing to listen. We have this opportunity now to create dialogue and conversation, rather than simply looking at the sales receipts.
Tell me how this whole social phenomena of the community and the sharing fits into Web data services?
Web data services provides immediate access to the delivery of this critical data into the business user's BI environment, so that the right timely decisions can be made
Yu: There is effectively a new class of BI applications as we have been discussing, that depends on a completely different set of data sources. Web data services is about this agile access and delivery of the right data at the right time.
With different business pressures that are surfacing everyday, this leads to a continuous need for more and more data sources. But, as Howard was talking about earlier, how do you handle all of that?
Web data services provides immediate access to the delivery of this critical data into the business user's BI environment, so that the right timely decisions can be made. It effectively takes these dashboards, reporting, and analytics to the next level for critical decision-making. So when we look deeper into this and how is this actually playing out, it's all about early and precise predictions.
Let's talk about a few examples. Government agencies are using Web data services to combat terrorism. So, you can be certain that they have all the state-of-the-art analysis tools, spatial mapping, etc. Web data services effectively turbo-charges these analyst tools and is giving them the highest precision in their threat analysis.
These intelligence agencies have access to open-source intelligence, social networks, blogs, forums, even Twitter feeds, and can see exactly what's happening in real time. They can do this predictive analysis and are much better positioned than ever to avert horrible acts of terrorism like 9/11.
Gardner: Howard, do you think, to Ron’s point, that we need to sidestep IT and the traditional purveyors of BI? Is this something that can be done by the end users themselves?
Competency centers
Dresner: It's a very interesting question, and a provocative one too, I might add. But, sidestep IT? Not all IT organizations are inflexible. Some of them certainly are. One of the things that I have advocated for years is the notion of competency centers, certainly in larger organizations. The idea of a competency center is to get the skills in a place, where they can do the most good and where they can really focus on being expedient.
Delivering something to the end user a year after they ask for it really isn't terribly useful. You need to be as agile as possible to respond to ever-changing business needs. There are a very few businesses out there that are static, where things aren’t moving very quickly. In most organizations and most markets, things move pretty darn quickly, and you have to be able to respond to them.
If you don't respond to the users quickly, they find a way to solve their problems themselves, and that really has become an issue in many organizations. I’d like to say that it's a minority, but it's not. It's a majority of them, where IT is going down a slightly a different path, sometimes a dramatically different path, than the end users.
Surprisingly, there are some IT organizations that are pretty well aligned and they are responsive. So, it's not a situation where the end users need to completely discount IT, but some IT organizations have become pretty inflexible. They are focused myopically on some internal sources and are not being responsive to the end user.
To the extent that they can find new tools like Web data services to help them be more effective and more efficient, they are totally open to giving line of business self-service capabilities.
You need to be careful not to suffer from what I call BI myopia, where we are focused just on our internal corporate systems or our financial systems. We need to be responsive. We need to be inclusive of information that can respond to the user's needs as quickly as possible, and sometimes the competency center is the right approach.
I have instances where the users do wrest control and, in my latest book, I have four very interesting case studies. Some are focused on organizations, where it was more IT driven. In other instances, it was business operations or finance driven.
Yu: There is, in most cases, a middle ground, and IT certainly isn't looking for more things to do. To the extent that they can find new tools like Web data services to help them be more effective and more efficient, they are totally open to giving line of business self-service capabilities.
Gardner: Ron, whether it's the IT department and a fully sanctioned tool and approach that they are supporting or whether it's self-service, we can't just open up the fire hose and have all of this content dump into our business and analytics activities.
What do you bring to the table in terms of not only getting access to Web data services, but also cleansing them, vetting them, putting them in the right format, and making sure it's secure and their privacy issues are being adhered to? What's the value add to go beyond access into a qualitative set of highly valued assets?
Start with the use case
Yu: Sometimes, the problem we face, when we talk about BI, is that we immediately start talking about the software, the servers, and the things that we needed to build. BI really starts with the business use case.
What is it that the line of business is trying to do and can we develop the right facilities in order to work on that project? Yet, if those projects don't become so overbearing that you just create IT project gridlock, then I think we have something new to say.
For example, in leading financial services companies, what they're looking for is on this theme of early and precise predictions. How can you leverage information sources that are publicly available, like weather information, to be able to assess the precipitation and rainfall and even the water levels of lakes that directly contribute to hydroelectricity?
If we can gather all that information, and develop a BI system that can aggregate all this information and provide the analytical capabilities, then you can make very important decisions about trading on energy commodities and investment decisions.
Web data services effectively automates this access and extraction of the data and metadata and things of that nature, so that IT doesn't have to go and build a brand new separate BI system every time line of business comes up with a new business scenario.
Web data services effectively automates this access and extraction of the data and metadata and things of that nature, so that IT doesn't have to go and build a brand new separate BI system every time line of business comes up with a new business scenario.
Gardner: Again, to this notion of the fire hose, you are not just opening up the spigot. You're actually adding some value and helping people manage and control this flow, right?
Yu: Exactly. It's about the preciseness of the data source that the line of business already understands. They want to access it, because they're working with that data, they're viewing that data, and they're seeing it through their own applications every single day.
But, that data is buried deep within the application in the database and the only way that they can do this through the traditional ways is through opening up a new IT ticket and asking their database, their data warehouse, or their application to be updated. That just is very time-consuming and very expensive for everyone involved.
Gardner: To your point earlier, you end up getting a significant latency, and it's probably precisely the kind of Web services data that you want to get closer to real time in order to analyze what's going on.
Voice of the customer
Yu: That's exactly the case. The voice of the customer provides huge financial and exposure protection for product vendors. For example, if a tire manufacturer had the ability to monitor consumer sentiment, they would be able to investigate and even issue early recalls well before tragic events happen, which would create even larger financial loses and huge damage on the brand.
Gardner: Ron, help me understand a little bit better what Kapow Technologies brings in terms of this Web data services support. How does that also relate to a larger BI solution that incorporates Web data services.
Yu: We're going a little bit into the technical side of things now. Effectively, Kapow Web Data Server, which is our product, is a platform that provides IT and some line of business users, who actually have more of a technical aptitude, the ability to visually interact with the data sources through the Web, HTML and the Ajax front-end of an application or Web page or a Web portal.
Effectively, you visually program and give instructions through point-and-click, which gives you precise navigation through all of the forms and as deep as you want to go into that Website or Web application.
As you point and click, you can give instructions about extracting the data and even enriching the data. For example, going to LinkedIn, you see that there are certain images that are assigned to specific data. With our product, you can interpret those graphical images and give them a value.
Our product effectively gives you that precise surgical navigation and extraction of any data from exactly the application that you're working with . . .
Our product effectively gives you that precise surgical navigation and extraction of any data from exactly the application that you're working with to create an RSS feed, a REST service, or, in a case of traditional BI, even loading it directly into a SQL database with a one-button deployment.
There is no programming involved. So, you can imagine how incredibly productive this is for IT. You don't have to waste time writing SQL scripts, application programming interfaces (APIs), and things of that nature. It enables that easy access and moves on to the higher value of what IT can deliver, which is on the application and presentation side.
Gardner: Howard, in your work with your clients and your research for your new book, did you encounter any examples that you can recall where folks have taken this to heart and moved beyond the traditional content types that BI has supported? What sort of experience, paybacks, and benefits have they enjoyed?
Not just internal sources
Dresner: The answer is yes. There are a number of good examples. Obviously, I encourage everybody to order a copy of the new book, which is out next month. But, including other sources than just internal sources gives you a better perspective. It creates a much more interesting and rich tapestry of the business and the market in which it lives.
One of the organizations I dealt with is in the hospitality business. Understanding their market, understanding what their competition is doing, what offers that they are providing means that they have to go to those Websites, as well as accessing some social networking sites.
They have to understand what's the customer sentiment is out there and what sort of offers their competition is offering on a Sunday night, for example, in order for them to remain competitive. You have to understand the changing trends, if you want to be a “hip hotel chain.” What does that mean? What's changing socially in those particularly geographies and markets that you play in that you need to be aware of and respond to.
The same thing is true in other industries. Another one of the organizations I worked with is in the healthcare industry. So understanding your patient requirements is important, if you want to be a more patient-oriented organization. What are their changing needs? What are their desires? What are those things that they expect from their service provider? You are not going to get that from your internal database?
Providing access to external content in conjunction with the content from your internal systems gives you a greater perspective. How many times have we heard, "Gee, if I'd only known that, I could have made a better decision or I could have framed the decision-making process more effectively?" That's really where we are in the history of BI right now.
We need to provide a better perspective, more complete and more timely perspective, in order to frame the decision-making processes. Going back to my original point, and really the central point of the book, how do we get everybody in the organization aligned with the mission to make sure that we're all fulfilling our particular role within the organization and using things like BI and the right sorts of data to achieve that purpose?
But, when you look outside the firewall -- and I'm talking about all these public data sources and even partners -- how do you collaborate better with your partners? All of these things are Web enabled.
Yu: I agree, Howard, and I think that's just the tip of the iceberg. If we look at the spirit of what corporate performance management or enterprise performance management is supposed to deliver, BI systems are really dealing with operational data and financial data within the firewall. But, when you look outside the firewall -- and I'm talking about all these public data sources and even partners -- how do you collaborate better with your partners? All of these things are Web enabled.
How do you bring things together from outside the firewall and integrate them with the operational and financial data? That challenge will really be a huge payoff, once IT organizations and CIOs can leverage Web data services for this huge payoff within that enterprise, whether it's the next generation of BI for business-to-employee (B2E) applications, business-to-business (B2B) with their partners, or even business-to-consumer (B2C) applications.
Gardner: Ron, I wonder if you have any examples, folks that have gone out and gathered these Web data services? What sort of uses have they put them to and what paybacks have they encountered?
Partners and B2B
Yu: We've talked a lot about public Web data sources. Let's talk about partners and B2B. One of the Fortune 500 financial services companies was required by regulatory compliance to report on their treasury transactions on 10,000 treasury transactions per day.
They had several analysts fully dedicated to logging in to each of their top 100 banking partners and extracting information, loading it into an Excel spreadsheet, and then normalizing the data and cleansing the data You know that when you use manual efforts, you will never get precise around the data quality, but that was the best facility that they had.
Then, they would take that Excel spreadsheet, load that into a database, and put a BI tool on top of that to provide their transactional dashboard. They spent three years evaluating technologies and trying to build the solution on their own and they failed.
So they came to Kapow Technologies and implemented a proof of concept within three weeks. They were able to get three of their top banking partners to develop a BI dashboard to monitor and manage these transactions and the full deployment in three months. Now, they are looking to expand that to other aspects of their business.
Gardner: I think we've learned a lot here about Web data services. Ron, where do you see it going in the future? How does this move beyond the vision that we already have developed here?
Yu: As Howard has been advocating about getting the right data, once you get the data access right, where the data is accurate, noise-free and timely, then the future of BI will really be about automated decision making.
We got a taste with some of the examples that I talked about with financial services and working with the partners, but also investment decisions and things like that. In the same way that we've seen that in financial decisions around making buy/sell decisions in an automated predictive way, there is this same opportunity that exists across all industries.
Gardner: Howard, do you agree that future BI is increasingly an automated affair?
Dresner: There are certainly places where we ought to be automating BI. Decision automation certainly. But, to my way of thinking, BI is involved in empowering users and making them smarter. There is a tremendous amount of room for improvement there.
As I said, I've been on this beat for 20 years now, and certainly have seen improvements in the tools, across the board, from the bottom of the stack all the way to the top, and we can certainly see increased penetrations in the use.
The next hurdle is applying the technology a little bit more effectively. That's really where we have fallen far short, not understanding why we are implementing the technology -- let's give everybody BI and a data warehouse and hope for the best. Not that there hasn't been any goodness associated with it, but certainly not one that is requisite with the investments that have been made.
Going back to what I said, earlier in the broadcast, the focus upon the performance-directed cultures and using the technology as an enabler to support those cultures is really where I think organizations need to apply their thinking.
Gardner: I'm afraid we'll have to leave it there. We've been discussing how Web data services play a critical role in allowing companies to gather and refine their analytics to engage in better market strategies and better buying decisions and to join and explore business development opportunities. Helping us to deal in a future BI and the role of Web data services, we've been joined by Howard Dresner, president and founder of Dresner Advisory Services. Thanks so much, Howard.
Dresner: My pleasure. Thanks for having me.
Gardner: Also, we have been joined by Ron Yu, vice president of marketing at Kapow Technologies. Thank you, Ron.
Yu: Thanks, Dana. I had a great time.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. Thanks for listening, and come back next time.
Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Learn more. Sponsor: Kapow Technologies.
See popular event speaker Howard Dresner's latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.
Transcript of first in a series of sponsored BriefingsDirect podcasts with Kapow Technologies on Web Data Services and how harnessing the explosion of Web-based information inside and outside the enterprise buttresses the value and power of business intelligence. In Part Two, Kapow co-founder and CTO Stefan Andreasen and Forrester analyst Jim Kobielus discuss how Web data services provide ease of access to data from a variety of sources. Copyright Interarbor Solutions, LLC, 2005-2009. All rights reserved.
