Ocean Observatories Initiative: Cloud and Big Data Come Together to Give Scientists Unprecedented Access to Essential Climate Information

Transcript of a BriefingsDirect podcast on how cloud and big data come together to offer climate researchers a treasure trove of ongoing, real-time information.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: VMware.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you're listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on a fascinating global ocean studies initiative that defines some of the superlatives around big data, cloud, and middleware integration capabilities.

We'll be exploring the Ocean Observatories Initiative (OOI) and its accompanying Cyberinfrastructure Program. This undertaking by the National Science Foundation aims to provide an unprecedented ability to study the Earth's oceans and climate using myriad distributed data centers and literally oceans' worth of data.

The scale and impact of the science's importance is closely followed by the magnitude of the computer science needed to make that data accessible and actionable by scientists. In a sense, the OOI and its infrastructure program are constructing a big data-scale programmable and integratable cloud fabric.

We’ve gathered three leaders to explain the OOI and how the Cyberinfrastructure Program may not only solve this set of data and compute problems, but perhaps establish a path to how future massive data and analysis problems are solved.

Here to share their story on OOI are our guests:

• Matthew Arrott, Project Manager at the OOI Cyberinfrastructure. Matthew's career spans more than 20 years in design leadership and engineering management for software and network systems. He’s held leadership positions at Currenex, DreamWorks SKG, Autodesk, and the National Center for Supercomputing Applications. His most recent work has been with the University of California as e-Science Program Manager while focusing on delivering the OOI Cyberinfrastructure capabilities.
  

• Michael Meisinger, Managing Systems Architect for the Ocean Observatories Initiative Cyberinfrastructure. Since 2007, Michael has been employed by the University of California, San Diego. He leads a team of systems architects on the OOI Project. Prior to UC San Diego, Michael was a lead developer in an Internet startup, developing a platform for automated customer interactions and data analysis. Michael holds a master's degree in computer science from the Technical University of Munich and will soon complete a PhD in formal services-oriented computing and distributed systems architecture.
  

• Alexis Richardson, Senior Director for the VMware Cloud Application Platform. He is a serial entrepreneur and a technologist. Previously, he was a founder of RabbitMQ and the CEO of Rabbit Technologies Limited, which was acquired by VMware in April of 2010. Alexis plays a role in both the cloud and messaging communities, as well as a leading role in Advanced Message Queuing Protocol (AMQP). He is a co-founder of the CloudCamp conferences, and a co-chair of the Open Cloud Computing Interface at the Open Grid Forum. [Disclosure: VMware is a sponsor of BriefingsDirect podcasts.]
  

Michael Meisinger, could you sum up the OOI for our audience? Let us know a little bit about how it came about.

Ocean Observatories Initiative

Michael Meisinger: Thanks, Dana. The Ocean Observatories Initiative is a large project. It's a US National Science Foundation project that is intended to build a platform for ocean sciences end users and communities interested in this form of data for an operational life span of 30 years.

It comprises a construction period of five years and will integrate a large number of resources and assets. These range from typical oceanographic assets, like instruments that are mounted on buoys deployed in the ocean, to networking infrastructure on the cyberinfrastructure side. It also includes a large number of sophisticated software systems.

I'm the managing architect for the cyberinfrastructure, so I'm primarily concerned with the interfaces through the oceanographic infrastructure, including beta interfaces, networking interfaces, and then primarily, the design of the system that is the network hardware and software system that comprises the cyberinfrastructure.

As I said, OOI’s goals include serving the science and education communities with their needs for receiving, analyzing, and manipulating ocean sciences and environmental data. This will have a large impact on the science community and the overall public, as a whole, because ocean sciences data is very important in understanding the changes and processes of the earth, the environment, and the climate as a whole.

Ocean sciences, as a discipline, hasn't yet received as much infrastructure and central attention as other communities. So the OOI initiative is a very important to bring this to the community. It has an almost $400 million construction budget, and an annual operations budget of $70 million for a planned lifetime of 25 to 30 years.

Gardner: Matthew Arrott, what is the big hurdle here in terms of a compute issue that you've faced. Obviously, it's a tremendously important project with a tremendous amount of data, but from a purely compute requirements perspective, what makes this so challenging?

Matthew Arrott: It has a number of key aspects that we had to address. It's best to start at the top of the functional requirements, which is to provide interactive mission planning and control of the overall instrumentation on the 65 independent platforms that are deployed throughout the ocean.

The issue there is how to provide a standard command-and-control infrastructure over a core set of 800 instruments, about 50 different classes of instrumentation, as well as be able to deploy -- over the 30-year lifecycle -- new instrumentation brought to us by different scientific communities for experimentation.

The next is that the mission planning and control is meant to be interactive and respond to emergent changes. So we needed an event-response infrastructure that allowed us to operate on scales from microseconds to hours in being able to detect and respond to the changes. We needed an ability to move computing throughout the network to deal with the different latency requirements that were needed for the event-response analysis.

Finally, we have computational nodes all the way down in the ocean, as well as on the shore stations, that are accepting or acquiring the data coming off the network. And we're distributing that data in real time to any one who wants to listen to the signals to develop their own sense-and-response mechanisms, whether they're in the cloud, in their local institutions, or on their laptop.

Domain of control

The fundamental challenge was the ability to create a domain of control over instrumentation that is deployed by operators and for processing and data distribution to be agile in its deployment anywhere in the global network.

Gardner: Alexis Richardson, it sounds like a very interesting problem to solve. Why is this a good time to try to solve this? Of course, big data, cloud, doing tremendous amounts of services orientation across middleware and a variety of different formats and transports, is all very prominent in the enterprise now. Given that, what makes this, such an interesting pursuit for you in thinking about this from a software distribution and data distribution perspective?

Alexis Richardson: It really comes down to the scale of the system and the ability of technologies to meet the scale need today. If we had been talking about this 12 years ago, in the year 2000, we would have been talking about companies like Google and Yahoo, which we would not have considered to be of moderate scale.

Since then, many companies have appeared. For example, Facebook, which has many hundreds of millions of users connecting throughout the world, shares vast amounts of data all the time.

It's that scale that's changed the architecture and deployment patterns that people have been using for these applications. In addition to that, many of these companies have brought out essentially a platform capability, whereby others, such as Zynga, in the case of Facebook, can create applications that run inside these networks -- social networks in the case of Facebook.

We can see the OOI project is essentially bringing the science needed to collaborate between vast numbers of sensors and signals and a comparatively smaller number of scientists, research institutions, and scientific applications to do analytics in a similar way as to how Facebook combines what people say, what pictures they post, what music they listen to with everybody’s friends, and then allow an application to be attached to that.

So it’s a huge technology challenge that would have been simply infeasible 12 years ago in the year 2000, when we thought things were big, but they were not. Now, when we talk about big data being masses of terabytes and petabytes that need to be analyzed all the time, then we’re starting to glimpse what's possible with the technology that’s been created in the last 10 years.

It’s a huge technology challenge that would have been simply infeasible 12 years ago.

Arrott: I’d like to actually go one step further than that. The challenge goes beyond just the big data challenge. It also now introduces, as Alexis talked about, the human putting in what they say in their pictures. It introduced that the concept of the instrument as an equal partner with the human in the participation in the network.

So you now have to think about what it means to have a device that’s acting like a human in the network, and the notion that the instrument is, in fact, owned by someone and must be governed by someone, which is not the case with the human, because the human governs themselves. So it represents the notion of an autonomous agent in the network, as well as that agent having a notion of control that has to stay on the network.

Gardner: I’d like to try to explain for our audience a bit more about what is going on here. We understand that we have a tremendous diversity of sensors gathering in real-time a tremendous scale of data. But we’re also talking about automating the gathering and distribution of that data to a variety of applications.

Numerical framework

We’re talking about having applications within this fabric, so that the output is not necessarily data, but is a computational numerical framework that’s then distributed. So there's computation being done at the data level, and then it has to be regulated. Certain data goes to certain people for certain reasons, under certain circumstances.

So there's a lot of data, a lot of logic, and a lot of scale. Can one of you help step me through it all a bit more to understand the architecture of what’s being conducted here?

Meisinger: The challenge, as you mentioned, is very heterogeneous. We deal with various classes of sensors, classes of data, classes of users, or even communities of users, and with classes of technological problems and solution spaces.

So the architecture is based on a tiered model or in a layered model of most invariant things at the bottom, things that shouldn’t change over the lifetime of 30 years to serve the highest level of attention.

Then, we go into our more specialized layered architecture where we try to find optimal solutions using today’s technologies for high-speed messaging, big data, and so on. Then, we go into specialized solutions for specific groups of users and specific sensors that are there as last-mile technologies to integrate them into the system.

Then as you go towards the core, you approach the invariants of the system.

So you basically see an onion layer model of the architecture, externalization outside. Then as you go toward the core, you approach the invariants of the system.

What are the invariants? We recognized that a system of this scale and a system of this heterogeneity cannot be reinvented every five years as part of the typical maintenance. So as a strongly scalable and extensible system, it's distributed in its nature, and as part of the distribution, the most invariant parts are the protocols and the interactions between the distributed entities on the system.

We found that it's essential to define a common language, a common format, for the various applications and participants of the network, including sensor and sensor agents, but also higher-level software services to communicate in a common format.

This architecture is based on defining a common interaction format. It’s based on defining a common data format. You mentioned the complex numerical model. A lot of things in this architecture are defined so that you have an easier model of reaching many heterogeneous communities by ingesting and getting specific solutions into the system, representing them consistently and then presenting them again in the specific format for the audience.

Our architecture is strongly communication-oriented, service-oriented, message-oriented, and federated.

As Matthew mentioned, it’s an important means to have the individual resources, agents, provide their own policies, not having a central bottleneck in the system or central governing entity in the system that defines policies.

Strongly federated

So it’s a strongly federated system. It’s a system that’s strongly technology-independent. The communication product can be implemented by various technologies, and we’re choosing a couple of programming languages and technologies for our initial reference implementation, but it’s strongly extensible for future communities to use.

Gardner: One of the aspects of this that was particularly interesting to me is that this is very much a two-way street. The scientists who are gathering their analysis can very rapidly go back to these sensors, go back to this compute fabric, this fusion of data, and ask it to do other things in real-time; or to bring in data from outside sources to compare and contrast, to find the commonalities and to find what it is that they’re looking for in terms of trends.

Could one of you help me understand why this is a two-way street, and how that's possible given the scale and complexity?

Arrott: The way to think about it, first and foremost, is to think of it as its four core layers. There is the underlying network resource management layer. We talk about agents. They supply that capability to any process in the system, and we create devices that process.

The next layer up is the data layer, and the data layer consists of two core parts. One is the distribution system that allows for data to be moved in real-time from the source to the interested parties. It’s fundamentally a publish-subscribe (pub-sub) model. We're currently using point-to-point as well as topic-based subscriptions, but we're quickly moving toward content-based routing, which is more based on the the selector that is provided by the consumer to direct traffic toward them.

The other part of the data layer is the traditional harvesting or retrieval of data from historical repositories.

The other part of the data layer is the traditional harvesting or retrieval of data from historical repositories.

The next layer up is the analytic layer. It looks a lot like the device layer, but is focused on the management of processes that are using the big data and responding to new arrival of data in the network or change in data in the network. Finally, there is the fourth layer, which is the mission planning and control layer, which we’ll talk about later.

Gardner: Alexis, when you saw the problem that needed to be solved here, you had a lot of experience with advanced message queuing protocol (AMQP), which I'd like you to explain to us, and you also understand the requirements of a messaging system that can accomplish what Matthew just described.

So tell me about AMQP, why this problem seems to be the right fit for that particular technology, RabbitMQ, and a messaging infrastructure in general.

Richardson: What Matthew and Michael have described can be broken down into three fundamental pieces of technology.

Lot of chatter

Number one, you have a lot of chatter coming from these devices -- machines, people, and other kinds of processes -- and that needs to get to the right place. It's being chattered or twittered away and possibly at high rates and high frequencies. It needs to get to just the set of receivers following that stream, very similar to how we understand distribution to our computers. So you need what’s called pub-sub, which is a fundamental technology.

In addition, that data needs to be stored somewhere. People need to go back and audit it, to pull it out of the archive and replay it, or view it again. So you need some form of storage and reliability built into your messaging network.

Finally, you need the ability to attach applications that will be written by autonomous groups, scientists, and other people who don’t necessarily talk to one another, may choose these different programming languages, and may be deploying our applications, as Matthew said, on their own servers, on multiple different clouds that they are choosing through what you would like to be a common platform. So you need this to be done in a standard way.

AMQP is unique in bringing together pub-sub with reliable messaging with standards, so that this can happen. That is precisely why AMQP is important. It's like HTTP and email SMTP, but it’s aimed at messaging the publish-subscribe reliable message delivery in a standard way. And RabbitMQ is one of the first implementations, and that’s how we ended up working with the OOI team -- because RabbitMQ provides these and does it well.

Gardner: Now we’ve talked a lot about computer science and some of the thorny issues that have been created as a result of this project, but, I’d also like to go back to the project itself, and give our listeners a sense of what this can accomplish. I’ve heard it described as "the Hubble Telescope of oceans.

It's the notion that we're providing capabilities that do not currently exist for oceanographers.

"

Let’s go back to the oceanography and the climate science. What can we accomplish with this, when this data is delivered in the fashion we’ve been discussing, where the programmability is there, where certain scientists can interact with these sensors and data, ask it to do things, and then get that information back in a format that’s not raw, but is in fact actionable intelligence?

Matthew, what could possibly happen in terms of the change in our understanding of the oceans from this type of undertaking?

Arrott: The way to think about this is not so much from the fact that we know exactly what will happen. It's the notion that we're providing capabilities that do not currently exist for oceanographers. It can be summed up as continual presence in the oceans at multiple scales through multiple perspectives, also known as the different classes of instrumentation that are observed in the ocean.

Another class of instrumentation is deployed specifically for refocusing. The scope of the OOI is such that it is considered to be observing the ocean at multiple scales -- coastal, regional, and global. It is an expandable model such that other observatories, as well as additions to the OOI network, can be considered and deployed in subsequent years.

This allows us now, as Alexis talked about, to attach many different classes of applications to the network. One of the largest classes of applications that we’ll attach to the network are the modeling, in particular the nowcast and forecast modeling.

Happening at scale

Through those observations about the ocean now, about what the ocean will be, and to be able to ground-truth those models going forward, based on data arriving in the same time as the forecasts, provides for a broad range of modeling that has been done for a fair amount of time, but it now allows it to happen at scale.

Once you have that ability to actually model the oceans and predict where it’s going, you can use that to refocus the instrumentation on emergent events. It's this ability to have long-term presence in the ocean, and the ability to refocus the instrumentation on emergent events, that really represents the revolutionary change in the formation of this infrastructure.

Meisinger: Let me add, I'm very fascinated by The Hubble Space Telescope as something that produces fantastic imagery and fantastic insights into the universe. For me as a computer scientist, it’s often very difficult to imagine what users of the system would do with the system.

I’d like to see the OOI as a platform that’s developed by the experts in their fields to deploy the platforms, the buoys, the cables, the sensors into the ocean that then enables the users of the system over 25 years to produce unprecedented knowledge and results out of that system.

The primary mission of our project is to provide this platform, the space telescope in the ocean. And it’s not a single telescope. In our case, it's a set of 65 buoys, locations in the ocean, and even a cable that runs a 1,000 miles at the seafloor of the Pacific Northwest that provides 10 gigabit ethernet connectivity to the instrument, and high power.

The primary mission of our project is to provide this platform, the space telescope in the ocean.

It’s a model where scientists have to compete. They have to compete for a slot on that infrastructure. They'll have to apply for grants and they'll have to reserve the spot, so that they can accomplish the best scientific discoveries out of that system.

It’s kind of the analogy of the space telescope that will bring ocean scientists to the next level. This is our large platform, our large infrastructure that have the best scientists develop and research to best results. That’s the fascination that I see as part of this project.

Gardner: For the average listener to understand, is this comparable to tracking weather and the climate on the surface? Many of us, of course, get our weather forecasts and they seem to be getting better. We have satellites, radar, measurements, and historical data to compare, and we have models of what weather should do. Is this in some ways taking the weather of the oceans? Is it comparable?

Arrott: Quite comparable. There's a movement to instrument the Earth, so that we can understand from observation, as opposed to speculation, what the Earth is actually doing, and from a notion of climate and climate change, what we might be doing to the Earth as participants on it.

The weather community, because of the demand for commercial need for that weather data, has been well in advance of the other environmental sciences in this regard. What you'll find is that OOI is just one of several ongoing initiatives to do exactly what weather has done.

The work that I did at NCSA, was with the atmospheric sciences community was very clear at the time. What could they do if they had the kind of resources that we now have here in the 21st century? We've worked with them and modeled much of our system based on the systems that they built, both in the research area, and in the operational area in programs such as Nova.

Science more mature

Gardner: So, in a sense, we're following the path of what we’ve done with the weather, and understanding the climate on land. We’re now moving into the oceans, but at a time when the computer science is more mature, and in fact, perhaps even much more productive.

Back to you Alexis Richardson. This is being sponsored by the US National Science Foundation, so being cost efficient is very important, of course. How is it that cloud computing is being brought to bear, making this productive, and perhaps even ahead of where the whole weather and predicting weather has been, because we can now avail ourselves of some of the newer tools and models around data and cloud infrastructure?

Richardson: Happily, that’s an easy one. Imagine if a person or scientist wanted to process very quickly a large amount of data that’s come from the oceans to build a picture of the climate, the ocean, or anything to do with the coastal proprieties of the North American coast. They might need to borrow 10,000 or 20,000 machines for an hour, and they might need to have a vast amount of data readily accessible to those machines.

In the cloud, you can do that, and with big data technologies today, that is a realistic proposition. It was not five to 10 years ago. It’s that simple.

Obviously, you need to have the technologies, like this messaging that we talked about, to get that data to those machines so they can be processed. But, the cloud is really there to bring it altogether and to make it seem to the application owner like something that’s just ready for them to acquire it, and when they don’t need it anymore, they can put it back and someone else can use it.

Its common execution infrastructure subsystem is built in order to enable this access to computation and big data very quickly.

Gardner: Back to you Michael. How do you view the advent of cloud computing as a benefit to this sort of initiative? We have a piece of it from Alexis, but I’d like to hear your perspective on why cloud models are enabling this perhaps at an unprecedented scale, but also at a most efficient cost?

Meisinger: Absolutely. It does enable computing at unprecedented scale for exactly reasons that Alexis mentioned. A lot of the earth's environment is changing. Assume that you’re interested in tracking the effect of a hurricane somewhere in the ocean and you’re interested in computing a very complex numerical model that provides certain predictions about currents and other variables of the ocean. You want to do that when the hurricane occurs and you want to do it quickly. Part of the strategy is to enable quick computation on demand.

The OOI architecture, in particular, its common execution infrastructure subsystem, is built in order to enable this access to computation and big data very quickly. You want to be able to make use of execution provider’s infrastructure as a service very quickly to run your own models with the infrastructure that the OOI provides.

Then, there are other users that want to do things more regularly, and they might have their own hardware. They might run their own clusters, but in order to be interoperable, and in order to have excess overflow capabilities, it’s very important to have cloud infrastructure as a means of making the system more homogenous.

So the cloud is a way of abstracting compute resources of the various participants of the system, be they commercial or academic cloud computing providers or institutions that provide their own clusters as cloud systems, and they all form a large compute network, a compute fabric, so that they can run the computation in a predictable way, but also then in a very episodic way.

Cloud as enabler

I really see that the cloud paradigm is one of the enablers of doing this very efficiently, and it enables us as a software infrastructure project to develop the systems, the architecture, to actually manage this computation from a system’s point of view in a central way.

Gardner: Alexis, because of AMQP and the VMware cloud application platform, it seems to me that you’ve been able to shop around for cloud resources, using the marketplace, because you’ve allowed for interoperability among and between platforms, applications, tools, and frameworks.

Is it the case that leveraging AMQP has given you the opportunity to go to where the compute resources are available at the lowest cost when that’s in your best interest?

Richardson: The dividend of interoperability for the end user and the end customer in this platform environment is ultimately portability -- portability through being able to choose where your application will run.

Michael described it very well. A hurricane is coming. Do you want to use the machines provided by the cloud provider here for this price? Do you want to use your own servers? Maybe your neighboring data center has servers available to you, provided those are visible and provided there is this fundamental interoperability through cloud platforms of the type that we are investing in. Then, you will be able to have that choice. And that lets you make these decisions in a way that you could not do before.

Providing a strong platform or a strong technological footprint that’s not specific to any technology is a great benefit to the community out there.

Gardner: I’m afraid we’re almost out of time, but I want to try to compare this to what this will allow in other areas. It’s been mentioned by Alexis and others that this has got some common features to Twitter, Facebook, or Zynga.

We think of the social environment because of the scale, complexity, and the use of cloud models. But we’re doing far more advanced computational activities here. This is simply not a display of 140 characters, based on a very rudimentary search, for example. These are at the high performance computing (HPC) level, supercomputer-level types of requests and analysis.

So are we combining the best of a social fabric approach and the architecture behind that to what we’ve been traditionally exposed to in high-performance computing and supercomputing? If so, what does that mean for how we could bring this to other types of uses in the future? I’ll throw this out to any of you. How are we doing the best of the old and the new computing, and what does that mean for the future?

Meisinger: This is the direction in which the future will evolve, and it’s the combination of proven patterns of interaction that are emerging out of how humans interact applied to high-performance computing. Providing a strong platform or a strong technological footprint that’s not specific to any technology is a great benefit to the community out there.

Providing a reference architecture and a reference implementation that can solve these problems, that social network for sensor networks and for device computation will be a pattern that can be leveraged by other interested participants, either by participating in the system directly or indirectly, where it’s just taking that pattern and the technologies that come with it and basically bringing it to the next level in the future. Developing it as one large project in a coherent set really yields a technology stack and architecture that will carry us far into the future.

Arrott: With all the incremental change that we're introducing is taking the concepts of Facebook and of Twitter and the notions of Dropbox, which is the ability to move a file to a shared place so someone else can pick it up later, which was really not possible long ago. I had to do an FTP server, put up an HTTP server to accomplish that.

Sharing processes

What we are now adding to the mix is not sharing just artifacts, but we’re actually sharing processes with one another, and then specifically sharing instrumentation. I can say to you, "Here, have a look through my telescope." You can move it around and focus it.

Basically, we introduced the concept of artifacts or information resources, as well as the concept of a taskable resource, and the thing that we’re adding to that which can be shared are taskable resources.

Gardner: I’m just going to throw out a few blue-sky ideas that it seems this could be applicable to ... things like genetics and the human genome, but on an individual basis; or crime statistics, in order to have better insight into human behavior at a massive scale; or perhaps even healthcare, where you’re diagnosing specific types of symptoms and then correlating them across entire regions or genetic patterns that would be brought to bear on those symptoms.

Am I off-base? Is this science fiction? Or am I perhaps pointing to where this sort of capability might go next?

It’s a platform where you can plug in your own system or subsystem that you can then make available to whoever is connected to that platform.

Richardson: The answer to your question is, "Yes," if you add one little phrase into that: in real-time. If, you’re talking about crime statistics, as events happen on the streets, information is gathered and shared and processed. As people go on jobs, if information is gathered, shared, and processed on how people are doing, then you will be able to have the kind of crime or healthcare benefits that you described. I’m sure we could think of lots of use cases. Transport is another one.

Arrott: At the institution in which the OOI Cyberinfrastructure is housed, California Institute of Telecommunication and Information Technology (Calit2), all of the concerns that you’ve mentioned are, in fact, active development research programs, all of which have yielded significant improvements in the computational environment for that scientific community.

Gardner: Michael, last word to you. Where do you see this potentially going in terms of the capability? Obviously, it's a very important activity, with the oceans. But the methods that you’re defining, the implementations that you’re perfecting, where do you see them being applied in the not-too-distant future?

Meisinger: You’re absolutely right. This pattern is very applicable and it’s not that frequent that a research and construction project of that size has an ability to provide an end-to-end technology solution to this challenge of big data combined with real-time analysis and real-time command and control of the infrastructure.

What I see that’s evolving into is, first of all, you can take the solutions build in this project and apply it to other communities that are in need for such a solution. But then it could go further. Why not combine these communities into a larger system? Why not federate or connect all these communities into a larger infrastructure that all is based on common ideas, common standards, and that still enables open participation?

It’s a platform where you can plug in your own system or subsystem that you can then make available to whoever is connected to that platform, whoever you trust. So it can evolve into a large ecosystem, and that does not have to happen under the umbrella of one organization such as OOI.

Larger ecosystem

It can happen to a larger ecosystem of connected computing based on your own policies, your own technologies, your own standards, but where everyone shares a common piece of the same idea and can take whatever they want and not consume what they’re not interested in.

Gardner: And as I said earlier, at that very interesting intersection of where you can find the most efficient compute resources available and avail yourself of them with that portability, it sounds like a really powerful combination.

We’ve been talking about how the Ocean Observatories Initiative and its accompanying Cyberinfrastructure Program have been not only feeding the means for the ocean to be better understood and climate interaction to be better appreciated, but we’re also seeing how the architecture behind that is leading to the potential for many other big data, cloud fabric, real-time, compute-intensive applications.

Everyone shares a common piece of the same idea and can take whatever they want and not consume what they’re not interested in.

I’d like to thank our guests, Matthew Arrott, Project Manager at the OOI and the initiative for the Cyberinfrastructure. Thank you so much, Matthew.

Arrott: Thank you.

Gardner: We’ve also been joined by Michael Meisinger, Managing Systems Architect for the OOI Cyberinfrastructure. Thank you, Michael.

Meisinger: Thanks, Dana.

Gardner: And Alexis Richardson, the Senior Director for VMware Cloud Application Platform. Thank you, Alexis.

Richardson: Thank you, very much.

Gardner: And this is Dana Gardner, Principal Analyst at Interarbor Solutions. Thanks to you, our audience, for listening, and come back next time.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: VMware.

Transcript of a BriefingsDirect podcast on how cloud and big data come together to offer climate researchers a treasure trove of ongoing, real-time information. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

You may also be interested in:

• Roundtable: Revlon and SAP executives describe accretive benefits from aggressive cloud adoption
  
• Case Study: Strategic Approach to Disaster Recovery and Data Lifecycle Management Pays Off for Australia's SAI Global
• Virtualization Simplifies Disaster Recovery for Insurance Broker Myron Steves While Delivering Efficiency and Agility Gains Too
• SAP Runs VMware to Provision Virtual Machines to Support Complex Training Courses
• Case Study: How SEGA Europe Uses VMware to Standardize Cloud Environment for Globally Distributed Game Development
• Germany's Largest Travel Agency Starts a Virtual Journey to Get Branch Office IT Under Control

For Steria, Cloud Not So Much a Technology as a Catalyst to Responsive and Agile Business

Transcript of a sponsored BriefingsDirect podcast on how IT service delivery company Steria standardizes processes in the cloud for improved delivery.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Performance podcast series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your co-host and moderator for this ongoing discussing of IT innovation and how it's making an impact on people’s life.

Once again, we're focusing on how IT leaders are improving performance of their services to deliver better experiences and payoffs for businesses and end users alike. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

Now, we're joined by our co-host for this sponsored podcast series, Chief Evangelist at HP, Paul Muller. Welcome, Paul. Where are you coming from today?

Paul Muller: Hi, Dana. Today, I'm in a fortunate position. I've been at home now for nearly two weeks running, which is something of a record. I'm down here in Melbourne, Australia.

Gardner: I am glad you can join us from home. We have a fascinating show today, because we are going to learn about how a prominent European IT-enabled business services provider, Steria, is leveraging cloud services to manage complexity and better services to customers. Getting more from cloud services seems to be a huge part of the IT landscape these days.

Paul, is that what you are finding -- that the cloud model is starting to impact this whole notion of effective performance across services in total?

Muller: This is a conversation I've been having a lot lately. The word cloud gets thrown around a lot, but when I drill into the topic, I find that customers are really talking about services and integrating different services, whether they are on-premises, in the public cloud arena, or even that gray land, which is called outsourcing. [Follow Paul on Twitter.]

It's the ability to integrate those different supply models -- internal, external, publicly sourced cloud services -- that really differentiate some of the more forward-leaning organizations from those who are still trying to come to grips with what it means to adopt a cloud service.

Gardner: Maybe a year or two ago, we were focused on the "how" with cloud, and now we seem to be moving beyond that to the "what," what you get regardless of how you do it. Does that sound about right?

Muller: You couldn’t have put it better. The way I had it described to me recently is that it’s moving away from talking about the plumbing to talking about what you're trying to produce. That that’s really the fundamental change that has occurred in the last 18 months.

Business opportunity

We've all come to realize that cloud isn’t so much a technology issue, as it is a business opportunity. It’s an opportunity to improve agility and responsiveness, while also increasing flexibility of cost models, which is incredibly important, especially given the uncertain economic outlook that not only different countries have, but even different segments within different countries.

Take something like the minerals and resources areas within my own country, which are booming right now. Whereas, if you look at other areas of business, perhaps media, or particularly print media, right now, they're going through the opposite type of revolution. They're trying to work out how to adjust their cost to declining demand.

Gardner: With that, let’s get to our guest. He's been a leading edge adopter for improving IT service delivery for many years, most recently as the IT Service Management (ITSM) Solution Manager at Steria, based near Paris.

Please join me in welcoming Jean-Michel Gatelais. Welcome to BriefingsDirect, Jean-Michel.

Jean-Michel Gatelais: Thank you very much. Yes, at Steria, I'm in charge of the Central ITSM Solution we provide for our customers, and I am in-charge of the Global ITSM Program Roadmap, including the ongoing integration from ServiceCenter 6 to Service Manager 9. I'm also responsible for the quality of service that we deliver with this solution, and of the transition of new customers on this platform.

Gardner: Let’s start at a high level, Jean-Michel. Because you've been doing this for quite some time with a focus on IT service delivery and ITSM, has this changed quite a bit in just the past few years? If so, what’s different now about IT service delivery than just say few years ago?

Gatelais: It has changed a lot. In fact, few years ago it was something that was very atomic, with different processes and with people running the service with different tools. About three to five years ago, people began to homogenize the processes to run the service, and we saw that in Steria.

In Steria, we bought some companies and we grew. We needed to establish common processes to proceed by a common platform, and that what’s what we did with Service Manager. Now, the way we deliver service is much more mature for all the processes and for the ITSM processes.

Gardner: Paul Muller, how does that jibe with what you're seeing? It sounds like he's very representative of the market in total.

Muller: The desire to standardize processes is a really big driver for organizations as they look to improve efficiency and effectiveness. So it's very similar what we're seeing. In fact, I was going to ask Jean-Michel a question. When you talk about homogenizing processes or improving consistently, how does that help the organization? How does that help Steria and its customers perform better?

IT provider

Gatelais: This allows us to deliver the service, whatever the location or organization, because we're an IT provider. We provide services for our customers that can be offshore, nearshore, in Steria local premises, and even in the plant premises. All the common processes and the solution allow us to do to this independently of the customer. Today with this process, we're able to run services for more than 200 customers.

Gardner: I suppose we should learn a bit more about Steria. You are primarily in Europe and the UK. Tell us a bit about your business, who your customers are, and perhaps some of the high-level goals and strategies that you're pursuing.

Gatelais: Steria is an IT service provider. We are about a little more than 40 years old. Our business is mainly in system integration, application management, business process outsourcing, and infrastructure management services.

We have big customers in all sectors of industry and services, such as public sector, banking, industry, telecom, and so on. We have customers both in France and UK mainly, but in the whole of Europe also. For example, we have British Telecom, Orange, and the public sector in the UK, with police etc.

Gardner: I see among your services that you are delivering cloud Workplace on Command, for example, Infrastructure On Command. Is this a bigger part of your business now? Do you find that servicing your cloud customers is dominating some of your strategic thinking?

We have an industrialized solution, allowing our customers to order infrastructure in a couple of minutes.

Gatelais: Yes. Actually, it’s growing day after day. We launched our cloud offering about 18 months ago. Now we can say that we have an industrialized solution, allowing our customers to order infrastructure in a couple of minutes. And this is really integrated with the whole service management solution and the underlying infrastructure.

Gardner: I suppose this gets to this self-service mentality that we are seeing, Paul. End users are seeking a self-service type of approach. They know that they can get services quite easily through a variety of consumer-based means. They're looking for similar choice and enablement in their business dealings.

It seems that an organization like Steria is at the forefront of attracting that sense of enablement and empowerment and then delivering it through a cloud infrastructure. They're interesting on two levels: one, they're delivering cloud and enablement, but they are also using cloud to power their own ability to do so.

Muller: I don’t know if Jean-Michel has seen this, but we see almost a contradiction within enterprise users of cloud. We see groups that will quite readily go out and adopt cloud services. The so-called consumerization trend is quite prevalent, especially with what I would describe as simple services. For example, office automation tools, collaboration tools, etcetera.

Yet, simultaneously, we see reluctance sometimes, particularly for the IT organization, to let go and cloud source services and applications. I sometimes refer to them as "application huggers" or "server huggers."

Relinquish control

In other words, if they can’t see it or touch it, they're reluctant to relinquish control. The most fascinating part for me is that you can often find those two behaviors inside the very same organization. Sometimes, the same person can have diametrically opposed views about the respective merits of those two approaches. Does that make sense?

Gardner: We should put the question directly to Jean-Michel. Are you selling and delivering cloud services to the IT department or others? Maybe we could call that shadow IT?

Gatelais: We do both. In fact, the cloud today is used both for internal organizations and also for our customers. Then, the cloud offering set-up asks to study a business model to study the way we will sell such service. For us, at the central level at Steria, there is no difference between internal delivery and delivery for our customers.

Gardner: That’s pretty interesting. Do you find that you've had to tailor your services for those non-IT users? Is there something about billing, invoicing, or self-serve that you've put in place in order to better accommodate the non-IT part of the market?

Gatelais: No. In fact, what we're trying to do is to standardize, as much as possible, the basic offering we propose. On top of that, we have additional requests from our customers. Then, we try to adapt our offering to the specific request.

Providing infrastructure services is not so difficult, but providing platform-as-a-service (PaaS) features can be.

Providing infrastructure services is not so difficult, but providing platform-as-a-service (PaaS) features can be. Even software as a service (SaaS) can be simpler than PaaS, because you provide some package services, startup services, instead for platform services. It’s very consumer specific.

Gardner: So you have the opportunity to go with a fairly standardized approach, but then you can customize on top of that. I'd like to hear some more about your different services. I understand that there’s something called Steria Advanced Remote Services or STARS. How does that fit into the mix, Jean-Michel?

Gatelais: STARS is the ITSM platform Steria rolled out about five years ago, and today this is a framework. It's mainly based on HP products, because it's running on HP Service Manager online, Business Service Manager (BSM), and Operations Orchestration.

We see this platform as a service enabler, both service support platform and the service enabler, because we use it to manage and activate the services we propose to our customer, including cloud services, security services, and our new offering, Workplace On Command services.

STARS is the solution to manage value-added services Steria is offering to its customers.

Muller: I have a question for Jean-Michel. When a customer thinks about taking services that maybe they used to run internally and moving those services to Steria, how important is it for them to maintain visibility and control, as they are thinking about moving to cloud?

Depends on the customers

Gatelais: It depends on the customers. You have some customers that are ready to use the services you provide on a common environment, but you also have customers requiring more specific solutions that we can give to them. Steria is developing some facilities to roll out and to instantiate the platforms for dedicated environments.

For example, the STARS solution, with Service Manager in the solution, we can deploy it, instantiate it, when the customer requires it.

Muller: Just following on from that, there's a perception that when you move to cloud services, people don’t really care about visibility, metrics, and service-level reports, because that’s all part of the service-level agreement (SLA). Do you find that customers actually want to see, how their service is performing -- what's the availability and level of security? Do they look for that level of reporting from you?

Gatelais: It depends on the customers. Some are really outsourcing the services. They would only complain if they met some problems on the services.

But other customers want to have the visibility on the quality of service that is delivered by Steria. That means that we need to be able to publish the SLA we have for our offering, but also to publish monthly, for example, the key performance indicators (KPIs) of this platform.

It’s the KPI discussion that is of such great interest to enterprises today.

Muller: And that is certainly a perfect question, because, Dana, it’s the KPI discussion that is of such great interest to enterprises today.

Gardner: Right, and I'm impressed that Steria can manage this variety and be able to provide to each of these customers what they want on their own terms, which is, as you point out, is really what they're calling for.

For you as a provider, that must really amount to quite a bit of complexity. How do you get a handle on that ability to maintain your own profitability while dealing with this level of variability and the different KPIs and giving the visibility to them?

Gatelais: One of the advantages of the cloud structure is that you have to ask these questions in advance. That means that when Steria is designing a new offering, we first design the business model. In fact, that will allow us either to propose some shared services, or for the client that has requested it, some visibility to the services, but based on standard platforms. We try to remain standard in what we propose, and the flexibility is in the configuration of what we propose.

Gardner: How about providing the visibility so that the sense of confidence, which is also so important in these early years of cloud adoption, is maintained? Do you provide specific views, insights, dashboards? What is it that you can provide to your customers so that they feel themselves in control even though they are no longer in a sense running these systems?

Gatelais: We provide the KPIs that are published for the service offering. This will include such information as service availability rates, outage problems, change management, and also activity reporting.

Strategic decisions

Gardner: Let’s look at this for a moment through the eyes of some of your customers, Jean-Michel. They're able to make their own strategic decisions better, knowing what they can do on-premises and what they can do to outsourcing models. They can make determinations about what is core and what’s context for their own capabilities and differentiation. What has that meant for them?

Do you have any anecdotes or insights into some of the benefits to their overall business that they have been able to make, because they can look to an organization like Steria and say, "Here, you do it. We're going to focus on something else?"

Gatelais: Yes. The example I can give is the flexibility the service offering can give to the customers in the software development area.

For example, it allows you to set up some development platforms for a limited period of time, allowing product development. With the service we offer, when the project is finished and you enter into the application management mode, the plant is able to say, "I stopped the server." It's backed up, and if six months later the customer wants to develop a new release of this software, then we would restore his environment. In the meantime, he won't have the use of the platform, but he'll be able to continue his development. This is very flexible.

Gardner: Paul, you must be seeing a lot of this that for many adopters with the test dev, quality assurance, the need for elasticity for those builds and environments around the test and development lifecycle. This sort of provides the killer use case for cloud.

The notion of tying all of that capital equipment up and leaving it idle for that period of time is simply not tenable.

Muller: Yes, but on and off-premises. The interesting part is that the development and test process is such a resource-intensive process, while you are in the middle of that process. But the minute you are done with it, you go from being almost 100 percent busy and consuming 100 percent of the resources, to, in some cases, doing nothing, as Jean-Michel said, for months, possibly, even years, depending on the nature of the project.

The notion of tying all of that capital equipment up and leaving it idle for that period of time is simply not tenable. The idea of moving all of that into a flex up-flex down model is probably one of the single most commonly pursued use cases for both public and private cloud today.

The other one, as Jean-Michel has already spoken to, is that the idea of more discrete services, particularly that of helpdesk, is just going crazy in terms of adoption by customers.

Gardner: Jean-Michel, how about some of the different sectors of the market? Do government clients of yours in Europe and the UK approach this any differently than the private sector? And, do small-to-medium-size businesses (SMBs) seem to be approaching your services or have different requirements than the larger enterprises?

Gatelais: The main difference between government and the private sector is the security issue. Most of governments ask for more confidentiality. They're very often reluctant to share their data or their business, with others. For such clients, we need to have a dedicated offering.

Dedicated offering

For example, in the UK, a customer from government didn’t want to run their services on shared platforms and asked for a dedicated environment. Because the whole ITSM offering from Steria is running on just one environment, we were able to instantiate such services only for their use.

Muller: That’s an interesting topic right there, Dana. I don’t know whether you're seeing this a lot in your interactions with clients, but the whole idea that cloud is a shared resource pool works brilliantly on paper.

But as Jean-Michel said, practically speaking, for reasons of data sovereignty, for reasons of security, and in some cases for regulatory reasons, the customer will insist that the service be effectively a hosted solution. It’s not that different from almost a traditional outsourcing situation, would you say, Jean-Michel?

Gatelais: Yes.

Gardner: One of the things I am seeing is some of the vision in terms of cloud a few years ago was that one size would fit all, or that it’s cookie cutter, and that there won’t be a need for high variability. But I think what we are actually seeing in practice, and Jean-Michel is certainly highlighting this, is that the KPIs are going to be different for organizations.

There are going to be different requirements for public and private, large and small, jurisdiction by jurisdiction, regulation and compliance. You really need to be able to have the flexibility, not just at the level of infrastructure, but at the level of the types of services, the way that they're built, invoiced, and measured and delivered.

They're interesting for small organizations, because they don’t have to heavily invest in solutions, and we're able to propose shared solutions.

Gatelais: The way we propose the services is they're interesting for small organizations, because they don’t have to heavily invest in solutions, and we're able to propose shared solutions. This is SaaS, this is cloud, and for them it’s very interesting, because it is much more cheaper.

Gardner: Well, we are going to be coming close to the end of our time. Jean-Michel, I wonder if you have any thoughts for those who might be embarking on something like a STARS capability.

They will be thinking about what they should put in place in order to accommodate the complexity, the security, being able to have granular services that they can deliver regardless of location to the variety of different types of clients. What do you advise others who would be pursuing a similar objective?

Gatelais: With such offerings you have to design and think much more than before, to think before running out your solution. You need to be clear on what you want to propose to what kind of customers, where is the market, and then to design your offering according to this. Then, build your business model according to those assumptions.

Gardner: In North America, we might say that that’s skating to where the hockey puck is going to be, rather than where it is.

Gatelais: Yes.

KPIs that matter

Muller: A question from me, Dana, for Jean-Miche. Right now, I've got a couple of metrics, a couple of KPIs, that matter to me really deeply. From your perspective, are there one or two KPIs that you're looking at at the moment that either make you really happy or that are a cause for concern for you, as you think about business and delivering your services. What are the KPIs that matter to you?

Gatelais: What is very difficult for new services is to evaluate the actual return on investment (ROI). You can establish a business model, a business plan to see if what you will do, you will make some profit with it, but it's much more difficult is to evaluate the ROI.

If I don’t buy this service, it would cost me an amount; if I buy this service, okay, it will cost the service fee, but what would I spend next to that. This is very difficult to measure.

Muller: And it's probably one of the most important KPIs in business, wouldn’t you say, Dana?

Gardner: Absolutely, yes.

Gatelais: It may be basic, but you should take the configuration management process. That is very important, even in cloud offerings. It's very difficult to make evident that if you do some configuration management, you will have higher a ROI than if you don’t do it.

It's very difficult to make evident that if you do some configuration management, you will have higher a ROI than if you don’t do it.

Muller: The cost justification of the investment is the challenge?

Gatelais: Exactly. Today, even internally in Steria, it's much more difficult to get approval to develop and to improve configuration management, because people don’t see the interest, as you don’t sell it directly. It's just a medium to improve your service.

Muller: That’s such a good point. And Dana, it's one of the great benefits. This is going to sound a little bit like an infomercial, but it's worth stating. One of the reasons we've been moving so much of our own management software to the cloud is because it's behind the scenes. It's often seen as plumbing, and people are reluctant to invest often in infrastructure and plumbing, until it has proven its benefit.

It's one of the reasons we've moved to a more variable cost model, or at least have made it available for organizations who might want to dip their toe in the water and show some benefits before they invest more heavily over time.

Distinct line

Gardner: Historically, Paul, it's been difficult to draw a distinct line between technology investments and business payoffs and paybacks, even though we have general productivity numbers to support it.

But now, with that greater insight into the management capabilities along the way, when you do everything as a service, you can meter, you can measure, and you can pay as you go. You're really starting to put in place the mechanisms for determining quite distinctly what the payoffs are from investments in IT at that critical business payoff level. So I think that’s a very interesting development in the market.

Muller: The transparency improves, and because you have a variable cost model, it lowers the pain threshold in terms of people being willing to experiment with an idea, see if it works, see if it has that payoff, that ROI. If it doesn’t, stop doing it, and if it does, do more of it. It's really, really very simple.

Gardner: Right, much less of an art and a bit more of a science, but in a good way.

Muller: Absolutely.

Gardner: I'm afraid we are going to have to leave it there. I'd like to thank you all for joining our discussion, and of course, I'd like to thank our supporter for this series, HP Software, and remind our audience that they can carry on this dialogue with Paul Muller through the Discover Performance Group on LinkedIn.

You can also gain more insights and gather more information on the best of IT performance management at www.hp.com/go/discoverperformance.

And with that, please join me in thanking today's guests, our co-host, Chief Evangelist at HP, Paul Muller. Thanks so much, Paul.

Muller: Good talking to you again, Dana.

Gardner: And also a huge thanks to Jean-Michel Gatelais, IT Service Management Solution Manager at Steria, based near Paris. Thanks so much, Jean-Michel.

Gatelais: You're welcome. It was a pleasure.

Gardner: I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your co-host, and moderator for this ongoing discussion of IT innovation and how it's making an impact on people’s lives. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: HP.

Transcript of a sponsored BriefingsDirect podcast on how IT service delivery company Steria standardizes processes in the cloud for improved delivery. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

You may also be interested in:

• Where cloud computing takes us: Hybrid services delivery of essential information across all types of applications
  
• With CMS 10, HP puts workload configuration data newly in hands of those who can best use it to manage services delivery
  
• Investing Well in IT With Emphasis on KPIs Separates Business Leaders from Business Laggards, Survey Results Show
• Expert Chat with HP on How Better Understanding Security Makes it an Enabler, Rather than Inhibitor, of Cloud Adoption
• Expert Chat with HP on How IT Can Enable Cloud While Maintaining Control and Governance
• Continuous Improvement and Flexibility Are Keys to Successful Data Center Transformation, Say HP Experts

Where Cloud Computing Ultimately Takes Us: Hybrid Services Delivery of Essential Information Across All Types of Apps

Transcript of a BriefingsDirect podcast from the HP Discover 2012 Conference on hybrid services delivery and converging the evolving elements of cloud computing.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Performance podcast series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your co-host and moderator for this ongoing discussing of IT innovation and how it's making an impact on people’s life.

Once again, we're focusing on how IT leaders are improving performance of their services to deliver better experiences and payoffs for businesses and end users alike. This time, we’re coming to you directly from the recent HP Discover 2012 Conference. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

We’re now joined by two top HP evangelists to discuss the concepts around HP’s Converged Cloud. Please join me in welcoming our co-host Paul Muller, the Chief Software Evangelist at HP. Welcome.

Paul Muller: Hi, Dana. How are you doing?

Gardner: I'm doing great. Good to be with you again. We are also here with Christian Verstraete, Chief Technologist for Cloud Strategy at HP. Welcome back, Christian.

Christian Verstraete: Thank you, Dana.

Gardner: We've been hearing an awful lot around the notion of an HP converged cloud, and it has a lot of different aspects to it. There are a lot of different products to support it -- public, private, application development, data services, and analysis services -- but one thing that really caught my attention and notice was that you’ve separated the notion of hybrid computing from hybrid delivery. Can you help me understand better why they're different and what HP means by hybrid delivery?

Verstraete: Hybrid computing typically is combining private and public clouds. We feel that many of our customers still have a traditional environment, and that traditional environment will not go away anytime soon. However, they're actually looking at combining that traditional environment, the data that’s in that traditional environment and some of the functionality that's out there, with the public cloud and the private cloud.

The whole concept of hybrid delivery is tying that together. It goes beyond hybrid computing or hybrid cloud. It adds the whole dimension of the traditional environment. And, to our mind, the traditional environment isn't going to go away anytime soon.

Gardner: One of the things we’ve also seen in the evolution of public cloud is that things are very segmented. There are data services, infrastructure services, and workloads that you can put in, based on certain platforms using certain tools and APIs.

What you seem to be saying at HP is that that should be deconstructed and allowed to be more of a lifecycle, converged. Paul, help me understand how the traditional understanding of cloud computing as segments of infrastructure services has changed?

Muller: From that perspective, the converged cloud is really about three things for us. The first is having greater levels of choice. The key point that Christian just made is that you can't afford to live in the world of, "It’s just public; it's just private; or I can ignore my traditional investments and infrastructure." Choice is critical, choice in terms of platform and application.

The second thing, though, is that in order to get great choices, you need consistency as an underlying platform to ensure that you're able to scale your people, your processes, and more importantly, your investments across those different environments.

Consistent confidence

The last one is probably the biggest area of passion for me -- confidence. We spoke a little bit earlier about how so many clients, as they move to cloud, are concerned about the arm’s-length relationship they have with that provider. How can I get back the confidence in security and service levels, and make sure that that confidence is consistent across both my on-premises and-off premises environments?

Gardner: Another thing we've seen to date is an emphasis on workloads, just creating elastic-compute resources for things like an environment to run an application. But you seem to have a much deeper emphasis on data services. Why is data more important than, or as important as, workloads -- or have we moved beyond the importance of workloads?

Verstraete: People have started looking at cloud from pure infrastructure, reuse, and putting workflows in some particular places in infrastructure. The world is moving beyond that at the moment. On one end, you have software as a service (SaaS) starting to play and getting integrated in a complete cloud environment and a complete cloud function.

We also have to realize that, in 2011, the world created about 1.8 zettabytes of data, and that data has a heck of a lot of information that enterprises actually need. And as enterprises understand what they can get out of the data, they want that data right there at their fingertips. What makes it even more interesting is that 90 percent of that data is unstructured.

We've been working for the last 30 years with structured data. We know all about databases and everything, but we have no clue about unstructured data. How do I know the sentiments that people have compared to my brand, my business, my product? That's the sort of question that's becoming important, because if you want to do warranty management or anything else, you want to understand how your users feel. Hence, the importance of all of this data.

We know all about databases and everything, but we have no clue about unstructured data.

Gardner: Perhaps we should say information instead of data.

Verstraete: You're right.

Muller: I’d add something else to what Christian just said. We were here with the Customer Advisory Board. We had a pre-meeting prior to the actual conference, and one of them said something I thought was kind of interesting, remarkable actually.

He said, "If I think back 30 years, my chief concern was making sure the infrastructure was functioning as we expected it to. As I moved forward, my focus was on differentiating applications." He said, "Now that I'm moving more and more of the first two into the cloud, my focus really needs to be on harnessing the information and insight. That’s got to become the core competency and priority of my team."

Verstraete: There's one element to add to that that we shouldn't forget, and that is the end-user. When you start talking about converged clouds -- we're not there yet, but we're getting there -- it's really about having one, single user experience. Your end-user doesn't need to know that this function runs in a public cloud, that function runs in a private cloud, or that function runs in the traditional environment.

No. He just wants to get there and use whatever it is. It's up to IT to define where they put it, but he or she just wants to have to go one way, with one approach -- and that's where you get this concept of a unique user experience. In converged cloud that’s absolutely critical.

Composite hybrids

Gardner: Another term that was a bit fresh for me here was this notion of composite hybrid applications. This was brought up by Biri Singh in his discussion. It sounds as if more and more combinations of SaaS, on-premises, virtualized, physical, and applications need to come together. In addition to that, we're going to be seeing systems of record moving to some variety of cloud or combination of cloud resources.

The question then is how can we get to the data within all of those applications to create those business processes that need to cut across them? Is that what you're talking about with Autonomy and IDOL? Is that the capability we are really moving toward, combining data and information from a variety of sources, but in a productive and useful way?

Verstraete: Absolutely. You got it spot on, Dana. It's really about using all of the information sources that you have. It's using your own private information sources, but combining them with the public information sources. Don’t forget about those. Out of that, it's gathering the information that's relevant to the particular thing that you're trying to achieve, be it compliance, understanding how people think about you, or anything else.

The result is one piece of information, but it may come from multiple sources, and you need an environment that pulls all of that data and gets at that data in a useful form, so you can start doing the analysis and then portraying the information, as you said, in a way that is useful for you. That's what IDOL and Autonomy does for us in this environment.

Muller: I am going to add something to that, which is, of course, not yesterday, not today, but in real-time. One of the critical elements to that is being able to access that information in real-time. All of us are active in social media, and that literally reflects your customer’s attitudes from minute to minute.

One of the critical elements to that is being able to access that information in real time.

Let me give you a use-case of how the two come together. Imagine that you have a customer on a phone call with a customer service operator. You could use Autonomy technology to detect, for example, the sound of their voice, which indicates that they're stressed or that they're not happy.

You can flag that and then very quickly go out to your real-time structured systems and ask, "How much of an investment has this client made in us? Are they are high net worth customer to us or are they a first-time transactor? Are they active in the social media environment? What are they saying about us right now?"

If the pattern is one that may be disadvantageous to the company, you can flag that very quickly and say, "We want to escalate this really quickly to a manager to take control of the situation, because maybe that particular customer service rep needs some coaching or needs some help." Again, not in a week’s time, not in a month’s time, but right there, right now. That’s a really important point.

Gardner: This is a bit of a departure. Thinking about systems of record again, one of the obstacles that folks have is to get a single view of the customer. You might have to dig into three or four databases and cut across multiple applications.

They are all internal, but you would get some very powerful insights that you could extend to your business processes -- sales, marketing, research into what new requirements will be coming into products and services, more efficiency in how you could provide service and support to those customers, and so on.

Abstraction in the cloud

We’re elevating that now to an abstraction in the cloud where almost an unlimited amount of information could be brought to bear on a question about a customer or a business process.

This really is a radical departure, and very powerful. But what's missing for me is how I actually avail myself of it. It's a good vision, but if I am a developer, a business analyst, or a leader in a company and I want a dashboard that gets me this information, how do we get this fire hose and make it manageable and actionable?

Verstraete: There are two different elements in this. The first thing is that we’re using IDOL 10, which is basically the combination, on one hand, of Autonomy and, on the other hand, of Vertica. Autonomy is for unstructured data, and Vertica for structured data, so you get the two coming together.

We’re using that as the backbone for gathering and analyzing the whole of that information. We've made available to developers a number of APIs, so that they can tap into this in real-time, as Paul said, and then start using that information and doing whatever they want with it.

Obviously, Autonomy and Vertica will give you the appropriate information, the sentiment, and the human information, as we talked about. Now, it's up to you to decide what you want to do with that, what you want to do with the signals that you receive. And that's what the developer can do in real-time, at the moment.

The great challenge is not lack of data or information, but it's the sheer volume.

Gardner: Paul, any thoughts in making this fire hose of data actionable?

Muller: Just one simple thought, which is meaning. The great challenge is not lack of data or information, but it's the sheer volume as you pointed out, when a developer thinks about taking all of the information that's available. A simple Google query or a Bing query will yield hundreds, even millions of results. Type in the words "Great Lakes," and what are you going to get back? You'll get all sorts of information about lakes.

But if you’re looking, for example, for information about depth of lakes, where the lakes are, where are lakes with holiday destinations, it's the meaning of the query that's going to help you reduce that information and help you sort the wheat from the chaff. It's meaning that's going to help developers be more effective, and that's one of the reasons why we focus so heavily on that with IDOL 10.

Gardner: And just to quickly follow up on that, who decides the meaning? Is this the end user who can take action against this data, or does it have to go through IT and a developer and a business analyst? How close can we get to those people at an individual level so that they can ascertain the meaning and then act on it?

Muller: It's a brilliant question, because meaning in the old sense of the term -- assigning meaning is a better way of putting it -- was ascribed to the developer. Think about tagging a blog, for example. What is this blog about? Well, this blog might be about something as you’re writing it, but as time goes on, it might be seen as some sort of historic record of the sentiment of the times.

So it moves from being a statement of fact to a statement of sentiment. The meaning of the information will change, depending on its time, its purpose, and its use. You can't foresee it, you can't predict it, and you certainly can't entrust a human with the task of specifically documenting the meaning for each of those elements.

Appropriate meaning

What we focus on is allowing the information itself to ascribe its own meaning and the user to find the information that has the appropriate meaning at the time that they need it. That's the big difference.

Gardner: So the power of the cloud and the power of an engine like IDOL and Vertica brought to bear is to be bale to serve up the right information to the right person at the right time -- rather than them having to find it and know what they want.

Verstraete: Exactly, that's exactly what it is. With that information they can then start doing whatever they want to do in their particular application and what they want to deliver to their end-user. You’re absolutely spot-on with that.

Gardner: Let's go to a different concept around the HP Converged Cloud, this notion of a virtual private cloud. It seems as if we’re moving toward a cloud of clouds. You don’t seem to want to put other public cloud providers out of business.

You seem to say, Let them do what they do. We want to get in front of them and add value, so that those coming in through our [HP] cloud, and accessing their services vis-à-vis other clouds, can get better data and analysis, security, and perhaps even some other value-added services. Or am I reading this wrong?

Many customers don’t have the transparency to understand what is really happening, and with transparency comes trust.

Verstraete: No, you’re actually reading this right. One of the issues that you have with public clouds today isn't a question of whether public cloud is secure or not secure or whether it's compliant or not compliant. Many customers don’t have the transparency to understand what is really happening, and with transparency comes trust.

A lot of our customers tell us, "For certain particular workloads, we don’t really trust this or that cloud, because we don’t really know what they do. So give us a cloud or something that delivers the same type of functionality, but where I can understand what is done from a security perspective, a process perspective, a compliance perspective, an SLA perspective, and so on?

They ask: "Where can I have a proper contract, not these little Ts and Cs that I tick in the box? Where can I have the real proper contract and understand what I'm getting into, so that I can analyze my potential risk and decide what security I want to have, and what risk I'm prepared to take?"

Gardner: So the way in which I would interface with the HP managed services cloud of clouds would be through SLAs and key performance indicators (KPIs), and the language of business risk, rather than an engineer’s check list. Is that correct?

Muller: Absolutely, exactly right. That's the important point. Christian talks about this all the time. It’s not about cloud; it’s about the services, and it’s about describing those services in terms of what a businessperson can understand. What am I going to get, what cost, at what quality, at what time, at what level of risk and security? And can I find the right solution at the right time?

Registry requirement

Gardner: I always go back to the notion that service-oriented architecture (SOA) came first and then the concepts around cloud and SaaS came later. And I still hold that, because there are certain elements of cloud that go right back to a registry and repository, enterprise service bus (ESB) with APIs and integration points, and the ability to deliver services across a variety of different systems, outputs, and devices.

One of the things that’s interesting about SOA is the requirement for that registry. You have something called the HP Cloud Marketplace, which is a layer on top of the converged cloud or within the converged cloud.

As a business, how do I start thinking about how I might start using the HP cloud to make new and better revenue, using some of these data services, recognizing the security, and being able to not just do IT differently, but actually do business differently?

Is there anything you can tell me about the HP Cloud Marketplace that would help people understand how there is a business opportunity here, too?

Verstraete: The marketplace isn’t there yet at the moment. It’s on its way. One of the elements that we're trying to do with HP Cloud Services in particular is to provide developers with a rich environment in which they can actually develop their applications.

We propose that once their applications are developed, once they are happy about that application, that they put that application in the marketplace. Through the marketplace, we will promote all the applications to our customer base and to our prospects, so that they can decide which service and applications they want to use. This will give business to the original developer.

Through the marketplace, we will promote all the applications to our customer base and to our prospects, so that they can decide which service and applications they want to use.

Gardner: Paul, could you add to that?

Muller: Dana, you and I have talked about this one before. You're one of the few industry analysts who really understands the fact that enterprise architecture’s concepts and constructs are critical to somebody trying to establish cloud.

Everything you spoke about, the notion of what services I have, where I can find them, who is providing them to me, keeping track of the relationships and the communication, the protocols, the contracts between each of those, is absolutely critical. The marketplace is one element of that. It helps you manifest that, but of course, it has to be used in concert with enterprise architecture principles.

Gardner: So a layer of governance on this marketplace would allow for that KPI- and AP-based language of business to allow for granular permission, access control, and a lower risk ability to use public services in an enterprise setting?

Verstraete: In some of the early versions of that marketplace that we've been working on, one of the concepts that we put in place is basically to say that if you're an enterprise, and the IT responsible for that enterprise will decide, amongst all the applications that are available in marketplace, which IT applications that are available to my company. I, as a user, then go in and see only what I'm eligible to use.

So you get these elements, where you can start within a very large service catalog. You zoom in and get a service catalog, which is specific for a particular enterprise. That’s part of that governance that Paul was just talking about. That’s where these things start to manifest themselves.

Gardner: If we go back full circle to earlier in our discussion talking about data and analytic services, perhaps a permission-governed filter combining what application services with what data services are either available or should be made available, gets us very close to a whole new way of using IT to do business?

Data and sovereignty

Muller: You've touched on a really important point here. You mentioned data, and the minute you mention data and cloud, any CIO on the planet that I speak to, certainly any regulator, will use two words -- "data" and "sovereignty." "Where is my data allowed to be at any point in time?"

That's such a critical point. It's one of the reasons we’re such a big fan of choice. When we think about cloud, and as Christian mentioned, we’re very open to other cloud providers integrating and working with us. With different regulators and in different countries, you’re going to want to see different types of approaches taken.

HP obviously isn’t going to be able to meet every permutation of that. Our partners will be able to find those markets, specialize in those areas, and provide that sort of regulatory comfort for that particular customer. We, of course, want to embrace them and integrate them into our platform.

Gardner: Before we break off, I’d like to ask you some of your impressions about the users here. You've been talking with CIOs and leaders within business. Christian, first with you, does anything jump out as interesting from the marketplace that perhaps you didn’t anticipate? Where are they interested most in this notion of the HP Converged Cloud?

Verstraete: A lot of customers, at least the ones that I talk to, are interested in how they can start taking advantage of this whole brand-new way with existing applications. A number of them are not ready to say, "I'm going to ditch what I have, and I am going to do something else." They just say, "I'm confident with and comfortable with this, but can I take advantage of this new functionality, this new environment? How do I transform my applications to be in this type of a world?" That's one of the elements that I keep hearing quite a lot.

A lot of customers are interested in how they can start taking advantage of this whole brand-new way with existing applications.

Gardner: So a crawl-walk-run, a transition, a journey. This isn’t a switch you flip; this is really a progression.

Verstraete: That is why the presence of the traditional environment, as we said at the beginning, is so important. You don’t take the 3,000 applications you have, plug them around, they all work, and you forget about a traditional environment. That's not how it works. It's really that period to start moving, and to slowly but surely start taking the full advantage of what this converged cloud really delivers to you.

Gardner: Paul, what is that community here telling you about their interests in the cloud?

Muller: A number of things, but I think the primary one is just getting ahead of this consumerization trend and being able to treat the internal IT organization and almost transforming it into something that looks and feels like an external service provider.

So the simplicity, ease of consumption, transparency of cost, the choice, but also the confidence that comes from dealing with that sort of consumerized service, is there, whether it's bringing your own device or bringing your own service or combining it on- and off-premises together.

Verstraete: Chris Anderson in his HP Discover keynote said something that resonated quite a lot with me. If you, as a CIO, want to remain competitive, you'd better get quick, and you'd better start transforming and move. I very much believe that, and I think that's something that we need, that our CIOs actually need to understand.

Gardner: I'm afraid we’ll have to leave it there. I want to thank our two guests, Christian Verstraete, the Chief Technologist for Cloud Strategy at HP. Thank you so much.

Verstraete: Thank you, Dana.

Gardner: And our co-host, Paul Muller, the Chief Software Evangelist at HP. Thank you, Paul.

Muller: It's always great having the opportunity to catch up with you, Dana.

Gardner: And I’ll also thank our audience for joining us for this special HP Discover Performance podcast, coming to you from the HP Discover 2012 Conference in Las Vegas.

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of HP sponsored discussions. Thanks again for joining, and come back next time.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast from the HP Discover 2012 Conference on hybrid services delivery and converging the evolving elements of cloud computing. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

You may also be interested in:

• HP Expert Chat Explores How Insight Remote Support and Insight Online Bring Automation, Self-Solving Capabilities to IT Problems
• Investing Well in IT With Emphasis on KPIs Separates Business Leaders from Business Laggards, Survey Results Show
• Expert Chat with HP on How Better Understanding Security Makes it an Enabler, Rather than Inhibitor, of Cloud Adoption
• Expert Chat with HP on How IT Can Enable Cloud While Maintaining Control and Governance
• Expert Chat on How HP Ecosystem Provides Holistic Support for VMware Virtualized IT Environments