MIT Media Lab Computing Director Details the Virtues of Cloud Computing for Agility and DR

Transcript of a Briefings Direct podcast on how MIT researchers are reaping the benefits of virtualization.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.

Dana Gardner: Hello, and welcome to a special BriefingsDirect podcast series coming to you directly from the VMworld 2014 Conference. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host throughout this series of BriefingsDirect IT strategy discussions.

Gardner

We’re here in San Francisco the week of August 25 to explore the latest developments in hybrid cloud computing, user computing, software-defined data center (SDDC), and virtualization infrastructure management.

Our next innovator case study interview focuses on the MIT Media Lab in Cambridge, Massachusetts and how they're exploring the use of cloud and hybrid cloud and enjoying such use benefits as speed, agility and disaster recovery (DR). 

To learn more about how the MIT Media Lab is using cloud computing, we’re joined by Michail Bletsas, research scientist and Director of Computing at the MIT Media Lab. Welcome.

Michail Bletsas: Thank you. 

Gardner: Tell us about the MIT Media Lab. How big is the organization? What’s your charter?

Bletsas: The organization is one of the many independent research labs within MIT. MIT is organized in departments, which do the academic teaching, and research labs, which carry out the research.

Bletsas

The Media Lab is a unique place within MIT. We deviate from the normal academic research lab in the sense that a lot of our funding comes from member companies, and it comes in a non-direct fashion. Companies become members of the lab, and then we get the freedom to do whatever we think is best.

We try to explore the future. We try to look at what our digital life will look like 10 years out, or more. We're not an applied research lab in the sense that we're not looking at what's going to happen two or three years from now. We're not looking at short-term future products. We're looking at major changes 15 years out.

I run the group that takes care of the computing infrastructure for the lab and, unlike a normal IT department, we're kind of heavy on computing. We use computers as our medium. The Media Lab is all about human expression, which is the reason for the name and computers are one of the main means of expression right now. We're much heavier than other departments in how many devices you're going to see. We're on a pretty complex network and we run a very dynamic environment.

Major piece

A lot has changed in our environment in recent years. I've been there for almost 20 years. We started with very exotic stuff. These days, you still build exotic stuff, but you're using commodity components. VMware, for us, is a major piece of this strategy because it allows us a more efficient utilization of our resources and allows us to control a little bit the server proliferation that we experienced and that everybody has experienced.

We normally have about 350 people in the lab, distributed among staff, faculty members, graduate students, and undergraduate students, as well as affiliates from the various member companies. There is usually a one-to-five correspondence between virtual machines (VMs), physical computers, and devices, but there are at least 5 to 10 IPs per person on our network. You can imagine that having a platform that allows us to easily deploy resources in a very dynamic and quick fashion is very important to us.

We run a relatively small operation for the size of the scope of our domain. What's very important to us is to have tools that allow us to perform advanced functions with a relatively short learning curve. We don’t like long learning curves, because we just don’t have the resources and we just do too many things.

You are going to see functionality in our group that is usually only present in groups that are 10 times our size. Each person has to do too many things, and we like to focus on technologies that allow us to perform very advanced functions with little learning. I think we've been pretty successful with that.

We really need to interact with our infrastructure on a much shorter cycle than the average operation.

Gardner: So your requirements are to support those 350 people with dynamic workloads, many devices. What is it that you needed to do in your data center to accommodate that? How have you created a data center that’s responsive, but also protects your property, and that allows you to reduce your security risk?

Bletsas: Unlike most people, we tend to have our resources concentrated close to us. We really need to interact with our infrastructure on a much shorter cycle than the average operation. We've been fortunate enough that we have multiple, small data centers concentrated close to where our researchers are. Having something on the other side of the city, the state, or the country doesn’t really work in an environment that’s as dynamic as we are.

We also have to support a much larger community that consists of our alumni or collaborators. If you look at our user database right now, it’s something in the order of 3,500, as opposed to 350. It’s a very dynamic in that it changes month to month. The important attributes of an environment like this is that we can’t have too many restrictions. We don’t have an approved list of equipment like you see in a normal corporate IT environment.

Our modus operandi is that if you bring it to us, we’ll make it work. If you need to use a specific piece of equipment in your research, we’ll try to figure out how to integrate it into your workflow and into what we have in there. We don’t tell people what to use. We just help them use whatever they bring to us.

In that respect, we need a flexible virtualization platform that doesn’t impose too many restrictions on what operating systems you use or what the configuration of the VMs are. That’s why we find that solutions, like general public cloud, for us are only applicable to a small part of our research. Pretty much every VM that we run is different than the one next to it. 

Flexibility is very important to us. Having a robust platform is very, very important, because you have too many parameters changing and very little control of what's going on. Most importantly, we need a very solid, consistent management interface to that. For us, that’s one of the main benefits of the vSphere VMware environment that we’re on.

Public or hybrid

Gardner: Of course, virtualization sounds like a great fit when you have such dynamic, different, and varied workloads. But what about taking advantage of cloud, public cloud, and hybrid cloud to some degree, perhaps for disaster recovery (DR) or for backup failover. What's the rationale, even in your unique situation, for using a public or hybrid cloud?

Bletsas: We use hybrid cloud right now that’s three-tiered. MIT has a very large campus. It has extensive digital infrastructure running our operations across the board. We also have facilities that are either all the way across campus or across the river in a large co-location facility in downtown Boston and we take advantage of that for first-level DR.

A solution like the vCloud Air allows us to look at a real disaster scenario, where something really catastrophic happens at the campus, and we use it to keep certain critical databases, including all the access tools around them, in a farther-away location.

It’s a second level for us. We have our own VMware infrastructure and then we can migrate loads to our central organization. They're a much larger organization that takes care of all the administrative computing and general infrastructure at MIT at their own data centers across campus. We can also go a few states away to vCloud Air [and migrate our workloads there in an emergency].

We know that remote events are remote, until they happen, and sometimes they do.

So it’s a very seamless transition using the same tools. The important attribute here is that, if you have an operation that small, 10 people having to deal with such a complex set of resources, you can't do that unless you have a consistent user interface that allows you to migrate those workloads using tools that you already know and you're familiar with.

We couldn’t do it with another solution, because the learning curve would be too hard. We know that remote events are remote, until they happen, and sometimes they do. This gives us, with minimum effort, the ability to deal with that eventuality without having to invest too much in learning a whole set of tools, a whole set of new APIs to be able to migrate.

We use public cloud services also. We use spot instances if we need a high compute load and for very specialized projects. But usually we don’t put persistent loads or critical loads on resources over which we don’t have much control. We like to exert as much control as possible.

Gardner: I'd like to explore a little bit more this three-tiered cloud using common management, common APIs. It sounds like you're essentially taking metadata and configuration data, the things that will be important to spin back up an operation should there be some unfortunate occurrence, and putting that into that public cloud, the vCloud Air public cloud. Perhaps it's DR-as-a-service, but only a slice of DR, not the entire data. Is that correct?

Small set of databases

Bletsas: Yes. Not the entire organization. We run our operations out of a small set of databases that tend to drive a lot of our websites. A lot of our internal systems drive our CRM operation. They drive our events management. And there is a lot of knowledge embedded in those databases.

It's lucky for us, because we're not such a big operation. We're relatively small, so you can include everything, including all the methods and the programs that you need to access and manipulate that data within a small set of VMs. You don’t normally use them out of those VMs, but you can keep them packaged in a way that in a DR scenario, you can easily get access to them.

Fortunately, we've been doing that for a very long time because we started having them as complete containers. As the systems scaled out, we tended to migrate certain functions, but we kept the basic functionality together just in case we have to recover from something.

We are fortunate enough to have a very good, intimate knowledge of our environment. We know where each piece lies. That’s the benefit of running a small organization

In the older days, we didn’t have that multi-tiered cloud in place. All we had was backups in remote data centers. If something happened, you had to go in there and find out some unused hardware that was similar to what you had, restore your backup, etc.

Now, because most of MIT's administrative systems run under VMware virtualization, finding that capacity is a very simple proposition in a data center across campus. With vCloud Air, we can find that capacity in a data center across the state or somewhere else.

Gardner: For organizations that are intrigued by this tiered approach to DR, did you decide which part of those tiers would go in which place? Did you do that manually? Is there a part of the management infrastructure in the VMware suite that allowed you to do that? How did you slice and dice the tiers for this proposition of vCloud Air holding a certain part of the data?

Bletsas: We are fortunate enough to have a very good, intimate knowledge of our environment. We know where each piece lies. That’s the benefit of running a small organization. We occasionally use vSphere’s monitoring infrastructure. Sometimes it reveals to us certain usage patterns that we were not aware of. That’s one of the main benefits that we found there.

We realized that certain databases were used more than we thought. Just looking at those access patterns told us, “Look, maybe you should replicate this." It doesn’t cost much to replicate this across campus and then maybe we should look into pushing it even further out.

It is a combination of having a visibility and nice dashboards that reveal patterns of activity that you might not be aware of even in an environment that's not as large as ours.

Gardner: We’re here at VMworld 2014. There's been quite a bit of news, particularly in the vCloud Air arena. We've talked and heard about betas for ObjectStore and for virtual private cloud. Are these of interest to you now that you’ve done a hybrid cloud using DR-as-a-service? Does anything else intrigues you?

Standard building blocks

Bletsas: We like the move toward standardization of building blocks. That’s a good thing overall, because it allows you to scale out relatively quickly with a minor investment in learning a new system. That’s the most important trend out there for us. As I've said, we're a small operation. We need to standardize as much as possible, while at the same time, expanding the spectrum of services. So how do you do that? It’s not a very clear proposition.

The other thing that is of great interest to us is network virtualization. MIT is in a very peculiar situation compared to the rest of the world, in the sense that we have no shortage of IP addresses. Unlike most corporations where they expose a very small sliver of their systems to the outside world and everything happens on the back-end, our systems are mostly exposed out there to the public internet.

We don’t run very extensive firewalls. We're a knowledge dissemination and distribution organization and we don’t have many things to hide. We operate in a different way than most corporations. That shows also with networking. Our network looks like nothing like what you see in the corporate world. The ability to move whole sets of IPs around our domain, which is rather large and we have full control over, is a very important thing for us.

It allows for much faster DR. We can do DR using the same IPs across the town right now because our domain of control is large enough. That is very powerful because you can do very quick and simple DR without having to reprogram IP, DNS Servers, load balancers, and things like that. That is important.

That is very powerful because you can do very quick and simple DR without having to reprogram IP, DNS Servers, load balancers, and things like that.

The other trend that is also important is storage virtualization and storage tiering and you see that with all the vendors down in the exhibit space. Again, it allows you to match the application profile much easier to what resources you have. For a rather small group like ours, which can't afford to have all of its disk storage and very high-end systems, having a little bit of expensive flash storage, and then a lot of cheap storage, is the way for us to go.

The layers that have been recently added to VMware, both on the network side and the storage side help us achieve that in a very cost-efficient way.

Gardner: The benefits of having a highly virtualized environment -- including the data center, including the end user computing endpoints -- gives you that flexibility of taking workloads and apps from development to test to deployments. So there's a common infrastructure approach there, but also a common infrastructure across cloud, hybrid cloud, and DR.

So it’s sort of a snowball effect. The more virtualization you're adapting, the more dynamic and agile you can be across many more aspects of IT.

Bletsas: For us, experimentation is the most important thing. Spinning out a large number of VMs to do a specific experiment is very valuable and being able to commandeer resources across campus and across data centers is a necessary requirement for something like an environment like this. Flexibility is what we get out of that and agility and speed of operations.

In the older days, you had to go and procure hardware and switch hardware around. Now, we rarely go into our data centers. We used to live in our data centers. We go there from time to time but not as often as we used to do, and that’s very liberating. It’s also very liberating for people like me because it allows me to do my work anywhere.

Gardner: Very good. I'm afraid we’ll have to leave it there. We’ve been discussing the virtues of cloud computing and hybrid cloud computing with the MIT Media Lab. I’d like to thank our guest, Michail Bletsas, research scientist and Director of Computing at the MIT Media Lab in Cambridge, Mass. Thanks so much.

Bletsas: Thank you.

Gardner: And also a big thank you to our audience for joining this special podcast series coming to you directly from the 2014 VMworld Conference in San Francisco.

I'm Dana Gardner; Principal Analyst at Interarbor Solutions, your host throughout this series of VMware-sponsored BriefingsDirect IT discussions. Thanks again for listening and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.

Transcript of a Briefings Direct podcast on how MIT researchers are reaping the benefits of virtualization. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

• University of New Mexico Delivers Efficient IT Services by Centralizing on VMware Secure Cloud Automation
• A Gift That Keeps Giving, Software-Defined Storage Now Demonstrates Architecture-Wide Benefits
• The Open Group panel: Internet of things poses massive opportunities and obstacles
• Navicure Gains IT Capacity Optimization and Performance Monitoring Using VMware vCenter Operations Manager
• How Waste Management builds a powerful services continuum across IT operations, infrastructure, development, and processes
• VMware vCloud Hybrid Service Powers Journey to Zero-Cost Applications Support for City of Melrose
• Millennium Pharmacy Takes SaaS Model to New Heights VIA Policy-Driven Operations Management and Automation
• Thomas Duryea Consulting provides insights into how leading adopters successfully solve cloud risks

VMware CTO Steve Herrod on How the Software-Defined Datacenter Benefits Enterprises

Transcript of a BriefingsDirect podcast on how pervasive software enablement helps battle IT datacenter complexity.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: VMware.

Get the latest announcements about VMware's cloud strategy and solutions by tuning into VMware NOW, the new online destination for breaking news, product announcements, videos, and demos at: http://vmware.com/go/now.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you're listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on the intriguing concept of the software-defined datacenter. We'll look at how some of the most important attributes of datacenter capabilities and performance are now squarely under the domain of software enablement.

We'll see how those who are now building and managing datacenters are gaining heightened productivity, delivering far better performance, and enjoying greater ease in operations and management -- all thanks to innovations at the software-infrastructure level.

A top technology leader at VMware, Steve Herrod has championed this vision of the software-defined datacenter and how the next generation of foundational IT innovation is largely being implemented above the hardware. [Disclosure: VMware is a sponsor of BriefingsDirect podcasts.]

We're here with him now to further explore how advances in datacenter technologies and architecture are, to an unprecedented extent, being driven primarily through software. Please join me in welcoming to BriefingsDirect, Steve Herrod, Chief Technology Officer and Senior Vice President of Research & Development at VMware. Welcome, Steve.

Steve Herrod: Thanks, Dana. It’s a great topic. I'm really looking forward to sharing some thoughts on it.

Gardner: We appreciate your being here. We've heard a lot over the decades about improving IT capabilities and infrastructure management, but it seems that many times we peel back a layer of complexity and we get some benefits, and we find ourselves like the proverbial onion, back at yet another layer of complexity.

Complexity seems to be a recurring inhibitor. I wonder if this time we're actually at a point where something is significantly different. Are we really gaining ground against complexity at this point?

Herrod: It’s a great question, because complexity is associated with IT and why we'll do it differently this time. I see two things happening right now that give us a great shot at this.

One is purely on expectations. All of the opportunities we have as consumers to work with cloud computing models have opened up our imagination as to what we should expect out of IT and computing datacenters, where we can sign up for things immediately, get things when we want them, and pay for what we use. All those great concepts have set our expectations differently.

A good shot

Simultaneously, a lot of changes on the technology side give us a good shot at implementing it. When you combine technology that we'll talk about with the loosened-up imagination on what can be, we're in a great spot to deliver the software-defined datacenter.

Gardner: You mentioned cloud and this notion that it’s a liberating influence. Is this coming from the technologists or from the business side? Is there a commingling on that concept quite yet?

Herrod: It’s funny. I see it coming from the business side, which is the expectation of an individual business unit launching a product. They now have alternatives to their own IT department. They could go sign up for some sort of compute service or software-as-a-service (SaaS) application. They have choices and alternatives to circumvent IT. That's an option they didn't have in the past.

Fundamentally, it comes down to each of us as individuals and our expectations. People are listening to this podcast when they want to, quickly downloading it. This also applies to signing up for email, watching movies, and buying an app on an app store. It's just expected now that you can do things far more agilely, far more quickly than you could in the past, and that's really the big difference.

Gardner: Steve, folks are getting higher expectations based on what they encounter on their consumer side of technology consumption. We see what the datacenters are capable of from the likes of Google and Facebook. Is it possible for enterprises to also project that sort of productivity and performance onto what they're doing, and maybe now that we've gone through an iteration of these vast datacenters, to do it even better?

Herrod: I have a lot of friends at Facebook, Zynga, and Google, running the datacenters there, and what’s exciting for me is that they have built a fully software-defined datacenter. They're doing a lot of the things we are talking about here. But there are two unique things about their datacenters.

When you go into the business world, they don't have legions of people to run the infrastructure.

One is that they have hundreds or even thousands of PhDs who are running this infrastructure. Second, they're running it for a very specific type of application. To run on the Google datacenter, you write your applications a very specific way, which is great for them. But when you go into the business world, they don't have legions of people to run the infrastructure, and they also have a broad set of applications that they can’t possibly consider rewriting.

So in many ways, I see what we're doing is taking the lesson learned in those software-defined datacenters, but bringing it to the masses, and bringing it to companies to run all of their applications and without all of the people cost that they might need otherwise.

Gardner: Let’s step back for some context. How did we get here? It seems that hardware has been sort of the cutting edge of productivity, when we think of Moore’s Law and we look at the way that storage, networks, and server architecture have come together to give us the speeds and feeds that have led to a lot of what we take for granted now. Let’s go through that a little bit and think about why we're at a point where that might not be the case anymore.

Herrod: I like to look at how we got to where we are. I think that's the key to understanding where we're likely to go from here.

History of IT decisions

We started VMware out of a university, where we could take the time to study history and look at what had happened. I liked looking at existing datacenters. You can look through the datacenter and see the history of IT decisions of the past.

It's traditionally been the case that a particular new need led the IT department to go out and buy the right infrastructure for that new need, whether it’s batch processing, client/server applications, or big web farms. But these individually made decisions ended up creating the silos that we all know about that exist all over datacenters.

They now have the group that manages the mainframe, the UNIX administration group, and the client PC group, and none of them is using common people or common tools as much as they certainly would like to. How we got to where we are were isolated decisions for the right thing at the right time, without recognizing the opportunity to optimize across a broader set of the datacenter.

The whole concept of software-defined datacenters is looking holistically at all of the different resources you have and making them equally accessible to a lot of different application types.

Gardner: Earlier, I used the metaphor of an onion. You peel back complexity and you get more. But when it comes to the architecture of datacenters, it seems that the right comparison might be a snowball, which is layered on another layer, or it has been rolling and gathering as it goes, but not rationalized, not looked at holistically.

Every single day you hear about a new case where a business unit or an employee is able to circumvent IT to scratch the itch they have for some particular type of technology.

Are there some sorts of imperatives now that are driving people to do that? We talked about the cloud vision, but maybe it’s security, maybe it’s the economics, maybe it’s the energy issues, or maybe it's all those things together.

Herrod: It’s a little of each. First of all, I like the onion analogy, because it makes you cry, and I think that’s also key. But it’s a combination of requirements coming in at the same time that's really causing people to look at it.

Going back to the original discussion, it starts with the fact that there are choices now. Every single day you hear about a new case where a business unit or an employee is able to circumvent IT to scratch the itch they have for some particular type of technology, whether it's using Dropbox instead of the file servers that the company has, buying their own device and bringing it in, or just signing up for Amazon EC2, instead of using their local datacenter. These are all examples of them being able to go around IT.

But what often happens subsequently is that, when a security problem happens, when you realize that you are not in compliance, IT is left holding the bag. So we get an environment here where the user demand can be handled other ways, but IT has to be able to compete with those.

We have to let IT be a service provider and be able to be as responsive with those, so that they can avoid people going around them. But they still need to be responsible to the business when it comes time to show that Sarbanes-Oxley (SOX) compliance is appropriate or to make sure that your customer records aren’t leaked out to everyone else on the Internet.

That unique balance between the user choice and IT control is something we've all seen over the last several decades, and it’s showing up again at an even larger state.

New competition

Gardner: As you pointed out, Steve, IT isn’t just competing against itself. That is to say, maybe a 5 percent or 10 percent improvement over how well it did last year will be viewed as very progressive. But they're competing now against other datacenter architects. Maybe it’s a SaaS provider, maybe it’s a cloud provider, maybe it’s managed service provider (MSP) or telco that's now offering additional services.

We're really up against this notion that if you don’t architect your datacenter with that holistic software-defined mentality, and someone else does that, you're in trouble.

Herrod: It’s a great point. There are rate cards now for what you can use something else for. You might pay 7 cents per hour for this, or "this much" per transaction. IT departments in general have not traditionally had a good way of, first, even knowing how much they are costing, but second, optimizing to be competitive. So there's this awareness now of how much I'm spending and how long it takes. These metrics are causing this.

Gardner: Let’s revisit the context and the history here, looking at virtualization in particular. We've seen it extend beyond servers to data, storage, and also networking. Is this part of what you've got in your vision of software defined? Is it strictly virtualization, or does it encompass more? Help me understand how you've progressed in your thinking along these lines, particularly in regard to virtualization?

Herrod: We'll step back a little bit. VMware, over the last 13 years or so, has done a very good job of completely optimizing how servers are used in the datacenter. You can provision a new virtual machine (VM) in seconds. The cost has gone down in orders of magnitude. We've really done a good job on the compute and memory aspect of a datacenter.

It's absolutely crucial to look at the breadth of things that are involved in the datacenter.

But as you said, a couple of things have to happen from there. It's absolutely crucial to look at the breadth of things that are involved in the datacenter. We talk to customers now, and often they say, "Great, you've just lowered the cost and time taken to provision a new server. But when I put this in production, by the way, I care what LUN it ends up on, I have to look at what VLAN is there, and if it's in the right section of my firewall setup."

It might take seconds to provision a VM, but then it takes five days to get the rest of the solutions around it. So we see, first of all, the need to get the entire datacenter to be as flexible and fast moving as the pure server components are right now.

Again, if you look at the last couple of years, I would rate the industry -- ourselves and others -- as moving forward quite well on the storage side of things. There are still some things to do for sure, but storage, for the most part, has gotten a good head start on being fully virtualized and automated.

The big buzz around the industry right now has been the recognition that the network is the huge remaining barrier to doing what you want in your datacenter. Plenty of startups and all kinds of folks are working on software-defined networking. In fact, that's what we use as the term for the software-defined datacenter, because as networking follows as this big inhibitor, you'll be opened up to having a truly planned datacenter solution in place.

Now, we can break that down a little bit. It's important to talk about the technology piece of this. But when I say software-defined, I really look at three phases of how software comes in and morphs this existing hardware that you have.

The first step

The first step is to abstract away what people are trying to use from how it is being implemented. That's the core of what virtual even means, separating the logical from the physical. It gives you hardware independence. It enables basic mobility and all sorts of other good things.

The second phase is when you then pool all of these abstracted resources into what we call resource pools. Anyone who uses VMware software knows that we create these great clusters of computing horsepower and we allow vMotion and mobility within it.

But you need to think about that same notion of aggregation of resources at the storage and networking levels, so they become this great pool of horsepower that you can then dole out quite effectively. So after you've abstracted and pooled, the final phase is how you now automate the handling of this. This is where the real savings and speed come from.

Once you have pools of resources, when a new request comes in, you should be able to allocate storage, security, networking, and CPU very quickly. Likewise, when it goes away, you should be able to remove it and put it back into the pool.

That's a bit of a mouthful, but that's how I see the expansion. It first goes from just compute into storage, networking, security, and the other parts of the datacenter. Then simultaneously, you're abstracting each of these resources, pooling them, and then automating them.

When a new request comes in, you should be able to allocate storage, security, networking, and CPU very quickly.

Gardner: What's really fascinating to me are the benefits you get by abstracting to a virtualization and software-defined level -- the ability to implement with greater ease -- but that comes with underlying benefits around operations and management.

It seems to me that you can start to dial up and down, demonstrate elasticity at a far greater level, almost at that data-center level, looking at the service-level agreements (SLAs) and the key performance indicators (KPIs) that you need to adhere to and defining your datacenter success through a business metric, like an SLA.

Does it ring true with you that we're talking about some real management and operational efficiencies, as well as implementation efficiencies?

Herrod: It is, Dana, and we talk about it a few different ways. The transformation of datacenters, as we got started, was all about cost savings and capital expenses in financial terms. Let's buy fewer servers. "Let's not build another datacenter."

Get the latest announcements about VMware's cloud strategy and solutions by tuning into VMware NOW, the new online destination for breaking news, product announcements, videos, and demos at: http://vmware.com/go/now.

But the second phase, and where most customers are today, is all about operational efficiency. Not only am I buying less hardware, but I can do things where I'm actually able to satisfy, as you said, the KPIs or the SLAs.

Doing even more

I can make sure that applications are up and running with the level of availability they expect, with less effort, with fewer people, and with easier tools. And when you go from capital expense savings to operational improvements, you impact the ability for IT to do even more.

To take that one level further, whenever I hear people talk about cloud computing -- and everyone talks about this with all sorts of different impressions in mind -- I think of cloud as simply being about more speed. You can do something more quickly. You can expand something more quickly. And that's what this third phase after capital and operational savings is about, that agility to move faster.

As businesses’ success ties so closely to how IT does, the ability to move faster becomes your strategic weapon against someone else. Very core to all this is how can we operate more efficiently, while satisfying the specific needs of applications in this new datacenter.

Gardner: Another area that I hear about benefiting from this software defined datacenter is the ability to better reduce and manage risk, particularly around security issues. You're no longer dealing with multiple parties, like the group overseeing UNIX, the group overseeing PC, the group doing the x86 architectures. The likelihood for process cracks to develop and security issues to unfortunately crop up seem to be more likely under those circumstances.

But when you have got a more organized overview of management operations and architecting at a similar level, you can instantiate the best practices around security. Please address this issue of security as another fruit to be harvested from a software-defined datacenter.

Security means a lot of different things, and it has been affected by a number of different aspects.

Herrod: Security means a lot of different things, and it has been affected by a number of different aspects.

First of all, I agree that the more you can have a homogenous platform or a homogenous team working on something, the less variation and process you end up with, exactly as you said, Dana. That can allow you to be more efficient.

This is a replacement for the traditional world of ITIL, where they had to try to create some standard across very different back ends. That's a natural progression for getting rid of some of the human errors that come into problems.

A more foundational thing that I am excited about with the software-defined datacenter is how, rather than security being these physical concepts that are deployed across the datacenter today, you can really think of security logically as wrapping up your application. You can do some pretty interesting new things.

A quick segue on that -- the way most security works in datacenters today is through statically placed appliances, whether they're firewalls, intrusion detection, or something else. Then the onus is on you to fit your application in the right part of the datacenter to get the right level of protection that you have, and hopefully it doesn’t move out of that protection zone.

Follows the application

What we're able to deliver with the software-defined datacenter is a way that security is a trait associated with the application, and it essentially wraps and follows the application around. You've virtualized your firewall and you've built it into the fabric of how you're automating deployments. I see that as a way to change the game on how tight the security can be around an application, as well as making sure it's always around there when you deploy it.

Gardner: For end users the proof is in how they actually consume, relate to, and interact with the applications. Is there something about the applications specifically that the software-defined datacenter brings, a higher level of user productivity benefits? What's really going to be noticeable for the application level to end users?

Herrod: That's a great question. I'm an infrastructure guy, as are probably many people listening here, and it’s easy to forget that infrastructure is simply a means to an end. It's the way that you run applications that ultimately matters. So you have to look at what an application is and what its ideal state looks like. The idea of the software-defined datacenter is to optimize that application experience.

That very quickly translates into how quickly can I get my application from the time I want it until it's running. It dictates how often this application is up, what kind of scale it can handle as more people come in, and how secure it is. Ultimately, it's about the application. I believe the software-defined datacenter is the way to optimize that application experience for all the users.

Gardner: Steve, how about not just repaving cow paths in terms of how we deploy existing types of applications. Is there something inherent in a software-defined datacenter benefit that will work to our advantage on innovative new types of applications?

We are at a point where, depending on who you listen to, about 60 percent of all server applications are running virtual.

They could be for high performance computing, big data and analytics, or even when we go to mobile and we have location services folded into some of the way that applications are served up, and there is sort of a latency sensitive portion to this. Are there new types of apps that will benefit from this software-defined architecture?

Herrod: This is one of the most profound parts, if we get it right. I've been talking about can we collapse the silos that were created. Can we get all of our existing apps onto this common platform? We're doing quite well on that. We are at a point where, depending on who you listen to, about 60 percent of all server applications are running virtual, which is pretty amazing. But that also means there is 40 percent that aren’t. So I spend a lot of time understanding why they might not be today.

Part of it is that just as businesses get more comfortable and get there, their business critical apps will get onto the system, and that's working well. But there are applications that are emerging, as you talked about, where if we're not careful, they'll create the next generation of silos that we'll be talking about 10 years from now.

I see this all the time. I'll visit a company that has a purely virtualized pool, but they have also created their grid for doing some sort of Monte Carlo simulations or high-performance computing. Or they have virtualized everything except for their unified communication environment, which has a special team and hardware allocated to it.

We spend quite a bit of time right now looking at the impediments to having those run on top of virtualization, which might be performance related or something else. Then going beyond impediments to how can we make them even better when they are run on top of the virtualized platform.

Great applications

Some of the really interesting things we're able to show now with our partners are things I would have never dreamed of as great candidates when we started the company. But we're able to satisfy very strict real-time requirements, which means we can run some great applications used in various sorts of stock trading, but also used in things like voice over IP (VoIP) or video conferencing.

Another big area that's liable to create the next round of silos, if we're not careful, is the big data and Hadoop world. Lots of customers are kicking the tires and creating special clusters and teams to work on that. But just recently, we've shown that the performance of Hadoop on top of vSphere, our virtualization platform, can be great.

We can even show that we can make it far easier to set up. We can make Hadoop more available, meaning it won’t crash as often. And we can even do things where we make it more elastic than it already is. It can suck up as many resources in the software-defined datacenter as it wants, when it needs them, but it can also give them all back when it's not using them.

It’s really exciting to look across all these apps. At this point, I don’t see a reason why we can't get almost any type app that we're looking at today to fit into the software-defined datacenter model.

Gardner: That’s exciting, when we don’t have any of the stragglers or large portions of business functions that are cast off. It seems to me that we've reached the capability of mirroring the entire datacenter, whether it’s for purposes of business continuity or disaster recovery (DR), or backup and recovery. It gives us the choice of where to locate these resources, not at the individual server, virtual machine level, or application level, but really to move the whole darn datacenter, if that’s important, without a penalty.

Very rapidly, this notion of DR has been a driving reason for people to virtualize their datacenter.

For our last blue-sky direction with this conversation, are we at the point where we have fungibility, if you will, of datacenters, or are we getting to that point in the near future, where we can decide at a moment’s notice where we're going to actually put our datacenter, almost location independent?

Herrod: It’s a ways out, before we're just casually moving datacenters around, for sure. But I have seen some use cases today that are showing what's possible, and maybe I'll just give you a couple of examples.

DR has long been one of the real pains for IT to deal with. They have to replicate things across the country and keep two datacenters completely in sync, literally the same hardware, the same firmware layer, and all of that that goes into it.

Very rapidly, this notion of DR has been a driving reason for people to virtualize their datacenter. We have seen many cases now, where you're able to failover your entire datacenter, effectively copying the whole datacenter over to another one, keeping the logical constructs in place, but hosting in a completely different area.

To get that right, your storage needs to be moved, your network identities need to be updated, and those are things that you can script and do in an automated way, once you've virtualized the whole datacenter.

Fun example

Another really fun example I see more and more now is, as mergers and acquisitions happen, we've seen several cases where one company buys another. They both had fully virtualized their datacenter and they could put on a giant storage drive the datacenter at one company and begin to bring it up on the other side, once they copied it over there.

So the entire datacenter isn't moved yet, but I think there are clear indications of once you separate out where something runs and how it runs from what you are really after, it opens up the door for a lot of different optimizations.

Gardner: We're coming up on the end of our time, but we also have the big annual VMworld show in San Francisco coming up toward the end of August. I know you can’t pre-announce anything, but perhaps you can give us some themes. We've talked about a lot of things here today, but is there any particular themes that we have hit on that you think are going to be more impactful or more important in terms of what we should expect at VMworld?

Herrod: It will be exciting as always. We have more than 20,000 people expected. What I'm doing here is talking about a vision and generalities of what's happening, but you can certainly imagine that what we will be showing there will be the realities -- the products that prove this, the partnerships that are in place that can help bring it forward, and even some use cases and some success stories.

You need to get to the point where you are leveraging the full automation and mobility that exists today.

So expect it to be certainly giving more detail around this vision and making it very real with announcements and demonstrations.

Gardner: Last question, if I'm a listener here today, I'm intrigued, and I want to start thinking about the datacenter at the software-defined level in order to generate some of the benefits that we have been discussing and some of the vision that we have been painting, what’s a good way to start? How do you begin this process? What are a few foundational directives or directions that you recommend?

Herrod: I think it can sound very, very disruptive to create a new software-defined datacenter, but one of the biggest things that I have been excited about in this technology versus others is that there are a set of steps that you go through, where you're able to get some value along the way, but they are also marching you toward where you ultimately end up.

So to customers who are doing this, presumably most of you have done some basic virtualization, but really you need to get to the point where you are leveraging the full automation and mobility that exists today.

Once you start doing that, you'll find that it obviously is showing you where things can head. But it also changes some of the processes you use at the company, some of the organizational structures that you have there, and you can start to pave the way for the overall datacenter to be virtualized, as you take some of these initial steps.

It’s actually very easy to get started. You can make benefits along the way. Your existing applications and hardware work. So that would be my real entreaty -- use what exists today and get your feet wet, as we deliver the next round heading forward.

Gardner: We've been talking about the intriguing concept of the software-defined datacenter and we've been exploring how advances in datacenter technologies and architectural benefits that are being driven through software innovation can provide a number of technological and business benefits.

Please join me now in thanking our guest, Steve Herrod, Chief Technology Officer and Senior Vice President of Research & Development at VMware. Thanks so much, Steve.

Herrod: Great. I've enjoyed the time, Dana. Thanks.

Gardner: My pleasure. This is Dana Gardner, Principal Analyst at Interarbor Solutions. Thanks also to our audience for reading and listening to our discussion, and don't forget to come back next time for the next edition of BriefingsDirect.

Get the latest announcements about VMware's cloud strategy and solutions by tuning into VMware NOW, the new online destination for breaking news, product announcements, videos, and demos at: http://vmware.com/go/now.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: VMware.

Transcript of a BriefingsDirect podcast on how pervasive software enablement helps battle IT datacenter complexity. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

You may also be interested in:

• Ocean Observatories Initiative: Cloud and Big Data come together to give scientists unprecedented access to essential climate insights
  
• Case Study: Strategic Approach to Disaster Recovery and Data Lifecycle Management Pays Off for Australia's SAI Global
• Columbia Sportswear extends deep server virtualization to improved ERP operations, disaster recovery efficiencies
  
• Virtualization Simplifies Disaster Recovery for Insurance Broker Myron Steves While Delivering Efficiency and Agility Gains Too
• SAP Runs VMware to Provision Virtual Machines to Support Complex Training Courses
• Case Study: How SEGA Europe Uses VMware to Standardize Cloud Environment for Globally Distributed Game Development

New Levels of Automation and Precision Needed to Optimize Backup and Recovery in Virtualized Environments

Transcript of a BriefingsDirect podcast on the relationship between increased virtualization and the need for data backup and recovery.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: Quest Software.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you're listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on the relationship between increasingly higher levels of virtualization and the need for new data backup and recovery strategies.

We'll examine how the era of major portions of servers now being virtualized, has provided an on-ramp to attaining data lifecycle benefits and efficiencies. And at the same time, these advances are helping to manage complex data environments that consist of both physical and virtual systems.

What's more, the elevation of data to the lifecycle efficiency level is also forcing a rethinking of the culture of data, of who owns data, and when, and who is responsible for managing it in a total lifecycle across all applications and uses.

This is different from the previous and current system where it’s often a fragmented approach, with different oversight for data across far-flung instances and uses.

Lastly, our discussion focuses on bringing new levels of automation and precision to the task of solving data complexity, and of making always-attainable data the most powerful asset that IT can deliver to the business.

Here to share insights on where the data availability market is going and how new techniques are being adopted to make the value of data ever greater, we're joined by John Maxwell, Vice President of Product Management for Data Protection, at Quest Software. Welcome back, John. [Disclosure: Quest Software is a sponsor of BriefingsDirect podcasts.]

John Maxwell: Hi, Dana. Thanks. It’s great to be here to talk on a subject that's near and dear to my heart.

Gardner: Let’s start at a high level. Why have virtualization and server virtualization become a catalyst to data modernization? Is this an unintended development or is this something that’s a natural evolution?

Maxwell: I think it’s a natural evolution, and I don’t think it was even intended on the part of the two major hypervisor vendors, VMware and Microsoft with their Hyper-V. As we know, 5 or 10 years ago, virtualization was touted as a means to control IT costs and make better use of servers.

Utilization was in single digits, and with virtualization you could get it much higher. But the rampant success of virtualization impacted storage and the I/O where you store the data.

Upped the ante

If you look at the announcements that VMware did around vSphere 5, around storage, and the recent launch of Windows Server 2012, Hyper-V, where Microsoft even upped the ante and added support for Fibre Channel with their hypervisor, storage is at the center of the virtualization topic right now.

It brings a lot of opportunities to IT. Now, you can separate some of the choices you make, whether it has to do with the vendors that you choose or the types of storage, network-attached storage (NAS), shared storage and so forth. You can also make the storage a lot more economical with thin disk provisioning, for example.

There are a lot of opportunities out there that are going to allow companies to make better utilization of their storage just as they've done with their servers. It’s going to allow them to implement new technologies without necessarily having to go out and buy expensive proprietary hardware.

From our perspective, the richness of what the hypervisor vendors are providing in the form of APIs, new utilities, and things that we can call on and utilize, means there are a lot of really neat things we can do to protect data. Those didn't exist in a physical environment.

It’s really good news overall. Again, the hypervisor vendors are focusing on storage and so are companies like Quest, when it comes to protecting that data.

Gardner: As we move towards that mixed environment, what is it about data that, at a high level, people need to think differently about? Is there a shift in the concept of data, when we move to virtualization at this level?

First of all, people shouldn’t get too complacent.

Maxwell: First of all, people shouldn’t get too complacent. We've seen people load up virtual disks, and one of the areas of focus at Quest, separate from data protection, is in the area of performance monitoring. That's why we have tools that allow you to drill down and optimize your virtual environment from the virtual disks and how they're laid out on the physical disks.

And even hypervisor vendors -- I'm going to point back to Microsoft with Windows Server 2012 -- are doing things to alleviate some of the performance problems people are going to have. At face value, your virtual disk environment looks very simple, but sometimes you don’t set it up or it’s not allocated for optimal performance or even recoverability.

There's a lot of education going on. The hypervisor vendors, and certainly vendors like Quest, are stepping up to help IT understand how these logical virtual disks are laid out and how to best utilize them.

Gardner: It’s coming around to the notion that when you set up your data and storage, you need to think not just for the moment for the application demands, but how that data is going to be utilized, backed up, recovered, and made available. Do you think that there's a larger mentality that needs to go into data earlier on and by individuals who hadn’t been tasked with that sort of thought before?

See it both ways

Maxwell: I can see it both ways. At face value, virtualization makes it really easy to go out and allocate as many disks as you want. Vendors like Quest have put in place solutions that make it so that within a couple of mouse clicks, you can expose your environment, all your virtual machines (VMs) that are out there, and protect them pretty much instantaneously.

From that aspect, I don't think there needs to be a lot of thought, as there was back in the physical days, of how you had to allocate storage for availability. A lot of it can be taken care of automatically, if you have the right software in place.

That said, a lot of people may have set themselves up, if they haven’t thought of disaster recovery (DR), for example. When I say DR, I also mean failover of VMs and the like, as far as how they could set up an environment where they could ensure availability of mission-critical applications.

For example, you wouldn’t want to put everything, all of your logical volumes, all your virtual volumes, on the same physical disk array. You might want to spread them out, or you might want to have the capabilities of replicating between different hypervisor, physical servers, or arrays.

Gardner: I understand that you've conducted a survey to try to find out more about where the market is going and what the perceptions are in the market. Perhaps you could tell us a bit about the survey and some of the major findings.

Our survey showed that 70 percent of organizations now consider at least 50 percent of their data mission critical.

Maxwell: One of the findings that I find most striking, since I have been following this for the past decade, is that our survey showed that 70 percent of organizations now consider at least 50 percent of their data mission critical.

That may sound ambiguous at first, because what is mission critical? But from the context of recoverability, that generally means data that has to be recovered in less than an hour and/or has to be recovered within an hour from a recovery-point perspective.

This means that if I have a database, I can’t go back 24 hours. The least amount of time that I can go back is within an hour of losing data, and in some cases, you can’t go back even a second. But it really gets into that window.

I remember in the days of the mainframe, you'd say, "Well, it will take all day to restore this data, because you have tens or hundreds of tapes to do it." Today, people expect everything to be back in minutes or seconds.

The other thing that was interesting from the survey is that one-third of IT departments were approached by their management in the past 12 months to increase the speed of the recovery time. That really dovetails with the 50 percent of data being mission critical. So there's pressure on the IT staff now to deliver better service-level agreements (SLAs) within their company with respect to recovering data.

Terms are synonymous

The other thing that's interesting is that data protection and the term backup are synonymous. It's funny. We always talk about backup, but we don't necessarily talk about recovery. Something that really stands out now from the survey is that recovery or recoverability has become a concern.

Case in point: 73 percent of respondents, or roughly three quarters, now consider recovering lost or corrupted data and restoring those mission critical applications their top data-protection concern. Only 4 percent consider the backup window the top concern. Ten years ago, all we talked about was backup windows and speed of backup. Now, only 4 percent considered backup itself, or the backup window, their top concern.

So 73 percent are concerned about the recovery window, only 4 percent about the backup window, and only 23 percent consider the ability to recover data independent of the application their top concerns.

Those trends really show that there is a need. The beauty is that, in my opinion, we can get those service levels tighter in virtualized environments easier than we can in physical environments.

Gardner: We seem to have these large shifts in the market, one around virtualization of servers and storage and the implications of first mixed, and then perhaps a majority, or vast majority, of virtualized environments.

A company has to look at which policies or which solutions to put in place to address the criticality of data, but then there is a cost associated with it.

The second shift is the heightened requirements around higher levels of mission-critical allocation or designation for the data and then the need for much greater speed in recovering it.

Let's unpack that a little bit. How do these fit together? What's the relationship between moving towards higher levels of virtualization and being able to perhaps deliver on these requirements, and maybe even doing it with some economic benefit?

Maxwell: You have to look at a concept that we call tiered recovery. That's driven by the importance now of replication in addition to traditional backup, and new technology such as continuous data protection and snapshots.

That gets to what I was mentioning earlier. Data protection and backup are synonymous, but it's a generic term. A company has to look at which policies or which solutions to put in place to address the criticality of data, but then there is a cost associated with it.

For example, it's really easy to say, "I'm going to mirror 100 percent of my data," or "I'm going to do synchronous replication of my data," but that would be very expensive from a cost perspective. In fact, it would probably be just about unattainable for most IT organizations.

Categorize your data

What you have to do is understand and categorize your data, and that's one of the focuses of Quest. We're introducing something this year called NetVault Extended Architecture (NetVault XA), which will allow you to protect your data based on policies, based on the importance of that data, and apply the correct solution, whether it's replication, continuous data protection, traditional backup, snapshots, or a combination.

You can't just do this blindly. You have got to understand what your data is. IT has to understand the business, and what's critical, and choose the right solution for it.

Gardner: It's interesting to me that if we're looking at data and trying to present policies on it, based on its importance, these policies are going to be probably dynamic and perhaps the requirements for the data will be shifting as well. This gets to that area I mentioned earlier about the culture around data, thinking about it differently, perhaps changing who is responsible and how.

So when we move to this level of meeting our requirements that are increasing, dealing in the virtualization arena, when we need to now think of data in perhaps that dynamic fluid sense of importance and then applying fit-for-purpose levels of support, backup, recoverability, and so forth, whose job is that? How does that impact how the culture of data has been and maybe give us some hints of what it should be?

Maxwell: You've pointed out something very interesting, especially in the area of virtualization, just as we have noticed over the seven years of our vRanger product, which invented the backup market for virtualized environments.

What we see now are the traditional people who were responsible for physical storage taking over the responsibility of virtual storage.

It used to be, and it still is in some cases, that the virtual environment was protected by the person, usually the sys admin, who was responsible for, in the case of VMware, the ESXi hypervisors. They may not necessarily have been aligned with the storage management team within IT that was responsible for all storage and more traditional backups.

What we see now are the traditional people who were responsible for physical storage taking over the responsibility of virtual storage. So it's not this thing that’s sitting over on the side and someone else does it. As I said earlier, virtualization is now such a large part of all the data, that now it's moving from being a niche to something that’s mainstream. Those people now are going to put more discipline on the virtual data, just as they did the physical.

Because of the mission criticality of data, they're going from being people who looked at data as just a bunch of volumes or arrays, logical unit numbers (LUNs), to "these are the applications and this is the service level associated with the applications."

When they go to set up policies, they are not just thinking of, "I'm backing up a server" or "I'm backing up disk arrays,", but rather, "I'm backing up Oracle Financials," "I'm backing up SAP," or "I'm backing up some in-house human resources application."

Adjust the policy

And the beauty of where Quest is going is, what if those rules change? Instead of having to remember all the different disk arrays and servers that are associated with that, say the Oracle Financials, I can go in and adjust the policy that's associated with all of that data that makes up Oracle Financials. I can fine-tune how I am going to protect that and the recoverability of the data.

Gardner: That to me brings up the issue about ease of use, administration, interfaces, making these tools something that can be used by more people or a different type of person. How do we look at this shift and think about extending that policy-driven and dynamic environment at the practical level of use?

Maxwell: It's interesting that you bring that up too, because we've had many discussions about that here at Quest. I don't want to use the term consumerization of IT, because it has been used almost too much, but what we're looking at is, with the increased amount of virtual data out there, which just adds to the whole pot of heterogeneous environments, whether you have Windows and Linux, MySQL, Oracle, or Exchange, it's impossible for these people who are responsible for the protection and the recoverability of data to have the skills needed to know each one of those apps.

We want to make it as easy to back up and recover a database as it is a flat file. The fine line that we walk is that we don't want to dumb the product down. We want to provide intuitive GUIs, a user experience that is a couple of clicks away to say, "Here is a database associated with the application. What point do I want to recover to?" and recover it.

If there needs to be some more hands-on or more complicated things that need to be done, we can expose features to maybe the database administrator (DBA), who can then use the product to do more complex recovery or something to that effect.

It's impossible for these people who are responsible for the protection and the recoverability of data to have the skills needed to know each one of those apps.

We've got to make it easy for this generalist, no matter what hypervisor -- Hyper-V or VMware, a combination of both, or even KVM or Xen -- which database, which operating system, or which platform.

Again, they're responsible for everything. They're setting the policies, and they shouldn't have to be qualified. They shouldn't have to be an Exchange administrator, an Oracle DBA, or a Linux systems administrator to be able to recover this data.

We're going to do that in a nice pretty package. Today, there are many people here at Quest who walk around with a tablet PC as much as they do with their laptop. So our next-generation user interface (UI) around NetVault XA is being designed with a tablet computing scenario, where you can swipe data, and your toolbar is on the left and right, as if you are holding it using your thumb -- that type of thing.

Gardner: So, it's more access when it comes to the endpoint, and as we move towards supporting more of these point applications and data types with automation and a policy-driven approach or an architecture, that also says to me that we are elevating this to the strategic level. We're looking at data protection as a concept holistically, not point by point, not source by source and so forth.

Again, it seems that we have these forces in the market, virtualization, the need for faster recovery times, dealing with larger sets of data. That’s pushing us, whether we want to or even are aware of it, towards this level of a holistic or strategic approach to data.

Let me just see if you have any examples, at this point, of companies that are doing this and what it's doing for them. How are they enjoying the benefits of elevating this to that strategic or architecture level?

Exabyte of data

Maxwell: We have one customer, and I won't mention their name, but they are one of the top five web properties in the world, and they have an exabyte of data. Their incremental backups are almost 500 petabytes, and they have an SLA with management that says 96 percent of backups will run well, because they have so much data that changes in a week’s time.

You can't miss a backup, because that gets to the recoverability of the application. They're using our NetVault product to back up that data, using both traditional methods and integrated snapshots. Snapshot was on the technology tier as far as having tiered recovery scenario. They used NetVault in conjunction with hardware snapshots, where there is no backup window. The backup to the application is, for all practical purposes, instantaneous.

Then, they use NetVault to manage and even take that data that’s on disk and eventually move it to tape. The snapshots allow them to do that very quickly for massive amounts of data. And by massive amounts of data, I'm talking 100 million files associated with one application. To put that back in place at any point in time very quickly with NetVault orchestrating that hardware snapshot technology, that’s pretty mind blowing.

Gardner: That does give us a sense of the scale and complexity and how it's being managed and delivered.

You mentioned how Quest is moving towards policy-driven approaches, improving UIs, and extending those UIs to mobile tier. Are there any other technology approaches that Quest is involved with that further explain how some of these challenges can be met? I'm very interested in agentless, and I'm also looking at how that automation gets extended across more of these environments.

We're envisioning customer environments where they're going to have multiple hypervisors, just as today people have multiple operating system databases.

Maxwell: There are two things I want to mention. Today, Quest protects VMware and Microsoft Hyper-V environments, and we'll be expanding the hypervisors that we're supporting over the next 12 months. Certainly, there are going to be a lot of changes around Windows Server 2012 or Hyper-V, where Microsoft has certainly made it a lot more robust.

There are a lot more things for us exploit, because we're envisioning customer environments where they're going to have multiple hypervisors, just as today people have multiple operating system databases.

We want to take care of that, mask some complexity and allow people to possibly have cross-hypervisor recoverability. So, in other words, we want to enable safe failover of a VMware ESXi system to Microsoft Hyper-V, or vice versa..

There's another thing that’s interesting and is a challenge for us and it's something that has challenged engineers here at Quest. This gets into the concepts of how you back up or protect data differently in virtual environments. Our vRanger product is the market leader with more than 40,000 customers, and it’s completely agentless.

As we have evolved the product over the past seven years, we've had three generations of the product and have exploited various APIs. But with vRanger, we've now gone to what is called a virtual appliance architecture. We have a vRanger service that performs backup and replication for one or hundreds of VMs that exist either on that one physical server or in a virtual cluster. So this VM can even protect VMs that exist on other hardware.

Scalability

The beauty of this is first the scalability. I have one software app that’s running that’s highly controllable. You can control what resources are replicating, protecting, and recovering all of my VMs. So that’s easy to manage, versus having to have an agent installed in every one of those VMs.

Two, there's no overhead. The VMs don’t even know, in most cases, that a backup is occurring. We use the services, in the case of VMware, of ESXi, that allows us to go out there, snapshot the virtual volumes called VMDKs, and back up or replicate the data.

Now, there is one thing that we do that’s different than some others. Some vendors do this and some don’t, and I think one of those things you have to look at when you choose a virtual backup or virtual data protection vendor is their technical prowess in this area. If you're backing up a VM that has an application such as Exchange or SharePoint, that’s a live application, and you want to be able to synchronize the hypervisor snapshot with the application that’s running.

There’s a service in Windows called Volume Shadow Copy Service, or VSS for short, and one of the unique things that Quest does with our backup software is synchronize the virtual snapshot of the virtual disks with the application of VSS, so we have a consistent point-in-time backup.

To communicate, we dynamically inject binaries into the VM that do the process and then remove themselves. So, for a very short time, there's something running in that VM, but then it's gone, and that allows us to have consistent backup.

One of the beauties of virtualization is that I can move data without the application being conscious of it happening.

That way, from that one image backup that we've done, I can restore an entire VM, individual files, or in the case of Microsoft Exchange or Microsoft SharePoint, I can recover a mailbox, an item, or a document out of SharePoint.

Gardner: So the more application-aware the solution is, it seems the more ease there is in having this granular level of restore choices. So that's fit for purpose, when it comes to deciding what level of backup and recovery and support for the data lifecycle is required.

This also will be able to fit into some larger trends around moving a data center to a software level or capability. Any thoughts of how what you're doing at Quest fits into this larger data-center trend. It seems to me that it’s at the leading or cutting edge?

Maxwell: One of the beauties of virtualization is that I can move data without the application being conscious of it happening. There's a utility, for example, within VMware called vMotion Storage that allows them to move data from A to B. It's a very easy way to migrate off of an older disk array to a new one, and you never have to bring the app down. It's all software driven within the hypervisor, and it's a lot of control. Basically it’s a seamless process.

What this opens up, though, is the ability for what we're looking at doing at Quest. If there's a means to move data around, why can't I then create an environment where I could do DR, whether it's within the data center for hardware redundancy or whether it's like what we do here at Quest.

Replicate data

We replicate data amongst various Quest facilities. Then, we can bring up an application that was running in location A in point B, on unlike hardware. It can be completely different storage, completely different servers, but since they're VMs, it doesn’t matter.

That kind of flexibility that virtualization brings is going to give every IT organization in the world the type of failover capabilities that used to only exist for the Global 1000, where they used to have to set up a hot site or had to have a data center. They would use very expensive proprietary hardware-based replication and things like that. So you had to have like arrays, like servers, and all that, just to have availability.

Now, with virtualization, it doesn’t matter, and of course, we have plenty of bandwidth, especially here in the United States. So it’s very economical, and this gets back to our survey that showed that for IT organizations, 73 percent were concerned about recovering data, and that’s not just recovering a file or a database.

Here in California, we're always talking about the big one. Well, when the big one happens, whole bunches of server racks may fall over. In the case of Quest, we want to be able to bring those applications up in an environment that's in a different part of the country, with no fault zones and that type of thing, so we can continue our business.

Gardner: We just saw a recent example of unintended or unexpected circumstances with the Mid-Atlantic states and some severe thunderstorms, which caused some significant disruption. So we always need to be thoughtful about the unexpected.

Now, we are talking about actually putting data protection products in the cloud, so you can back up the data locally within the cloud.

Another thing that occurred to me while you were discussing these sort of futuristic scenarios, which I am imagining aren’t that far off, is the impact that cloud computing another big trend in the market, is bringing to the table.

It seems to me that bringing some of the cloud models, cloud providers, service models into play with what you have described also expands what can be done across larger sets of organizations and maybe even subsets of groups within companies. Any thoughts briefly on where some of the cloud provider scenarios might take this?

Maxwell: It’s funny. Two years ago, when people talked about cloud and data protection, it was just considering the cloud as a target. I would back up the cloud or replicate the cloud. Now, we are talking about actually putting data protection products in the cloud, so you can back up the data locally within the cloud and then maybe even replicate it or back it up back to on-prem, which is kind of a novel concept if you think about it.

If you host something up in cloud, you can back it up locally up there and then actually keep a copy on-prem. Also, the cloud is where we're certainly looking at having generic support for being able to do failover into the cloud and working with various service providers where you can pre-provision, for example, VMs out there.

You're replicating data. You sense that you have had a failure, and all you have to do is, via software, bring up those VMs, pointing them at the disk replicas you put up there.

Different cloud providers

Then, there's the concept of what you do if a certain percentage of all your IT apps are hosted in cloud by different cloud providers. Do you want to be able to replicate the data between cloud vendors? Maybe you have data that's hosted at Amazon Web Services. You might want to replicate it to Microsoft Azure or vice versa or you might want to replicate it on-premise (on-prem).

So there's going to be a lot of neat hybrid options. The hybrid cloud is going to be a topic that we're going to talk about a lot now, where you have that mixture of on-prem, off-prem, hosted applications, etc., and we are preparing for that.

Gardner: I'm afraid we're about out of time. You've been listening to a sponsored BriefingsDirect podcast discussion on the relationship between increasingly higher levels of virtualization and the need for new backup and recovery strategies.

We've seen how solving data complexity and availability in the age of high virtualization is making always attainable data the most powerful asset that an IT organization can deliver to its users.

I'd like to thank our guest. We've been joined by John Maxwell, Vice President of Product Management and Data Protection at Quest Software.

The cloud is where we're certainly looking at having generic support for being able to do failover into the cloud.

John, would you like to add anything else, maybe in terms of how organizations typically get started. This does seem like a complex undertaking. It has many different entry points. Are there some best practices you've seen in the market about how to go about this, or at least to get going?

Maxwell: The number one thing is to find a partner. At Quest, we have hundreds of technology partners that can help companies architect a strategy utilizing the Quest data protection solutions.

Again, choose a solution that hits all the key points. In the case of VMware, you can go to VMware’s site and look for VMware Ready-Certified Solutions. Same thing with Microsoft, whether it’s Windows Server 2008 or 2012 certified. Make sure that you are getting a solution that’s truly certified. A lot of products say they support virtual environments, but then they don’t have that real certification, and a result, they can’t do lot of the innovative things that I’ve been talking about .

So find a partner who can help, or, we at Quest can certainly help you find someone who can help you architect your environment and even implement the software for you, if you so choose. Then, choose a solution that is blessed by the appropriate vendor and has passed their certification process.

Gardner: I should also point out that VMworld is coming up next week. I expect that you'll probably have a big presence there, and a lot of the information that we have been talking about will be available in more detail through the VMworld venue or event.

Maxwell: Absolutely, Dana. Quest will have a massive presence at VMworld, both in San Francisco and Barcelona. We'll be demonstrating technologies we have today and also we will be making some major announcements and previewing some real exciting software at the show.

Gardner: Well, great. This is Dana Gardner, Principal Analyst at Interarbor Solutions. I'd like to thank our audience for listening, and invite them to come back next time.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: Quest Software.

Transcript of a BriefingsDirect podcast on the relationship between increased virtualization and the need for data backup and recovery. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

You may also be interested in:

• Data Explosion and Big Data Demand New Strategies for Data Management, Backup and Recovery, Say Experts
  
• Big Data and a Brave New World
• Big Data: Crunching the Numbers
• Case Study: Strategic Approach to Disaster Recovery and Data Lifecyce Management Pays off for Australia's SAI Global
• Enterprise Architecture and Enterprise Transformation: Related But Distinct Concepts That Can Change the World
• Capgemini's CTO on Why Cloud Computing Exposes the Duality Between IT and Business