Transcript
of a BriefingsDirect podcast on the future of software-defined storage and how it
will have an impact on storage-hungry technologies, especially VDI.
Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.
Dana Gardner: Hi, this is
Dana Gardner, Principal Analyst at
Interarbor Solutions, and you're listening to
BriefingsDirect.
Our
latest podcast explores how one of the most costly and complex parts of
any enterprises IT infrastructure -- storage -- is being dramatically
improved by the accelerating adoption of
software-defined storage.
The ability to choose low-cost hardware, to manage across different
types of storage, and radically simplified data storage via intelligent
automation means a virtual rewriting of the economics of data.
But
just as IT leaders seek to simultaneously tackle storage pain points of
scalability, availability, agility, and cost -- software-defined storage
is also providing significant strategic- and architectural-level
benefits.
We're here now with two executives from
VMware to unpack these efficiencies and examine the broad innovation behind the rush to exploit software-defined storage. Please join me now in welcoming our guests,
Alberto Farronato, the Director of Product Marketing for Cloud Infrastructure Storage and Availability at VMware. Hello, Alberto.
Alberto Farronato: Hello, Dana. Glad to be here, thanks.
Gardner: We're also here with
Christos Karamanolis, Chief Architect and a Principal Engineer in the Storage and
Availability Engineering Organization at VMware. Welcome, Christos.
Christos Karamanolis: Thank you. Glad to be here.
Gardner:
Alberto, we often focus on the speeds and feeds
and the costs -- the hard elements -- when it comes to storage and
modernization of storage. But what about the wider implications?
Software-defined storage is really changing something more fundamental than just data
and economics of data. How do you see the wider implications
of what’s happening now that software-defined storage is becoming more common?
Farronato: Software-defined
storage is certainly about addressing the cost issue of storage, but
more importantly, as you said, it’s also about operations. In fact, the
overarching goal that VMware has is to bring to storage the efficient
operational model that we brought to compute with server virtualization.
So we have a set of initiatives around improving storage on all levels,
and building a parallel evolution of storage to what we did with
compute. We're very excited about what’s coming.
Gardner: Christos, one of my favorite sayings is that "architecture is IT destiny." How you see software-defined storage at that
architectural level? How does it change the game?
Concept of flexibility
Karamanolis:
The fundamental architectural principle behind software-defined storage
is the concept of flexibility. It's the idea of being able to adapt to
different hardware resources, whether those are
magnetic disks,
flash storage, or other types of non-volatile memories in the future.
How
does the end user adapt their storage platform to the needs
they have in terms of the capabilities of the hardware, the ratios of
the different types of storage, the networking, the
CPU resources, and the memory resources needed for executing and providing their service to what's ahead?
That’s
one part of flexibility, but there is another very interesting part,
which is a very acute problem for VMware customers today. Their
operational complexity of provisioning storage for applications and
virtual machines (VMs) has been one way of packaging applications.
Today, customers
virtualize
environments, but also in general have to provision physical storage
containers. They have to anticipate their uses over time and have make
an investment up front in resources that they'll need over a long
period of time. So they create those logical unit number (
LUN) file services, or whatever that is needed, for a period of time that spans anything from weeks to years.
Software-defined
storage advocates a new model, where applications and VMs are
provisioned at the time that the user needs them. The storage resources
that they need are provisioned on-demand, exactly for what the
application and the user needs -- nothing more or less.
The
idea is that you do this in a way that is really intuitive to
the end-user, in a way that reflects the abstractions that user
understands -- applications, the data containers that the applications
need, and the characteristics of the application workloads.
So those two aspects of flexibility are the two fundamental aspects of any software-defined storage.
Gardner:
As we see this increased agility, flexibility, the on-demand
nature of virtualization now coupled with software-defined storage, how are organizations benefiting at a business level? Is there a
marker that we can point to that says, "This is actually changing things
beyond just a technology sphere and into the business sphere?"
Farronato:
There are several benefits and several outcomes of adopting
software-defined storage. The first that I would call out is the ability
to be much more responsive to the business needs -- and the changing
business needs -- in the form of what your application needs faster.
As
Christos was saying, in the old model, you had to guess ahead of time
what the applications will need, spend a lot of time trying to
preconfigure and predetermine the various services levels,
performance, availability and other things that our storage really would
be required by your application, and so spend a lot of time setting things up,
and then hopefully, down the line, consume it the way you thought you
would.
Difficult change management
In
many cases, this causes long provisioning cycles. It causes difficult
change management after you provision the application. You find that you
need to change things around, because either the business needs have
changed or what you guessed was wrong. For example, customers have to
face constant
data migration.
With
the policy-driven approach that Christos has just described -- with the
ability to create these storage services on-the-fly for a policy
approach -- you don’t have to do all that pre-provisioning and
preconfiguring. As you create the VMs and specify the requirements, the
system responds accordingly. When you have to change things, you
just modify the policy and everything in the underlying infrastructure changes accordingly.
Responsiveness, in my opinion,
is the one biggest benefit that IT will deliver to the business by
shifting to software-defined storage. There are many others, but I want
to focus on the most important one.
When you have to change things, you
just modify the policy and everything in the underlying infrastructure
will change accordingly.
Gardner: As
we gain more agility, that prompts more
use of software-defined storage, or in your case, Virtual SAN. With that acceleration of adoption, we begin to see more
beneficial consequences, such as better manageability of data as a lifecycle,
perhaps operations being more responsive to developers so that a
DevOps benefit kicks in.
Can
you explain what happens when software-defined storage becomes strategic at the
applications level, perhaps with implications across the entire data lifecycle?
Karamanolis:
One thing we already see, not only among VMware customers, but as a more
generic trend, is that infrastructure administrators -- the guys who do
the heavy-lifting in the
data centers
day in and day out, who manage much more beyond what is traditionally servers and applications -- are getting more and more
into managing networks and data storage.
Talking
about changing models here, what we see is that tools have to be
developed and software-defined storage is a key technology evolution
behind that. These are tools for those administrators to manage all
those resources that they need to make their day-to-day jobs happen.
Here, software-defined storage is playing a key role. With technology like
Virtual SAN,
we make the management of storage visible for people who are not
necessarily experts in the esoterics of a certain vendor's hardware. It allows more IT professionals to specify the requirements
of their applications.
Then, the software storage
platform can apply those requirements on the fly to provision,
configure, and dynamically monitor and enforce compliance for the policy
and requirements that are specified for the applications. This is a
major shift we see in the IT industry today, and it’s going to be
accelerated by technologies like Virtual SAN.
Gardner: When you go to software-defined storage, you
can get to policy level, automation, and intelligence when it comes to
how you're executing on storage. How does software-defined storage simplify storage overall?
Distributed platform
Karamanolis:
That's an interesting point, because if you think about this
superficially, we’ll now go from a single, monolithic storage entity to a
storage platform that is distributed, controlled by software, and can
span tens or sometimes hundreds of physical nodes and/or entities. Isn’t
complexity harder in the latter case?
The reality is
that whether it's because of necessity or because we've learned a
lot over the last 10 to 15 years about how to manage and control large
distributed systems, that there is a parallel evolution of these ideas of how
you manage your infrastructure, including the management of storage.
The user
has to be exposed to the consequences of the policy they choose. There
is a cost there for every one of those services.
As
we alluded to already, the fundamental model here is that the end user,
the IT professional that manages this infrastructure, expresses in a
descriptive way, what they need for their applications in terms of CPU,
memory, networking, and, in our case, storage.
What do
I mean by descriptive? The IT professional does not need to understand
all the internal details of the technologies or the hardware used at any
point in time, and which may evolve over a period of time.
Instead,
they express at a high level a set of requirements -- we call
them policies -- that capture the requirements of the application. For
example, in the case of storage, they specify the level of availability that is
required for certain applications and performance goals, and they can also specify things like the data protection policies for certain data sets.
Of
course, for all those things, nothing comes for free. So the user
has to be exposed to the consequences of the policy that they choose. There
is a cost there for every one of those services.
But
the key point is that the software platform automatically configures the
appropriate resources, whether they're arrayed across multiple physical
devices, arrayed across the network, or whether they get asynchronous
data as specified in a remote location in order to comply with certain
disaster recovery (DR) policies.
All
those things are done by the software, without the user having to worry
about whether the storage underneath is highly available storage, in
which case they need to be able to create only two copies of the data,
or whether it is of some low-end hardware for which that would require
three or four copies of the data. All those things are determined
automatically by the platform.
This is the new mode.
Perhaps I'm oversimplifying some of these problems, but the idea is that
the user should really not have to know the specific hardware configurations of
a disk array. If the requirements can not be met, it is because these
new technologies are not incorporated into the storage platform.
Policy driven
Farronato: Virtual SAN is a completely policy-driven product, and we call it VM-centric or
application-centric. The whole management paradigm for storage, when you
use Virtual SAN, is predicated around the VM and the policies that you
create and you assign to the VMs as you create your VMs, as you scale
your environment.
One of the great things that you can
achieve with Virtual SAN is providing differentiated service levels to
individual VMs from a single data store. In the past, you had to create
individual LUNs or volumes, assign data services like replication
or
RAID levels to each individual volume, and then map the application to them.
With
Virtual SAN, you're simply going to have a capacity container that
happens to be distributed across a number of nodes in your cluster -- and
everything that happens from that point on is just dropping your VMs into
this container. It automatically instantiates all the data services by
virtue of having built-in intelligence that interprets the requirements
of the policy.
One of the great things that you can
achieve with Virtual SAN is providing differentiated service levels to
individual VMs from a single data store.
That makes this system extremely simple
and intuitive to use. In fact, one of the core design objectives of
Virtual SAN is simplicity. If you look at a short description of the
system, the radically simple
hypervisor-converged storage means bringing that idea of eliminating the complexity of storage to the next level.
Gardner:
We've talked about simplicity, policy driven, automation, and
optimization. It seems to me that those add up very quickly to a
fit-for-purpose approach to storage, so that we are not
under-provisioning or over-provisioning, and that can lead to
significant cost-savings.
So let’s translate this back to economics. Alberto, do you have any thoughts on how we lower
total cost of ownership (TCO) through these SDS approaches of simplicity, optimization, policy driven, and intelligence?
Farronato: There are always two sides of the equation. There
is a
CAPEX and an
OPEX
component. Looking at how a product like Virtual SAN reduces CAPEX,
there are several ways, but I can mention a couple of key components or
drivers.
First, I'd call out the fact that it is an
x86
server-based storage area network (SAN). So it leverages server-side components to deliver shared
storage. By virtue of using server-side resources right off the bat
there are significant savings that you can achieve through lower-cost
hardware components. So the same hard drive or solid-state drive (SSD) that you deploy on a shared
external storage array could be on the order of 80 percent cheaper.
The other aspect that I would call out that
reduces the overall CAPEX cost is more along the lines of this, as you said,
consume on-demand approach or, as we put it in many other terms, grow-as-you-go.
With a scale-out model, you can start with a small deployment and a
small upfront investment.
You can then progressively
scale out as your environment grows by the much finer granularity that
you would with a monolithic array. And as you scale, you scale both
compute, but also
IOPs and that goes hand in hand with often the number of VMs that you are running out of your cluster.
System growth
So the system grows with the size of your
environment, rather than requiring you to buy a lot of resources upfront
that many times remain under-utilized for a long time.
On
the OPEX side, when things become simpler, it means that overall
administration productivity increases. So we expect a trend where
individual administrators will be able to manage a greater amount of capacity,
and to do so in conjunction with management of the virtual infrastructure
to achieve additional benefits.
Gardner:
Christos, Virtual SAN has been in general availability now for several
months, since March 2014, after being
announced last year at VMworld 2013. Now that it’s in place and growing in the
market, are there any unintended benefits or unintended consequences
from that total-cost perspective in real-world day-in, day-out
operations?
The system grows with the size of your
environment, rather than require you to buy a lot of resources upfront
that many times remain under-utilized for a long time.
I'm looking for ways in which a typical
organization is seeing software-defined storage benefiting them
culturally and organizationally in terms of skills, labor, and that sort
of softer metric.
Karamanolis: That’s a very
interesting point. Our technologists sometimes tend to overlook the
cultural shifts that technology causes in the field. In the case of
Virtual SAN, we see a lot of, as one customer put it, being empowered to
manage their own storage, in the vertical that we are controlling in
their IT organization, without having to depend on the centralized
storage organization in this company.
What
we really see here is a shift in paradigm about how our customers use
Virtual SAN today to enable them to have a much faster turnaround for
trying new applications, new workloads, and getting them from test and
dev into production without having to be constrained by the processes
and the timelines that are imposed by a central storage IT organization.
This is a major achievement, and the major tool for VMware administrators in the field,
which we believe is going to lead the way to a much wider adoption of
Virtual SAN and software-defined storage in general.
Gardner:
It sounds as if there's a simultaneous decentralized benefit
here, similar to what we saw 30 years ago in manufacturing.
Back in the day, you used to have an assembly line approach where one
linear process would lead to another, but when you do simultaneous
things, you can see a lot more productivity and innovation.
Do
you think that there is a parallel between software modernization and
manufacturing 30 years ago?
Managing storage
Karamanolis:
Certainly we have a parallel here, taking into account the fact that
the customers, the IT professionals that manage storage, understand the
processes and the workflows without necessarily having to understand the
internals of the technology that implement those workflows.
This
is very much like being part of a production line and understanding the
big picture, but without having to understand all the little details of
every station of that production line. In both cases, you have a
fundamental scalability benefit going down that path.
I
say this this being fully aware that the real world is demanding. I
understand that there may be situations where the IT administrator,
whether a VMware admin or a storage expert, has to go and jump into the
situation and troubleshoot something that is going wrong.
With this approach you have a more granular way to control the service
levels that you deliver to your customers.
He
has to troubleshoot, for example, a performance issue, or understanding
what's happening under the covers when the requirements
specified don’t seem to match what they're getting.
And
what we do is we deliver, together with Virtual SAN in an integrated
fashion, sophisticated monitoring and reporting tools that help
customers not only understand what's happening in their system, but also
do an analysis of any situation end-to-end, all the way from the
application, down to the VM, the hypervisor and the resources the
hypervisor assigns to those VMs, and including the storage resources
that are consumed at any point in time across the cluster.
Those
are the tools that always have to come together with those simple
models we're introducing, because you need to be able to handle those
exceptional situations.
Gardner: How does this simplification and automation have a
governance, risk, and compliance (GRC) benefit?
Farronato:
With this approach you have a more granular way to control the service
levels that you deliver to your customers, to your internal customers,
and a more efficient way to do it by standardizing through polices rather
than trying to standardize service levels over a category of hardware.
Self-service consumption
You
can more easily keep track of what each individual application is
receiving, whether it’s in compliance to that particular policy that you
specified. You can also now enable self-service consumption more easily
and effectively.
We have, as part of our Policy-Based Management Engine,
APIs that will allow for integration with cloud automation frameworks, such as
vCloud Automation Center for
OpenStack, where end users will be able to consume a predefined category of service.
It
will speed up the provisioning process, while at the same time,
enabling IT to maintain that control and visibility that all the admins
want to maintain over how the resources are consumed and allocated.
You can also now enable self-service consumption more easily
and effectively.
Gardner:
I'm interested in hearing more examples about how this is being
used. But before we go to that, there's one questions that I
get a lot as an analyst.
Perhaps it's because people
come from different parts of IT, or they have specializations, but
people say, "We have software-defined storage, we have software-defined
networking, a highly virtualized data center, and the goal is to become a
software-defined data center, but I don't necessarily understand how
these come together in what order. How do I go about that?"
Help us understand the role and impact of
software-defined storage in the context of a larger software-defined
data center.
Karamanolis: This is a challenging
question, and I don’t know how far I can go in answering this. What
we're trying to do at VMware is allow our customers to experience the
various concepts of software-defined data center in a piecemeal
fashion.
They can address the most acute of their
problems, whether those are the traditional computer utilization
questions, or more recently, whether that is a network scalability and
flexibility question or a question of an easy-to-enter, low-cost storage
platform. So, yes, we provide integration and fully support integration
of all our software-defined aspects of the data center. That is in the
three dimensions I mentioned.
We will soon be posting some demos of this kind of working with
NSX,
for example. But we do not prescribe that our IT professional has to
use Virtual SAN with NSX, or vice versa, and only in that way. So Virtual
SAN can be used on its own, with more traditional network
configurations. NSX can replace those network infrastructure and it will
work seamlessly with Virtual SAN.
We see different
parts of adoption by different customers. Some of the bigger
enterprises, including financials, being more sophisticated and perhaps
more forward-looking, they are more aggressive with total software-defined
data center approach. Other customers are a bit more cautious and apply
software-defined principles in the main areas they are concerned with.
Value proposition
Farronato:
When you look at a product like Virtual SAN, one interesting finding,
after the first three months that the product has been available, is
that the value proposition is really resonating across pretty much all
customer segments, from the smaller
SMBs, all the way up to the larger enterprise customers.
While
it’s difficult to comment on the exact sequence as to how
software-defined data center has been deployed, it is interesting to see
that a technology like Virtual SAN is resonating pretty much across all the market segments, and so it expresses a
value proposition that is broadly applicable.
Gardner:
I suppose there are as many on-ramps to software-defined data center as
there are enterprises. So it's interesting that it can be done at that
custom level, based on actual implementation, but also have a strategic
vision or a strategic architectural direction. So, it's future-proof as
well as supporting legacy.
The value proposition is really resonating across pretty much all
customer segments, from the smaller SMBs, all the way up to the larger enterprise customers.
How about some examples? Do
we have either use-case scenarios or an actual organization that we can
look to and say that they have deployed these VSAN and they have
benefited in certain ways and they are indicative of what others should
expect?
Farronato: Let me give you some
statistics and some interesting facts. We can look at some of the early
examples where, in the last three months since the product has become
available, we've found a significant success already in the marketplace,
with a great start in terms of adoption from our customers.
We
already have more than 300 paying customers in just one quarter. That
follows the great success of the public beta that ran through the fall
and the early winter with several thousand customers testing and taking a
look at the product.
We are finding that
virtual desktop infrastructure (VDI)
is the most popular use case for Virtual SAN right now. There are a
number of reasons why Virtual SAN fits this model from the scale out, as
well as the fact that the hyper-converged storage architecture is
particularly suitable to address the storage issues of a VDI deployment.
DevOps, or if you want, preproduction environments,
loosely defined as test dev, is another area. There are disaster recovery targets in
combination with
vSphere Replication and
Site Recovery Manager. And some of the more aggressive customers are also starting to deploy it in production use cases.
In the last three months since the product has become
available, we've found a significant success already in the marketplace.
As
I said, the 300 customers that we already have span the gamut in terms
of size and names. We have large enterprises, banking, down to the
smaller accounts and companies, including education or smaller SMBs.
There
are a couple of interesting cases that we'll be showcasing at VMworld 2014 in late-August.
If you look at the session list, they're already available as actual use
cases presented by our customers themselves.
Adobe
will be talking about their massive implementation of Virtual SAN. And
for their our production environment, on their data analytics platform,
there will be another interesting use case with
TeleTech talking about how they have leveraged
Cisco UCS to progress VDI deployments.
VDI equation
Gardner:
I'd like to revisit the VDI equation for a moment, because one of the
things that’s held people up is the impact on storage, and the costs
associated with the storage to support VDI. But if you're able to bring
down costs by 50 percent, in some cases, using software-defined storage. That radically changes the VDI equation. Isn’t that the case, Christos,
where you can now say that you can do VDI cheaper than almost any other
approach to a virtualized desktop?
Karamanolis:
Absolutely, and the cost of storage is the main impediment in
organizations to implement a VDI strategy. With Virtual SAN, as Alberto
mentioned earlier, we provide a very compelling cost proposition, both
in terms of the capacity of the storage, as well as the performance you
gain out of the storage.
You get the needs, both capacity and performance of your VDI workloads
for a fraction of the cost you would pay for with a traditional disk
array storage.
Alberto already touched on
the cost of the capacity, referring to the difference in prices one can
get from server vendors and from the market, as opposed to single
hardware being procured as part of a traditional disk array.
I'd
like to touch on something that is an unsung hero of Virtual SAN and of
VDI deployment especially, and that's performance. Virtual SAN, as
should be clear by now, is a storage platform that is strongly
integrated with our hypervisor. Specifically, the data path
implementation and the distributed protocols that are implemented in
Virtual SAN are part of the
ESXi kernel.
That
means that, because of that, we can actually achieve very high
performance goals, while we minimize the CPU cycles that are consumed to
serve those high I/Os per second. What that means, especially for VDI,
is that we use a small slice of the CPU and memory of every single ESXi
host to implement this distributed software-driven storage
controller.
It
doesn't affect all the VMs that run on the same ESXi host, who have
already published extensive and detailed performance evaluations, where
we compare VDI deployments only on Virtual SAN versus using an external
disk array.
And even though Virtual SAN use percentage
is cut to be 10 percent of local CPU and memory on those hosts, the
consolidation ratio, the number of virtual desktops we run on those
clusters, is virtually unaffected, while we get the full performance that
is realized with an external, all-flash disk array. So this is the
value of Virtual SAN in those environments.
Essentially,
you get the needs, both capacity and performance of your VDI workloads,
for a fraction of the cost you would pay for with a traditional disk
array storage.
Gardner: We're only a few weeks from
VMworld 2014 in San Francisco, and I know there's going to be a lot of interest in
mobile and in desktop infrastructure for virtualized desktops and
applications.
Do you think that we can make some sort
of a determination about 2014? Maybe this is the year that we turn the
corner on VDI, and that that is a bigger driver to some of these higher
efficiencies. Any closing thoughts on the vision for software-defined
data center and VDI and the timing with VMworld. Alberto?
Last barrier
Farronato:
Certainly, one of the goals that we set ourselves for this Virtual SAN
release, solving the VDI use case, eliminating probably the last
barrier, and enabling a broader adoption of VDI across the enterprise,
and we hope that will materialize. We're very excited about what the
early findings show.
With respect to VMworld and some
of the other things that we'll be talking about at the conference with
respect to storage, we'll continue to explain our vision of
software-defined storage, talk about the Virtual SAN momentum, some of
the key initiatives that we are rolling out with our OEM partners around
things such as Virtual SAN Ready Nodes.
We're going
to talk about how we will extend the concept of policy management and
dynamic composition of storage services to external storage, with a
technology called Virtual Volumes.
There are many other things, and it's gearing up to be a very exciting VMworld Conference for storage-related issues.
Gardner:
Last word to you, Christos. Do you have any thoughts about why 2014 is
such a pivotal time in the software-defined storage evolution?
Karamanolis:
I think that this is the year where the vision that we've been talking
about, us and the industry at large, is going to become real in the eyes
of some of the bigger, more conservative enterprise IT organizations.
With
Virtual SAN from VMware, we're going to make a very strong case at
VMworld that this is a real enterprise-class storage system that's
applicable across a very wide range of use cases and customers.
With
actual customers using the product in the field, I believe that it is
going to be a strong evidence for the rest of the industry that
software-defined storage is real, it is solving real world problems, and
it is here to stay.
Together with opening up some of
the management APIs that Virtual SAN uses in VMware products to third
parties through this Virtual Volumes technology that Alberto mentioned,
we'll also be initiating an industry-wide initiative of making,
providing, and offering software-defined storage solutions beyond just
VMware and the early companies, mostly startups so far, that have been
adopting this model. It’s going to become a key industry direction.
I believe that it is
going to be a strong evidence for the rest of the industry that
software-defined storage is real, it is solving real world problems, and
it is here to stay.
Gardner:
You've been listening to a sponsored BriefingsDirect
podcast discussion on how one of the most costly and complex parts of
any enterprise’s IT infrastructure, storage, is being dramatically
changed by the accelerating adoption of software-defined storage.
And
we've heard how IT leaders are simultaneously tackling storage pain
points, such as scalability, availability, agility, and cost, while also
gaining significant strategic and architectural level benefits through
software-defined storage. Of course, probably the poster child
application for that is VDI.
So a big thank you to our
guests, Alberto Farronato, Director of Product
Marketing for Cloud Infrastructure, Storage, and Availability at VMware.
Thank you so much, Alberto.
Farronato: Thank you. It was great being with you.
Gardner:
And we've been joined also by Christos Karamanolis, Chief Architect
and a Principal Engineer in the Storage and Availability Engineering
Organization at VMware. Thanks so much, Christos.
Karamanolis: Thank you. It was a pleasure talking with you.
Gardner:
And also a big thank you to our audience for joining us once again on
BriefingsDirect. This is Dana Gardner, Principal Analyst at Interarbor
Solutions. Thanks for listening, and don't forget to come back next
time.
Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.
Transcript
of a BriefingsDirect podcast on the future of Virtual SAN and how it
will have an impact on storage-hungry technologies, especially VDI.
Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.
You may also be interested in:
Tell us about some of the drivers for your organization, and then also about some of your technology decisions.
