Transcript of BriefingsDirect podcast with Hewlett-Packard on series of Sept. 2 announcements on enterprise virtualization products and services.
Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Sponsor: Hewlett-Packard.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today, a sponsored podcast discussion about the growing and important topic of virtualization -- at multiple levels in IT organizations, that is to say, for applications, infrastructure/servers, as well as for clients and desktops.
We're going to talk about services and products in the marketplace, along with the demand and the economic and business payoffs that virtualization is already bringing to many companies. We expect virtualization technologies and techniques to bring even more productivity gains in the near future.
We're going to be discussing Hewlett-Packard’s (HP's) approach to virtualization and a series of announcements that came out on Sept. 2. [See slide show on announcements. See an accompanying interview with John Bennett, virtualization lead at HP Services. See an IDC white paper on business benefits of virtualization.]
We're here with Greg Banfield, consulting manager for the HP Consulting and Integration (C&I) Group infrastructure practice. Welcome to the show, Greg.
Greg Banfield: Thank you very much.
Gardner: Dionne Morgan also joins us. She is the worldwide marketing manager for HP’s Technology Services Group (TSG). Welcome, Dionne.
Dionne Morgan: Thank you.
Gardner: And we have Tom Norton, worldwide practice lead for Microsoft Services at HP. Hello, Tom.
Tom Norton: Hello.
Gardner: Virtualization, of course, has been with us for quite some time. The technologies of virtual machines and hypervisors have been around for a while, but this is really starting to gain ground for a variety of reasons. In many organizations, there are economic reasons, technology reasons, and business outcomes reasons.
People are finding that getting higher utilization is the only part of the story. We're also finding that virtualization is taking place in the context of larger IT undertakings, be it data center consolidation, application modernization, services oriented architecture (SOA), business continuity, and energy savings, just to name a few.
I want to start out by talking with Greg about "why now?" Why are the market and HP focused on virtualization as such a significant development in the market at this point in time?
Banfield: It comes down to a few things. It comes down to our customers asking what HP has done within our own data centers, and how we have done it, because we have gone through the transformation ourselves as a company and have gained a lot of experience around that. It also comes down to a few things in the economics around cost, cost of labor, cost of machines. The price of machines is going down; power is coming up.
They're looking into getting a better handle on using those servers, and using the access they have, and trying to fully utilize them to make sure that the applications that they serve within their company and with their users are fully utilized, and to take advantage of the new servers and technologies that are coming out today.
Gardner: Now, HP is in a unique position in that it has hardware, services, clients, software infrastructure, software management, and partnerships across multiple providers of virtualization technology. This seems to be almost a ready-made business with IT development in the marketplace. Tell us how HP views this opportunity as a company.
Norton: What’s interesting about virtualization is that, as companies have started to work with virtualization, the easy assumption is that you are really reducing the numbers of servers. But, as you expand your knowledge and your experience with virtualization, you start looking at comprehensive components in your environment or your infrastructure.
You start understanding what storage has to do with virtualization. You look at the impact of networks, when you start doing consolidation in virtualization. You start understanding a little bit more about security, for example.
Also, virtualization, in and of itself, is really allowing you to consolidate the sheer number of servers, but you still have the idea that each of those virtual servers needs to be managed. So, you get a better view about the overall impact of device management, as well as virtual machine management.
HP is unique in that ability to be able to understand it from a client perspective, from a server perspective, and, as I mentioned, storage, software, networks. It’s actually a tremendous opportunity for HP to work with our customers to give them an overall strategy of how all of those components work together to deliver the value they are looking for in virtualization, to look at cost, and, as you mentioned earlier, to look at flexibility, security, disaster recovery, and rapid presentation of application. We are in a unique position in the industry to be able to help our customers address all of those issues, which have an impact on virtualization.
Gardner: Virtualization, of course, has been targeted largely at individual server farms or data centers, but, as we are describing it, it really does impact quite a bit across the board for IT. I'm also wondering what the impact is on the business. Let’s go to Dionne. What are the business outcomes, values, or productivity benefits that virtualization supports and underscores and that help the IT people make the case for this investment?
Morgan: One of the key areas is cost reduction. Virtualization can help with major cost savings, and that can include savings in terms of the amount of hardware they purchase, the amount of floor space that’s utilized, the cost of power and cooling. So, it improves the energy efficiency of the environment, as well as just the cost of managing the overall environment.
A lot of customers look to virtualization to help reduce their cost and optimize how they manage the environment. Also, when you optimize the management of the environment, that can also help you accelerate business growth. In addition to cost reduction, customers are also beginning to see value in having more flexibility and agility to address the business demand.
You have this increased agility or ability to accelerate growth. It also helps to mitigate risk, so it’s helping improve the up-time of the environment. It helps address disaster recovery, and business continuity. In general, you can summarize the business outcomes in three areas: cost reduction, risk mitigation, and accelerated business growth.
Gardner: Virtualization also adds complexity. When you've got multiple instances running on single hardware, or hardware is virtualized, there is a management hurdle. Bringing this into play across both the physical and the virtual infrastructure is also another management hurdle. I wonder if any of our panel could help me understand a little bit more about doing this the right way from a management perspective.
Norton: What’s interesting about this is that when you get into a virtualized environment, there's a need to understand the heartbeat of the virtualized environment and understand what’s going on at the hardware level. As you grow up from there with the virtualized machines, you have to understand how the virtual machines themselves are performing, and then there's got to be an understanding of how the actual applications are performing within that virtual machine.
So, management and virtual machine management, overall a comprehensive approach to management, is critical to being successful. One of the other areas that is addressed through management -- as well talked about a lot -- is virtual machines sprawl.
Organizations have gone into virtualization with the hope of reducing the number of servers that they manage in their environment. They turn around with a reduced number of physical devices, but they actually end up with more servers. The control of virtual machines is less difficult, which is a good thing, because you have more flexibility. It also can become a burden, because you can quickly lose control of the sheer number of servers, and, the work that goes into managing those servers during patches, upgrades, and the security issues that go along with it.
So, virtual machine management is actually the key contributor to all this. When you think in terms of that, you really have to think in terms of both the actual management of the machine, the physical device, and understanding the utilization of a processor, the health of the computer itself, and then understanding the health of the virtual machines that sit up on top of it.
HP has a unique ability, because we've been working with virtualization since the 1990s. We've been working with virtualizing and understanding the physical nature of the devices for years, and our engineering groups now have invested a lot of time in working with our partners -- VMware, Microsoft, and Citrix -- to understand their virtual machine management and how our tools and their tools can work together and become integrated to provide that comprehensive view that is required now to really properly manage virtual machines.
Gardner: And, we're talking about a heterogeneous environment from the start with this. According to analyst reports, some 80 percent of enterprises are using virtualization on multiple platforms, with half using three or more platforms. So, this really becomes a critical management issue from that perspective.
Let’s go to Dionne and talk about what HP is calling a rethinking of infrastructure. We've talked about the paybacks as an economic incentive, an agility incentive. Organizations can use virtualization to support and augment some of their ongoing work towards consolidation, unification, modernization, part of the IT transformation in a long-term trend, but you are thinking that this is also a milestone at this point by rethinking infrastructure. I wonder if you could help us understand what you mean by that.
Morgan: Organizations need to think not only about their servers, their storage, and their network for the virtualization perspective, but to look at this from an integrated perspective and have an integrated management view of the data center. It’s not just about the technology. They also have to think about this in terms of the people, the processes, and the technology.
Tom was describing how we can help manage the physical and the virtual. In addition to that, we also need to look at how we manage the ongoing processes, which are going to be responsible for "operationalizing" the virtual environment. This could include the adoption of key industry best practices and standards.
Some best practices that come to mind, are what come from the Information Technology Infrastructure Library (ITIL), how you actually use these ITIL processes, and how you take it a step further and automate some of those processes. It’s an industry best practice for managing services that you deliver to the business.
It’s very important to look at the technology, both the physical and the virtual, the processes required to manage, the automation of processes to manage the virtual environment, and also the people within your organization, ensuring that they have the right skill sets and the right information to utilize and take advantage of this virtualization investment.
Gardner: Let’s take that point about personnel to Greg. Tell us about what the skill sets are? It sounds like this is a bit different. Is there training and the ability to bring your staff or IT operation staff up to spec on this? Is there a too much of a demand in the field for people and personnel with experience? What’s the outlook for the human resources aspects of virtualization?
Banfield: That’s a great question. One thing we have heard from the consulting side is that people understand, customers understand, and CIOs understand about the cost savings and those types of things.
What they are asking us, when we go and do these things, is "I understand we are going to save money. I understand my server count is going to go down. What I am struggling with is people and the processes. I have many processes to handle within my infrastructure and people, and I need to get them redeployed or re-energized into other things that will actually generate growth for our company, as opposed to just shepherding servers now as administrators or other types of things."
From that perspective again, ITIL, as we just mentioned, is a great tool that we can use in the processes. From HP’s perspective, our consultants have done this many times in house and with other customers. We bring to the table the know-how from doing this before, doing transportation projects that we can help the customer move from where they are today to where they need to be in a virtual perspective.
It's not from the infrastructure so much, although we can do that, but the bigger piece is how do we get where we are today, with our processes and people, to where we need to be from an infrastructure standpoint in a virtualized world.
So, yes, our folks are trained. We have many certified people on ITIL, virtualization, and our partner certification VMware and Microsoft. It’s a great opportunity for our customers to work with HP. We have a wealth of knowledge, both from a training perspective, from practical know-how, from just doing it before.
Gardner: I think we have a sense of the vision here, the promise, and also some of the challenges. So, HP on September 2 came out with a number of announcements, some methodologies. We are looking at virtualization from a strategy perspective, design perspective, the transition and integration basis, and then ongoing improvement and return on investments (ROI). Let’s look at the first two, strategy and design. What we are talking about in terms of the September 2 announcements on virtualization in regard to strategy and design?
Norton: Strategy is becoming even more important. Our customers are very aware, as everyone else is now, that they have many options available to them as far as virtualization, not only from a perspective of what to virtualize in their environment, but also from a number of partners and technology suppliers who have different views or different technologies to support virtualization.
Our customers, from a strategy and design perspective, have looked to us to provide some guidance that says "How can I get an idea of the net effect that virtualization can have in my environment? How can I present that and gain that experience, but at the same time understand my long- term view of where I want to go with virtualization, because there is so much available and there are so many different options? How do I make a logical and sensible first attempt at virtualization, where I can derive some business value quickly, but also match that up against strategy for a long-term vision?"
What we are trying to supply with these new services around virtualization is the idea that we can provide our customers with a strategy and a short-term proof of concept, or short-term, rapid, or accelerator, implementation of virtualization. Whether it's on the desktop side or on the server with Microsoft’s new Hyper-V, to give them that experience, they can have that experience contribute to a long-term vision as far as that long-term infrastructure design.
What we are trying to look for is taking the complexity out of an introduction to virtualization. We're trying to take the complexity out of the long-term vision and planning and give the customers an idea of what their journey looks like, rapidly introduce it, but in the right direction, so they are following their overall vision in gaining their overall business value.
Gardner: It sounds really important to bring all of the numerous aspects of IT that are affected by this onto the same page, under a road map with the same vision, and then get into a lifecycle perspective. Now, once we've got our vision, we have our perspective, and we have got all the people on board, it’s down to brass tacks, and then transition and integration. Greg, what’s in store for the HP community, vis-à-vis, this level of the deployment?
Banfield: Then, we would have our HP services consultant in integration come in and work with the customer. We've gone through the design phase and the strategy phase, and now we work with the customer to take what we've got on paper and get it going. Typically, we do something in a phased approach, because we're talking about some very large projects. As we've talked about for last 20 or 25 minutes here, it’s a complex environment that we're dealing with. We're dealing with multiple vendors, multiple business groups, and multiple applications, everything impacting a different thing.
We have the design, so we actually start going. We have solution architects and project management using best practices, working hand-in-hand with the customer to make sure that, as we go through this, and there’s changes involved, we are on track.
Of course, as you go through these projects, you have to keep going back, as Tom was mentioning, to your original strategy and your original design, and keep checkpoints. Are we still meeting the criteria for the business? Are we still taking what we have learned during the first two phases, its implementation, and the transition integration valid?
We keep reassessing, as in any large project we go through or anyone we would do. You validate against your milestones and checkpoints and then make adjustments as needed.
Gardner: And then, Dionne, as you mentioned earlier, the business outcomes are important, and the improvements in ROI come into play. So, it’s not enough just to deploy and sit back and wait for the benefits with virtualization. This is an ongoing process, very dynamic, changeable. I think one needs to tweak and manage their resources to improve that productivity to get that economic return. Can you tell us little bit more about what HP has in mind for this long-term economic value?
Morgan: Once you actually transition your solution into production, you have to look at the ongoing operations and the continual improvement of those services that you are providing back to the business. In terms of the ongoing operations, you have to continue to assess your people's skills and your operational processes.
HP provides services to assist with its ongoing operation to help to increase the stability of the virtualized environment. That includes everything from education courses, to software, technical support services, and hardware support services. We also have proactive services, which are really focused on that continual improvement phase of the lifecycle.
On a regular basis, we assess what’s happening in the organization from a people, a process, and a technology perspective. We benchmark against what’s happening in the industry, making recommendations on where a customer can actually improve, on some of those processes to improve efficiency, and to improve on the service level they are providing to the business. We also assist with the implementation of some of those process improvements.
If you look at this from a full lifecycle perspective, HP provides services to assist with everything from strategy, to design, to transition, to the ongoing operations and continual improvement.
Gardner: It was mentioned earlier that HP has gone about a good deal of this virtualization transformation itself. It also worked with some leading-edge customers to deploy and refine. Do we have any metrics, do we have any view into what this means in terms of payback? Is this iterative, minor, 10 percent? What kind of payback typically are we starting to see from a well-planned, well-organized, well-implemented virtualization strategy?
Norton: I don’t know if every company is going to be the same as far as what they may desire to achieve. We've had examples of customers. Greg’s group worked with a financial organization through an accelerator service, in other words going through the whole strategy and discovery phase and trying to measure their environment to look at capacity. They have seen reductions to go from 300 to servers in their environment to 30, at least in the sample of servers that have been evaluated.
That’s just one customer’s example, and everyone could potentially be different, but the idea is just the same. You can look at the number of physical devices and go through an analysis that will look at how these applications can be virtualized and what the utilization of the equipment is. You can have a simple reduction in the number of devices.
HP will also, like our own organization, look at the actual application that’s being virtualized. Maybe it’s not just the case of reducing the number of physical devices and having the same number of servers running. True savings come in when you’ve decided to reduce the number of instances of an application that maybe running on servers. You can add this sense of application virtualization.
The classic example in those cases is an organization that may have 200 remote Microsoft Exchange Servers in their environment. They can look at bringing those distributed remote workstation into a data center environment and find cost savings in administration and data protection. But, there’s still a huge expense in how those Exchange Servers are still sitting on virtual machines. So you still have 40 Exchange Servers and you are still managing each one of those.
Another saving gets involved in that too, where you decide, "I am actually going to reduce the shared number of that 40. I may reduce my Exchange Servers from X number of devices to a quarter of that." Then, you still have those devices that reduce the number of Exchange on servers running within that consolidated environment as well, and that dramatically affects that kind of cost saving.
Cost savings vary, but it can be dramatic. It can be as dramatic as CAPEX expenditures in the hardware base and it can be very dramatic from an application-management perspective or a server management perspective.
Organizations now are looking, like HP did, in both areas, reducing the shared number of physical devices in the data center, and reducing the number of instances of an application that are actually running on servers to provide you even greater benefit.
Gardner: I suppose, generally, what we are able to do now with virtualization is to match supply and demand with much more precision than we could in the past. In the past, we had to throw huge amounts of resources at a problem with brute force and sort of a blunt instrument approach in order to make sure that we could accommodate all sorts of demands and spikes and requirements,
Now, we are able to use virtualization to refine these supply-and-demand equations, so that we can pull resources at the infrastructure level, pull resources at the application level, and reduce a lot of waste and unnecessary or underutilized resource.
Banfield: Another thing that Tom was hitting on, besides this physical savings of the environment with power and air conditioning and things like that, is agility -- agility to market. As Tom was saying, you can now move applications and other things around. Your workforce becomes much more agile to address critical business needs in a very timely manner with virtualization. I think that’s key to our customers.
Gardner: So, if we want to move a whole new set of application to our Asia-Pacific operations and target a whole new set of customers there, the ramp up to doing that is much less time and much more something you can manage, rather than have to forklift upgrade, is that correct?
Banfield: Absolutely.
Gardner: As an analyst I get some questions frequently, and one of them I have to throw out to you guys, because it’s sort of an obvious one. Why would a company that makes a significant amount of money from hardware want to reduce the number of hardware instances? How does that help you, or what is the long-term implications that I am missing?
Norton: What happens is, as you are going to change in a platform, when you move from, say, individual instances of a device that sits in a branch office some place, and it’s maturing and it’s isolated, it’s disconnected in essence, because it’s separate from all the processes that have in the data center.
From a hardware perspective, it’s a great opportunity for HP, not only because we are changing some of these legacy platforms, as they will be sitting out in these remote offices, but we are enabling our customers to actually run on a much more effective and newer platform. It's a much more powerful platform, with direct connectivity to more powerful storage systems, and more powerful networks that run in datacenter.
It’s a plus for both. Our customers gain an advantage, because there are going to be savings overall in how much money they spend on that old equipment, how much maintenance cost they have, how much systems management they need to do for this device to sit out there or even sit in their data center, and have to be supported in much less efficient way.
We can save them money by moving them to more powerful, more efficient platforms. At the same time, it allows us to introduce our customers to these new devices, that provide them a wealth of benefits, from the performance perspective, on security, stability, and high availability. It’s a win for both organizations, along those views.
Gardner: Okay, let’s look at the actual announcements of Sept. 2. I'm going to break out one first, and that’s the desktop virtualization announcements -- virtual desktop infrastructure (VDI) solutions services, using Citrix XenDesktop.
Again, we're looking at a pretty radical shift in the types of end-user devices. We could start using some thin clients, and there is a security and risk reduction opportunity for bringing the data and applications-configuration information onto the server. The end users basically have a seamless environment. They're getting the same desktop and operating system that they’re accustomed to.
There are tremendous opportunities to save costs here. Let’s look before we drill into each of these announcements. Let’s just break out desktop, the virtual desktop and the infrastructure set. Tom, let’s go to you on that first. What’s the big deal here? What are we talking about, when we are going to reduce the amount of actual client-side activity vis-à-vis virtualization?
Norton: When our customers sit down and do a study, we help them look at the cost of managing client or end user devices in the field, not only from a help desk study but from a productivity study, from an application presentation viewpoint for the end user, the applications that they use and how they are presented is the heartbeat of business.
The data that they use is so sensitive, and so important to the organization as a whole. When they need help in keeping their productivity up, it can cost money or it can save the organization money. So, you look at changing somebody from a very insecure, volatile device in a remote environment that they use on a daily basis.
Gardner: So a local laptop for example.
Norton: A laptop, right. You can still create that rich experience that they are used to, but give yourself the security of knowing that the data that they are using is protected from theft and also protected, as far as archiving and search availability, from governmental regulations. They can give users some of that rich experience, but still have that protection. You can look at that device and understand the cost and complexity of either upgrading the device, presenting an application, or deploying an application to that device.
It’s extraordinarily expensive to do that and, if they can still get the experience of a more rapid presentation of the applications that they need to their job on a daily basis, both of those are incredibly valuable to the organization.
If you can get those two advantages, you are going to reduce help desk calls from your end user in the case of a disaster. If you have a notebook and it fails, for example, how do you get that person back up in working again, access the data they need, and access the applications they need?
You can accelerate that recovery. You are receiving enormous amounts of management and spending enormous amounts of money on every device every year. You can accelerate recovery and provide them the same rich experience that these new technologies allow us to do.
If you look at a virtual device now, you can say to the end user, "You will get the operating system that you need. You will get the application that you need. And, it will be in the environment that you expect to work in. You have the same user state you have had."
If you can combine all three of those in a virtualized environment, you are actually, in the end, providing more productivity for the end user, and, at the same time, cutting the management cost. You're also enabling yourself to cut other support costs in the organization, like how much money you spend to protect data, how much money you spent to restore data, or protect it from theft.
So there are enormous advantages to both, but it doesn’t work in every instance. If you have remote users who don’t have daily or hourly connectivity back to a host, it may not be to your advantage to use this technology there. But, for most organizations there is certainly a large part of their population that can take advantage of the technology.
Gardner: We've already seen a lot of this in use in some government organizations, particularly in intelligence and military communities, where they can’t take the risk of having an end device being lost or falling into the wrong hands. So, the stateless approach to computing is quite popular and proven there. Isn’t that right?
Norton: Right. You have a public sector, which is very sensitive, but you can imagine the same in terms of healthcare and financial organizations. You can extend that idea.
It may not just be sensitive data. It maybe repetitive tasks or frequent upgrades of applications. You have large segments of users who would have redundant equipment, and they have no need for a rich experience, but they may need an application refreshed on a predictable basis. This allows you to do that gracefully.
Gardner: Once again, this strikes me as aligning supply and demand -- what the end user actually needs in terms of resources, versus having the equivalent 20 years ago with a supercomputer on every desktop.
Norton: That’s correct.
Gardner: Let’s go to these announcements one by one very quickly, so we can give our audience a sense of the breadth and depth of this wave of addressing the virtualization issue. The first is HP Virtualization Accelerator Services. Greg, can you tell us quickly what this means?
Banfield: As we talked about, virtualization is a lifecycle, a journey. HP has Accelerator Services, which are predefined services from the consulting organization for customers to plug-in their module, to plug into where they are within their lifecycle. Because this is a lifecycle, customers could be at any point there, whether design or strategy. Maybe they're just starting, are half way through a project, or maybe towards the end of a virtualization project.
As we talked about, maybe the business outcomes weren’t exactly matched up with the original design, and they need some help in that area. Consulting integration comes with these Accelerator Services to help the customer through those difficult times or any point in the lifecycle to make sure that they are gaining the full value from their virtualization journey.
We can talk about each individual package, if you like, or a service, as we move forward.
Gardner: We'll come back to the services once we get through the major elements.
There are also the VDI services that we just discussed. I’ll just touch with Tom one more time on that. It seems to me also that with desktop virtualization, they were sort of getting the best of the old and the new. The old paradigm was centrally organized and managed, even back in the mini and mainframe days.
There were a lot of benefits to the organization for doing it that way, but the end user didn’t get the flexibility, the innovation, the freedom and flexibility, and so forth. Now, we're able to blend the two of them to get that centralized benefit for operations -- upgrades, maintenance, management, and aligning the supply and resources with the demand of the end user much more efficiently.
At the same time, we're giving users a Microsoft Windows desktop, where they can pick and choose, move, and get a lot of resources still using a browser. Am I off base here or are we really looking at the best of both worlds?
Norton: Absolutely. It does a number of things. Everybody uses PCs at home. The generation we are working with now has grown up with that equipment, so they are very accustomed to having a personalized work environment. They are used to having some flexibility to obtain applications and run them on their own devices.
They are accustomed to performance and also access to data, not having to wait for access, not having to wait for what historically has been a very slow change management process on mainframe based systems to add an application or change an application.
They are used to that agility, to that that high frequency of change. Up until now, many people have been resistant to make that move, because they don’t want to have that rich experience impacted. Now, you get that great benefit of having the rich experience, but, at the same time, you have the ability to take advantage of what consolidation means -- the predictability, disaster recovery, security, those types of developments which you never could get before in a more unpredictable world.
Through VDI, you really get that idea that you have the best of both, from a consolidation perspective as well as distributed to computing perspective.
We feel we are satisfying both ends. When we look at VDI, it’s kind of interesting. It touches both the back end systems and it touches the end user client. Sometimes with virtualization, people just think in terms of that back office, the server room, the datacenter transformation idea.
With VDI now, you have taken it and bridged that gap, to where you can do things on the desktop side, as you mentioned earlier, about taking advantage of thin clients. HP is producing some great thin-client technology. You can extend the life of current hardware, if you wish.
If you're mid-term in the lifecycle of a notebook or desktop, you are not really ready to retire it yet, but you don’t really want to spend a considerable amount of money to upgrade. You can extend the life of that device and make it more useful by combining that device with this type of technology.
At the same time, if you have a high performing device, this gives you the flexibility to just virtualize one application out to that device. So, it gives enormous flexibility on the front side.
On the back side, in the datacenter play, it allows you to do a lot of things. You can take advantage of all the benefits of blade technology, that whole idea we've discussed before about storage, virtualizing storage, and having better access to available storage.
You may run out of storage on a notebook device, but you can request and expand your storage capability on a storage area network (SAN) and go forward from there. So it’s unique in that it can address both the client side, the end-user facing side, and the efficiencies, and predictability, and performance that you want in a datacenter.
Gardner: Part and parcel of virtualization technology is the need for planning and ongoing management and then professional services, methodologies, and best practices. That’s why we have a number of virtualization support services announced as well.
I'll run down the list. HP Virtual Server Environment Solution Service. I assume this is about increasing energy, footprint, and resources.
HP High-Performance Computing Cluster Management Solution Service, HP Integrity/HP9000 Solution Service, HP Server Solution Project Management, HP Virtual Server Solution Planning and Design, HP Global Workload Manager Solution Services, and HP Virtual Desktop Infrastructure Solution Services.
Let’s go to Greg. What are the high points here? We don’t have too much time, so give us sort of an overview what these services involve and how comprehensive they are in terms of the whole series of virtualization opportunities that organizations will face?
Banfield: Because we're talking modular, all the services you mentioned are in a modular or tier fashion. Any one of these can be molded to the customer’s need, whether you have 200 servers or 1,000 servers.
The tiered approach makes it very easy for the customer to pick and choose what they need, depending on where they are in the lifecycle. They are predefined and data-sheeted, so the customer can read what they're going to receive from HP. These seven or eight different services address different points within the life cycle.
All these services do come with project management. Some customers, as I said, are half way through the lifecycle, or on their way, and maybe they just need a little help with project management types of activity. So HP can provide a PMI-certified person to come in there and help them, maybe just work with them to get the project back under control. Maybe it's off a little bit.
So any one of these is a great way for a customer to take a look at our solutions. Again, they are couched to be sort of a quick hit, easy to use. You don't have to just pick one service. If you have different needs, you can say, "I need to take the Virtual Server Environment Solution service, and I need Global Workload Manager to create my entire solution." Again, it's easy for the customer to understand, and then move forward with the project.
Gardner: Let’s look at how you can get started. These announcements are targeted at the U.S. initially, and you are taking it out globally during 2009, which isn’t that far away now. Tell us how an organization can get started and where they can develop this strategic overview of virtualization?
Norton: There are a couple of different ways that customers can engage to get this started. We can engage our customers to get this started first through our traditional sales organization. From a hardware perspective, we have our traditional enterprise account managers and their associated services client principals, the associated services managers who work on those accounts. That’s a very traditional way to engage with our technology teams, who provide these kind of services, both on a support perspective as well as a consulting services perspective.
But, there are other ways as well. You can work directly with our Microsoft Alliance members, you can work with our alliance teams. In this case, you're talking about Microsoft virtualization.
You can do it from a services perspective. Maybe you can take it from an enterprise account perspective, whether hardware or storage. Those channels should not change at all as far as how you would work with HP from a services perspective to get people to come in. If you look at those services, we set them up too. So, it's very easy for our customers to get engaged.
We are presenting services that allow the customer to get engaged, even if it’s a half-day workshop about virtualization. As Greg mentioned, we have strategies that can go two to three days. We have longer term proof of concepts that can go three to four weeks. We try to make it as easy as possible from a services perspective, and also from a sales perspective. We are very flexible.
These can even be introduced through our channels, where these can be sold through a channel and delivered by HP. We are trying to provide flexibility, as well as simplicity, in the services acquisition process for our customers, so that they don’t have to worry about who to talk to. When they need to talk, they can go directly to their traditional HP sales force and get introduced to these services.
Morgan: If I can also just add to that, once our customers began transitioning into production, they also should think about the people and the processes again. From a people perspective, we also announced on Sept. 2 some new education courses, which tie into what Greg and Tom were just describing.
For example, we have an education course on the HP Insight Dynamics BSC software. We have education courses on partition management, and we also announced an HP Virtualization Boot Camp, which covers global workload manager, and virtualization manager, capacity advisor, and a long list of technologies.
Customers should really think about getting their people trained in these technologies. And, from an ongoing operations perspective, we also announced some new software technical support services.
We already provide a lot of support services in the virtualization area, but what we're adding to that, is support for additional VMware products, such as VMware Workstation, VMware Lab Manager, VMware Site Recovery Manager, as well as the operational support for Citrix XenDesktop Server, and, of course, the new HP hardware, such as the HP ProLiant BL495C virtualization blade.
Gardner: Very good. I think we've covered a lot of territory here today, from vision down into actual product and service offerings. Clearly, this is going to be something that companies are going to be dealing with for a long time. We are already seeing forecasts now for virtualization to be growing broadly in the coming year. A 50 percent growth in 2008 alone, and 70 percent just in the previous two years.
So we appreciate everyone’s input, and wish you well on this series of announcements. We have been discussing virtualization at the application desktop and infrastructure server levels, as well as the road map and lifecycle management issues associated with that. We've been joined by Greg Banfield, a consulting manager for Hewlett-Packard, Consulting Integration Infrastructure group. Thank you, Greg.
Banfield: Thank you for having me.
Gardner: Dionne Morgan, worldwide marketing manager for HP’s Technology Services Group. Thank you, Dionne.
Morgan: You're welcome.
Gardner: And Tom Norton, worldwide practice lead for Microsoft Services at HP.
This is Dana Gardner, principal analyst at Interarbor Solutions. You have been listening to a sponsored BriefingsDirect podcast. Thanks, and come back next time.
Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Sponsor: Hewlett-Packard.
Transcript of BriefingsDirect podcast with Hewlett-Packard on series of Sept. 2 announcements on enterprise virtualization products and services. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.
Tuesday, September 02, 2008
Thursday, August 21, 2008
Pulse Provides Novel Training and Tools Configuration Resource to Aid in Developer Education, Preparedness
Transcript of BriefingsDirect podcast on Java training and education with Genuitec Pulse for Java and Eclipse.
Listen to the podcast. Sponsor: Genuitec.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.
Today, a sponsored podcast discussion about the complexity around Java training and education. The development toolset, the plug-ins, the community are all very fast-moving targets. It's difficult for trainers, educators -- not to mention the students and budding developers themselves -- to get a full grasp of what's expected of them, and then to find resources that are up-to-date and timely.
We're going to be discussing with some experts how better to organize Java training and education. We're going to look at the Eclipse profiles that can be organized and coordinated using Pulse, a Genuitec-organized tools configuration network function. We're going to learn about how organizations can organize their training, so that students can better anticipate what's expected of them in the real world.
To help us understand some of these issues and work toward some solutions, we're joined by Michael Cote, an analyst with RedMonk. Welcome to the show, Michael.
Michael Cote: Hello, everybody.
Gardner: We're also joined by Todd Williams, vice president of technology at Genuitec. Welcome to the show, Todd.
Todd Williams: Thanks very much, Dana.
Gardner: Lastly, we're joined by Ken Kousen, an independent technical trainer and president of Kousen IT, Inc. He's also an adjunct professor at Rensselaer Polytechnic Institute. Welcome, Ken.
Ken Kousen: Hi, Dana, glad to be here.
Gardner: As I mentioned, complexity and moving targets are part of the problem, but it also seems that there is a disconnect between book knowledge or training knowledge that one gathers about development and what happens in the real world, what goes on with code being checked in and checked out, and how teams are organized. It seems difficult for someone to anticipate what's really going happen.
Let's go to Ken first. Ken, what is the gap, from your perspective, between what students and budding developers get through training and in university settings, and then what's often expected of them in the real world?
Kousen: It's interesting. The gap between what's taught in academia and what's taught in the real world is very large, actually. The classes I teach tend to be in a master's level program, and I teach a couple of classes in developing enterprise applications that are specifically constructed to address this gap.
Academia will talk about abstractions of data structures, algorithms, and different techniques for doing things. Then, when people get into the real world, they have no idea what Spring, Hibernate, or any of the other issues really are.
It's also interesting that a lot of developments in this field tend to flow from the working professionals toward academia, rather than the other way around, which is what you would find in engineering, when I used to be in that area.
Gardner: Todd, when you're doing hiring or are doing development for business, and you're talking to your customers -- folks that use MyEclipse and your services for training and consulting -- are you seeing a worsening situation in terms of how to acquire qualified labor, or do people have a pretty good sense of where to go to find good Java developers?
Williams: Finding quality employees is always a challenge, and probably always will be. Part of what I see as being difficult, especially in the Java and Enterprise Java market, is the huge number of technologies that are being employed at different levels. Each company picks its own type of stack.
Ken mentioned Spring and Hibernate. There is also Java transaction API (JTA), Java server faces (JSF), and Struts, Web framework and persistence technologies, and application servers that are in use. Finding employees that fit with what you are trying to do today, with an eye toward being able to mature them into where you are going tomorrow, is probably going to always be the concern.
Gardner: Now, what's been going on with development, not just the function, but teams, the collaboration, agile types of activities, Scrum? It used to be that people could specialize, stay in one little niche, but now the "master of all trades" seems to be more in demand.
Let's go to Michael. Michael, is development fundamentally changing? When we think of developers, do we need to recast how we imagine them or conceive of them?
Cote: Yes. I think it's fair even to go to the extreme and say absolutely. You look at the employment patterns that most developers find themselves in, and they are not really working at some place three, five, ten, even twenty years. It's not realistic. So, specializing in some technology that essentially binds you to a job isn't really an effective way to make sure you can pay your bills for the rest of your life.
You have to be able to pick up quickly any given technology or any stack, whether it’s new or old. Every company has their own stack that they are developing. You also have to remember that there is plenty of old existing software out there that no one really talks about anymore. People need to maintain and take care of it.
So, whether you are learning a new technology or an old technology, the role of the developer now, much more so in the past, is to be more of a generalist who can quickly learn anything without support from their employer.
You're not going to get a lot of slack to learn things in a given time, paid training, and things like that. You're pretty much left on your own, or there are always cheaper alternatives to go to.
So the heat is really on developers to be Type A people who are always seeking out the best option.
Gardner: Alright. Well, now that we have scared anyone from ever wanting to be a developer, Ken, help us get a little bit closer to earth. What can students do, what can professors or instructors do, to help get more of this real-world perspective into what they do in these courses and in this preparation?
Kousen: It's interesting that while the various tools and technologies evolve, some of the basic principles always hold pretty fast. I've taught this class several times and I have to say that every time I've taught it, it's been very, very different, but the overall architectural issues are pretty constant.
Plus, what seems to follow in the industry are various trends, like an increased emphasis on testing, for example, the recent rise in dynamic languages, and things like that. The idea of continually trying to follow what's going on in the marketplace and seeing what's interesting seems to be very helpful.
I also emphasize to the students that a good source of information is to find some of the better open-source projects, and not necessarily join them, use them, or do anything with them, but follow what they do and see those projects as the communal efforts of some of the best developers in the world.
So, if they all say, "Oh yeah, we obviously have to have this source-control mechanism," then maybe that's an interesting thing that should be looked at, or this particular bug reporting tool, or whatever. I often emphasis that particular direction as well.
Gardner: How about that, Todd? Are these open-source communities, these chat rooms, these forums, the real, practical lab that the students and developer should be looking towards?
Williams: I think to a degree that it's certainly a practical lab that students have easy access to. Obviously, in open source, whether it’s something like the Eclipse Foundation, Apache, or what have you, they make a very explicit effort to communicate what they are doing through either bug reports, mail lists, and discussion groups. So, it's an easy way to get involved as just a monitor of what's going on. I think you could learn quite a bit from just seeing how the interactions play out.
That's not exactly the same type of environment they would see inside closed-wall corporate development, simply because the goals are different. Less emphasis is put on external communications and more emphasis is put on getting quality software out the door extremely quickly. But, there are a lot of very good techniques and communication patterns to be learned in the open-source communities.
Gardner: Now, when we go to community, that also means choice, which is a good thing. But, there is also a downside to choice. There are a lot of variables, many different things to look at. Tell us a little bit about the importance of profiling, and when you have got many new plug-ins to choose from, and you've got lots of commentary and social media being generated about what to use and what not to use.
Give us, Todd, if you could, some idea of the problem set that you saw in the marketplace a couple of years ago when you were thinking about Pulse.
Williams: Let me take a step back and quickly explain what Pulse is for those who aren't familiar with it. We built a general-purpose software provisioning system that right now we are targeting at the Eclipse market, specifically Eclipse developers.
For our initial release last November, we focused on providing a simple, intuitive way that you could install, update, and share custom configurations with Eclipse-based tools.
In Pulse 2, which is our current release, we have extended those capabilities to address what we like to call team-synchronization problems. That includes not only customized tool stacks, but also things like workspace project configurations and common preference settings.
Now you can have a team that stays effectively in lock step with both their tools and their workspaces and preferences.
What drove us to build something like this were a number of things. If you look at the Eclipse market, where we have been for a number of years, there are literally thousands of products and plug-ins for Eclipse. If you just want to go out and take a survey of them, or try some of them, it's a very daunting process for most people.
It starts out when you download Eclipse, go find some plug-ins, possibly looking into Eclipse Plug-in Central, find those update sites, type them in, download the plug-ins, and try them. This pattern repeats for quite some time, while the developer goes out and tries to figure out which of the plug-ins are good and which ones aren't.
With Pulse, we put these very popular, well-researched plug-ins into a catalog, so that you can configure these types of tool stacks with drag-and-drop. So, it's very easy to try new things. We also bring in some of the social aspects; pulling in the rankings and descriptions from other sources like Eclipse Plug-in Central and those types of things.
So, within Pulse, you have a very easy way to start out with some base technology stacks for certain kinds of development and you can easily augment them over time and then share them with others.
Gardner: Ken, help us understand how this can be used in the training and/or academic setting? What is it about Pulse that brings in more of the real world, and anticipates what choices developers are going to have once they get into the nitty-gritty of coding?
Kousen: Looking at academic and training settings, they are a little bit different. In a training setting, one of the real challenges the training classes face every time is getting the initial classroom set up correct. That is often very involved and complicated, because a lot of the tools involved are somewhat dependent on each other and dependent on environment variables and things like that.
So, trying to set up standard Pulse configurations and then being able to set up a classroom using those shared deployments is a very interesting opportunity. I haven't had the chance to do it yet, but I have definitely been talking to some training providers about giving that a shot.
I did try it in a classroom, and it's rather interesting, because one of the students that I had recently this year was coming from the Microsoft environment. I get a very common experience with Microsoft people, in that they are always overwhelmed by the fact, as Todd said, there are so many choices for everything. For Microsoft, there is always exactly one choice, and that choice costs $400.
I tried to tell them that here we have many, many choices, and the correct choice, or the most popular choice changes all the time. It can be very time consuming and overwhelming for them to try to decide which ones to use in which circumstances.
So, I set up a couple of configurations that I was able to share with the students. Once they were able to register and download them, they were able to get everything in a self-contained environment.
We found that pretty helpful, although I've got to say that this year the class size was sufficiently small, so that I don't know that we really got the same benefit we would get in a large classroom, where there would be many, many setup issues to deal with.
Gardner: So, almost mimicking a collaboration activity in a development setting, but in the classroom.
Kousen: Exactly.
Gardner: Are there any particular things that you learned from this exercise that those who might be evaluating and thinking about using Pulse could benefit from?
Kousen: It was pretty straightforward for everybody to use. We had to make sure that people using it had fast download speeds, but that had nothing to do with Pulse. That had to do with the size of Eclipse.
Of course, whenever you get students downloading configurations, they have this inevitable urge to start experimenting, trying to add in plug-ins, and replacing things. I did have one case where the configuration got pretty corrupted, not due to anything that they did in Pulse, but because of plug-ins they added externally. We just basically scrapped that one and started over and it came out very nicely. So, that was very helpful in that case.
Gardner: Michael, as you are listening to this, is there anything that jumps out at you in terms of understanding of Eclipse and its popularity, and then dealing with complexity that you could share?
Cote: I like the comparison of the Eclipse development world, versus visual studio, versus getting the one thing, because it is very accurate. That's sort of the ethos of Java -- maximum "choosability," if you will. It's one of these things in development that takes a long time to accept, but having lots of options is often more expensive and burdensome than having fewer options. Now that said, you want to make sure that you have good fewer options.
In every development team I have been involved with in my previous lives, as it were, anytime someone new comes onto the team, it’s always an extremely difficult issue just to get their tool chain setup correctly.
Having something wrong in the tool chain, the shared tools that the whole team uses, can really be quite disruptive. That's because the way that you assume your team members are going about solving problems is slightly wrong, and so they may not have the fully optimized way that your project is based around.
I guess you could call that the commercial application of that tediousness of setting up the configuration in more of an educational or a training environment. It's difficult to just sort of give someone a print out and say, go setup your stuff like this, because you are always missing little bits, and there is a lot of nuance in how things are exactly setup in the tool chains.
Gardner: Back to you, Todd at Genuitec. Have there been any surprises since you brought Pulse to market in how it’s being used? Are there unanticipated consequences that you would like to share -- the good ones anyway?
Williams: It's been interesting. We have seen a good number of people using Pulse, the way we anticipated it, sharing their tool stacks, and publishing them for their teams.
There seems to be a lot of people that use it privately. They don't share it with anyone, but they use it to manage multiple development profiles. So they might do C++ development one day and Java development the next, or what have you, and they like to keep custom tool stacks just for those things.
Even though they are kind of an island, and we made Pulse to share amongst teams, they find a lot of value in it, just to keep everything tidy.
Cote: If I can add to that, I personally haven't seen people using Pulse like this, because I haven't stuck my head in a developer shop when Pulse has been around. We would typically have a problem where -- across different versions of the project you are working on -- you would have your ID or your tools set up differently.
So, if you wanted to very quickly switch between those different versions, for example, to support or do some debugging in an old version, if there was some support issue, that switching cost between the two setups is a big part of going to fix an older version of something.
Nowadays, you have a lot of virtualization, so you can make this step a little easier, but you end up doing absurd things, like just having machines dedicated to specific versions of the software that you are working on.
If you can more easily switch between the profiles and the configurations that you have, then you can hopefully make it easier and less tedious to support these older products that you tend to have a lot of requests to support.
Gardner: Ken, did you see some advice that you might offer to those, either in academia or in the training field, things that they might want to consider as they are evaluating such things as Pulse?
Kousen: I agree with what the others were saying about the idea of setting up a series of alternative profiles that match the environment you are going to be working in.
I realized, as Michael and Todd were saying that, that I actually do that myself. I have a J2EE profile or Java EE profile, and I also have a regular Java profile, when I am working on different things, because there are certain shortcuts that won't conflict with anything in Java EE, if I use it in Java.
Eventually, I hope when you wind up adding Grails support or Groovy and Grails support to Pulse, it will probably have a configuration environment for that as well. The idea of having a variety of profiles that could each be used in its given time is very helpful.
I know that in a training environment we will definitely try to do that. We will be setting up alternative profiles that can be shared in a particular training class.
Academically, I like to leave things a bit more free form, although I agree that the initial setup is very helpful, because if the students don't have any feel for the environment at all, getting them over that initial hurdle is very, very helpful. After that, letting them experiment is always very, very useful. So that's good.
Gardner: Todd, Ken mentioned support for Ruby, dynamic languages, Groovy. Can you tip your hand a little bit and let us know what you've got in mind in that regard?
Williams: Actually, all of those things are in the Pulse catalog right now. Sometimes they are hard to find, because it's kind of big, but we added search to it to help you run them down. But, there are actually multiple Ruby solutions; I know Groovy is in there.
If a particular solution that you like isn't in there though, it's relatively straightforward to add it, not to the catalog, but you can still add it very, very easily to any of your profiles, either locally or shared.
So, the catalog is like a really good starting point that we try to keep up to date with what our users ask us to put into it. On the other hand, if it contains everything in the world, it gets a bit unwieldy as well.
Kousen: Dana, can I comment on that? I did speak very quickly on that issue. There is a Groovy plug-in in there. I was actually very pleased to see that, because I was concerned.
I've been using the Groovy plug-in for a while, and I wasn't sure whether that was going to be in the catalog at all. It did take me a while to find it, because it was filed under an area that I wasn't expecting, but once I put it in the search box, then it showed up immediately.
The only thing about Grails is that there isn't really a dedicated Grails plug-in yet, and the Groovy plug-in is really moving towards something like that. So, when that becomes available, I'm sure it will be incorporated into Pulse.
By the way, another issue that is very useful is that when I am browsing inside Pulse, just looking around to see what sort of components have been added, it's interesting to see what turns out to be popular; things that I hadn't really anticipated.
For example, I have been using Subclipse for the Subversion plug-in for a couple of years now. In browsing into the Version Control category I see that there are various other Subversion plug-ins as well and also others coming down the line. So that was another capability that I didn't anticipate and found rather interesting.
Gardner: Todd, looking forward a little bit, it seems that this profile information, while useful in a tactical sense, might actually have some strategic value too.
I'm thinking about the metadata that might be available through a profile and a definition of what a developer wants to do from an activity or behavioral or a pattern base. Then, applying that to when they do a search; perhaps refining the search based on their profile at that time, or perhaps using the profile in regard to when they do testing, bills, other aspects of lifecycle management for development.
Have you taken this a step further, where we could take these profiles and use them in a more strategic fashion, or is that something you are looking at?
Williams: That's a great question, Dana. Of course, we have a very large product plan for Pulse. We've only had it out since November, but you're right. We do have a lot of profile information, so if we chose to mine that data, we could find some correlations between the tools that people use, like some of the buying websites do.
People who buy this product also like this one, and we could make ad hoc recommendations, for example. It seems like most people that use Subversion also use Ruby or something, and you just point them to new things in the catalog. It's kind of a low-level way to add some value. So there are certainly some things under consideration.
Gardner: Michael, what do you think of that, taking more profile information; metadata about behaviors, uses, pattern of work, and then applying that to some of the other larger lifecycle activities in development?
Cote: Things like that work out really well, when you have the proliferation of choice that we were talking about earlier, where the systems can always be gained and everything.
This thing is a small enough subset that it doesn't happen, but just seeing sheer quantity-wise and rating-wise what people are using, helps you evaluate. I am probably making this figure up, but if there are 10 different unified modeling language (UML) plug-ins for Eclipse, then you need to somehow narrow down to the ones that are going to work out well for you.
The mixture of the fast and best way to get to that is really just to see which one is being used the most, because chances are, if people are still using it actively, its going to be a popular one. People are pretty fast to dump plug-ins that don't work well for them.
There is a place to capture the metadata or the usage data that's going around with things. That's the kind of thing that usually developers only get a chance to figure out when they are face to face with someone at a conference or other sort of events that don't happen as frequently as you might want to, to simply figure out which plug-in you might want to use.
Gardner: Any time you can take personalization information and automate that or refine searches and activities is certainly a productivity improvement, and Pulse really strikes me as setting up the opportunity to do that.
Cote: Absolutely.
Gardner: Alright. Let's stat wrapping up a little bit. Ken, any last thoughts as a technical trainer about where you would like to see this sort of capability go?
Kousen: I'm not exactly sure where I will be able to take advantage of it. Let me rephrase that. I think the current Pulse configuration is already very useful, and I'm not sure what else I need in order to start trying to incorporate it into an environment.
The only other issue that I wind up having in a training environment is setting things like environment variables onto the operating system. If there is some way we can get that into Eclipse for example, or rather into Pulse, rather than having to do it on the operating system itself -- maybe through the tools or whatever -- then that would be helpful. But I don't know. Right now, I think the situation is pretty good. I can't think of anything else concrete that I would want to add right there.
Gardner: Okay. Todd, thoughts about what educators and trainers should be considering as they look at something like Pulse, and how to exploit it and leverage it.
Williams: One thing that came to my mind, from a student's perspective, is the integrated development environments (IDEs) that are available right now; even the various configurations of Eclipse, are really made for professionals. When you take something like MyEclipse, there is just so much in it.
We need the ability to actually strip down the IDE to only what is needed for a particular exercise. For example, you could set up a profile for the first exercise of the class with just a limited set of tools that a new student would need to get their hands on. It limits the confusion factor. When you do the next exercise, you could easily update the profile; add a few additional tools to it.
So, you have kind of a selected discovery of additional tools and capabilities that coincide with the level of expertise the students are developing, as they are going up the learning curve in a particular course. I was just wondering. Is that the kind of thing that now we have enabled through having a technology like Pulse, that makes delivery of that straightforward, versus what had to be done before.
Gardner: Just for those interested in perhaps getting started, Pulse uses its network. How do people access that, how do they find it, how do you get started?
Williams: Sure. The Pulse website is www.poweredbypulse.com. There is a little 5 MB installer that you download and start running. If anyone is out in academia, and they want to use Pulse in a setting for a course, please fill out the contact page on the Website. Let us know, and we will be glad to help you with that. We really want to see usage in academia grow. We think it’s very useful. It's a free service, so please let us know, and we will be glad to help.
Gardner: Terrific. I want to thank our panelists for helping us dig a little bit into training issues, and some of the solutions that are welling up in the market to address them. We have been talking with Michael Cote, he is an analyst at RedMonk. Thank you Michael.
Cote: Absolutely.
Gardner: Todd Williams, vice president of technology, Genuitec. Appreciate your input, Todd.
Williams: Thanks, Dana, I have enjoyed it.
Gardner: Ken Kousen, independent technical trainer, president of Kousen IT, Inc., and adjunct professor at Rensselaer. Appreciate your experience and input, Ken.
Kousen: Oh, you are welcome, no problem.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You have been listening to a sponsored BriefingsDirect podcast. Thanks for listening, and come back next time.
Listen to the podcast. Sponsor: Genuitec.
Transcript of BriefingsDirect podcast on Java training and education with Genuitec Pulse for Java and Eclipse. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.
Listen to the podcast. Sponsor: Genuitec.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.
Today, a sponsored podcast discussion about the complexity around Java training and education. The development toolset, the plug-ins, the community are all very fast-moving targets. It's difficult for trainers, educators -- not to mention the students and budding developers themselves -- to get a full grasp of what's expected of them, and then to find resources that are up-to-date and timely.
We're going to be discussing with some experts how better to organize Java training and education. We're going to look at the Eclipse profiles that can be organized and coordinated using Pulse, a Genuitec-organized tools configuration network function. We're going to learn about how organizations can organize their training, so that students can better anticipate what's expected of them in the real world.
To help us understand some of these issues and work toward some solutions, we're joined by Michael Cote, an analyst with RedMonk. Welcome to the show, Michael.
Michael Cote: Hello, everybody.
Gardner: We're also joined by Todd Williams, vice president of technology at Genuitec. Welcome to the show, Todd.
Todd Williams: Thanks very much, Dana.
Gardner: Lastly, we're joined by Ken Kousen, an independent technical trainer and president of Kousen IT, Inc. He's also an adjunct professor at Rensselaer Polytechnic Institute. Welcome, Ken.
Ken Kousen: Hi, Dana, glad to be here.
Gardner: As I mentioned, complexity and moving targets are part of the problem, but it also seems that there is a disconnect between book knowledge or training knowledge that one gathers about development and what happens in the real world, what goes on with code being checked in and checked out, and how teams are organized. It seems difficult for someone to anticipate what's really going happen.
Let's go to Ken first. Ken, what is the gap, from your perspective, between what students and budding developers get through training and in university settings, and then what's often expected of them in the real world?
Kousen: It's interesting. The gap between what's taught in academia and what's taught in the real world is very large, actually. The classes I teach tend to be in a master's level program, and I teach a couple of classes in developing enterprise applications that are specifically constructed to address this gap.
Academia will talk about abstractions of data structures, algorithms, and different techniques for doing things. Then, when people get into the real world, they have no idea what Spring, Hibernate, or any of the other issues really are.
It's also interesting that a lot of developments in this field tend to flow from the working professionals toward academia, rather than the other way around, which is what you would find in engineering, when I used to be in that area.
Gardner: Todd, when you're doing hiring or are doing development for business, and you're talking to your customers -- folks that use MyEclipse and your services for training and consulting -- are you seeing a worsening situation in terms of how to acquire qualified labor, or do people have a pretty good sense of where to go to find good Java developers?
Williams: Finding quality employees is always a challenge, and probably always will be. Part of what I see as being difficult, especially in the Java and Enterprise Java market, is the huge number of technologies that are being employed at different levels. Each company picks its own type of stack.
Ken mentioned Spring and Hibernate. There is also Java transaction API (JTA), Java server faces (JSF), and Struts, Web framework and persistence technologies, and application servers that are in use. Finding employees that fit with what you are trying to do today, with an eye toward being able to mature them into where you are going tomorrow, is probably going to always be the concern.
Gardner: Now, what's been going on with development, not just the function, but teams, the collaboration, agile types of activities, Scrum? It used to be that people could specialize, stay in one little niche, but now the "master of all trades" seems to be more in demand.
Let's go to Michael. Michael, is development fundamentally changing? When we think of developers, do we need to recast how we imagine them or conceive of them?
Cote: Yes. I think it's fair even to go to the extreme and say absolutely. You look at the employment patterns that most developers find themselves in, and they are not really working at some place three, five, ten, even twenty years. It's not realistic. So, specializing in some technology that essentially binds you to a job isn't really an effective way to make sure you can pay your bills for the rest of your life.
You have to be able to pick up quickly any given technology or any stack, whether it’s new or old. Every company has their own stack that they are developing. You also have to remember that there is plenty of old existing software out there that no one really talks about anymore. People need to maintain and take care of it.
So, whether you are learning a new technology or an old technology, the role of the developer now, much more so in the past, is to be more of a generalist who can quickly learn anything without support from their employer.
You're not going to get a lot of slack to learn things in a given time, paid training, and things like that. You're pretty much left on your own, or there are always cheaper alternatives to go to.
So the heat is really on developers to be Type A people who are always seeking out the best option.
Gardner: Alright. Well, now that we have scared anyone from ever wanting to be a developer, Ken, help us get a little bit closer to earth. What can students do, what can professors or instructors do, to help get more of this real-world perspective into what they do in these courses and in this preparation?
Kousen: It's interesting that while the various tools and technologies evolve, some of the basic principles always hold pretty fast. I've taught this class several times and I have to say that every time I've taught it, it's been very, very different, but the overall architectural issues are pretty constant.
Plus, what seems to follow in the industry are various trends, like an increased emphasis on testing, for example, the recent rise in dynamic languages, and things like that. The idea of continually trying to follow what's going on in the marketplace and seeing what's interesting seems to be very helpful.
I also emphasize to the students that a good source of information is to find some of the better open-source projects, and not necessarily join them, use them, or do anything with them, but follow what they do and see those projects as the communal efforts of some of the best developers in the world.
So, if they all say, "Oh yeah, we obviously have to have this source-control mechanism," then maybe that's an interesting thing that should be looked at, or this particular bug reporting tool, or whatever. I often emphasis that particular direction as well.
Gardner: How about that, Todd? Are these open-source communities, these chat rooms, these forums, the real, practical lab that the students and developer should be looking towards?
Williams: I think to a degree that it's certainly a practical lab that students have easy access to. Obviously, in open source, whether it’s something like the Eclipse Foundation, Apache, or what have you, they make a very explicit effort to communicate what they are doing through either bug reports, mail lists, and discussion groups. So, it's an easy way to get involved as just a monitor of what's going on. I think you could learn quite a bit from just seeing how the interactions play out.
That's not exactly the same type of environment they would see inside closed-wall corporate development, simply because the goals are different. Less emphasis is put on external communications and more emphasis is put on getting quality software out the door extremely quickly. But, there are a lot of very good techniques and communication patterns to be learned in the open-source communities.
Gardner: Now, when we go to community, that also means choice, which is a good thing. But, there is also a downside to choice. There are a lot of variables, many different things to look at. Tell us a little bit about the importance of profiling, and when you have got many new plug-ins to choose from, and you've got lots of commentary and social media being generated about what to use and what not to use.
Give us, Todd, if you could, some idea of the problem set that you saw in the marketplace a couple of years ago when you were thinking about Pulse.
Williams: Let me take a step back and quickly explain what Pulse is for those who aren't familiar with it. We built a general-purpose software provisioning system that right now we are targeting at the Eclipse market, specifically Eclipse developers.
For our initial release last November, we focused on providing a simple, intuitive way that you could install, update, and share custom configurations with Eclipse-based tools.
In Pulse 2, which is our current release, we have extended those capabilities to address what we like to call team-synchronization problems. That includes not only customized tool stacks, but also things like workspace project configurations and common preference settings.
Now you can have a team that stays effectively in lock step with both their tools and their workspaces and preferences.
What drove us to build something like this were a number of things. If you look at the Eclipse market, where we have been for a number of years, there are literally thousands of products and plug-ins for Eclipse. If you just want to go out and take a survey of them, or try some of them, it's a very daunting process for most people.
It starts out when you download Eclipse, go find some plug-ins, possibly looking into Eclipse Plug-in Central, find those update sites, type them in, download the plug-ins, and try them. This pattern repeats for quite some time, while the developer goes out and tries to figure out which of the plug-ins are good and which ones aren't.
With Pulse, we put these very popular, well-researched plug-ins into a catalog, so that you can configure these types of tool stacks with drag-and-drop. So, it's very easy to try new things. We also bring in some of the social aspects; pulling in the rankings and descriptions from other sources like Eclipse Plug-in Central and those types of things.
So, within Pulse, you have a very easy way to start out with some base technology stacks for certain kinds of development and you can easily augment them over time and then share them with others.
Gardner: Ken, help us understand how this can be used in the training and/or academic setting? What is it about Pulse that brings in more of the real world, and anticipates what choices developers are going to have once they get into the nitty-gritty of coding?
Kousen: Looking at academic and training settings, they are a little bit different. In a training setting, one of the real challenges the training classes face every time is getting the initial classroom set up correct. That is often very involved and complicated, because a lot of the tools involved are somewhat dependent on each other and dependent on environment variables and things like that.
So, trying to set up standard Pulse configurations and then being able to set up a classroom using those shared deployments is a very interesting opportunity. I haven't had the chance to do it yet, but I have definitely been talking to some training providers about giving that a shot.
I did try it in a classroom, and it's rather interesting, because one of the students that I had recently this year was coming from the Microsoft environment. I get a very common experience with Microsoft people, in that they are always overwhelmed by the fact, as Todd said, there are so many choices for everything. For Microsoft, there is always exactly one choice, and that choice costs $400.
I tried to tell them that here we have many, many choices, and the correct choice, or the most popular choice changes all the time. It can be very time consuming and overwhelming for them to try to decide which ones to use in which circumstances.
So, I set up a couple of configurations that I was able to share with the students. Once they were able to register and download them, they were able to get everything in a self-contained environment.
We found that pretty helpful, although I've got to say that this year the class size was sufficiently small, so that I don't know that we really got the same benefit we would get in a large classroom, where there would be many, many setup issues to deal with.
Gardner: So, almost mimicking a collaboration activity in a development setting, but in the classroom.
Kousen: Exactly.
Gardner: Are there any particular things that you learned from this exercise that those who might be evaluating and thinking about using Pulse could benefit from?
Kousen: It was pretty straightforward for everybody to use. We had to make sure that people using it had fast download speeds, but that had nothing to do with Pulse. That had to do with the size of Eclipse.
Of course, whenever you get students downloading configurations, they have this inevitable urge to start experimenting, trying to add in plug-ins, and replacing things. I did have one case where the configuration got pretty corrupted, not due to anything that they did in Pulse, but because of plug-ins they added externally. We just basically scrapped that one and started over and it came out very nicely. So, that was very helpful in that case.
Gardner: Michael, as you are listening to this, is there anything that jumps out at you in terms of understanding of Eclipse and its popularity, and then dealing with complexity that you could share?
Cote: I like the comparison of the Eclipse development world, versus visual studio, versus getting the one thing, because it is very accurate. That's sort of the ethos of Java -- maximum "choosability," if you will. It's one of these things in development that takes a long time to accept, but having lots of options is often more expensive and burdensome than having fewer options. Now that said, you want to make sure that you have good fewer options.
In every development team I have been involved with in my previous lives, as it were, anytime someone new comes onto the team, it’s always an extremely difficult issue just to get their tool chain setup correctly.
Having something wrong in the tool chain, the shared tools that the whole team uses, can really be quite disruptive. That's because the way that you assume your team members are going about solving problems is slightly wrong, and so they may not have the fully optimized way that your project is based around.
I guess you could call that the commercial application of that tediousness of setting up the configuration in more of an educational or a training environment. It's difficult to just sort of give someone a print out and say, go setup your stuff like this, because you are always missing little bits, and there is a lot of nuance in how things are exactly setup in the tool chains.
Gardner: Back to you, Todd at Genuitec. Have there been any surprises since you brought Pulse to market in how it’s being used? Are there unanticipated consequences that you would like to share -- the good ones anyway?
Williams: It's been interesting. We have seen a good number of people using Pulse, the way we anticipated it, sharing their tool stacks, and publishing them for their teams.
There seems to be a lot of people that use it privately. They don't share it with anyone, but they use it to manage multiple development profiles. So they might do C++ development one day and Java development the next, or what have you, and they like to keep custom tool stacks just for those things.
Even though they are kind of an island, and we made Pulse to share amongst teams, they find a lot of value in it, just to keep everything tidy.
Cote: If I can add to that, I personally haven't seen people using Pulse like this, because I haven't stuck my head in a developer shop when Pulse has been around. We would typically have a problem where -- across different versions of the project you are working on -- you would have your ID or your tools set up differently.
So, if you wanted to very quickly switch between those different versions, for example, to support or do some debugging in an old version, if there was some support issue, that switching cost between the two setups is a big part of going to fix an older version of something.
Nowadays, you have a lot of virtualization, so you can make this step a little easier, but you end up doing absurd things, like just having machines dedicated to specific versions of the software that you are working on.
If you can more easily switch between the profiles and the configurations that you have, then you can hopefully make it easier and less tedious to support these older products that you tend to have a lot of requests to support.
Gardner: Ken, did you see some advice that you might offer to those, either in academia or in the training field, things that they might want to consider as they are evaluating such things as Pulse?
Kousen: I agree with what the others were saying about the idea of setting up a series of alternative profiles that match the environment you are going to be working in.
I realized, as Michael and Todd were saying that, that I actually do that myself. I have a J2EE profile or Java EE profile, and I also have a regular Java profile, when I am working on different things, because there are certain shortcuts that won't conflict with anything in Java EE, if I use it in Java.
Eventually, I hope when you wind up adding Grails support or Groovy and Grails support to Pulse, it will probably have a configuration environment for that as well. The idea of having a variety of profiles that could each be used in its given time is very helpful.
I know that in a training environment we will definitely try to do that. We will be setting up alternative profiles that can be shared in a particular training class.
Academically, I like to leave things a bit more free form, although I agree that the initial setup is very helpful, because if the students don't have any feel for the environment at all, getting them over that initial hurdle is very, very helpful. After that, letting them experiment is always very, very useful. So that's good.
Gardner: Todd, Ken mentioned support for Ruby, dynamic languages, Groovy. Can you tip your hand a little bit and let us know what you've got in mind in that regard?
Williams: Actually, all of those things are in the Pulse catalog right now. Sometimes they are hard to find, because it's kind of big, but we added search to it to help you run them down. But, there are actually multiple Ruby solutions; I know Groovy is in there.
If a particular solution that you like isn't in there though, it's relatively straightforward to add it, not to the catalog, but you can still add it very, very easily to any of your profiles, either locally or shared.
So, the catalog is like a really good starting point that we try to keep up to date with what our users ask us to put into it. On the other hand, if it contains everything in the world, it gets a bit unwieldy as well.
Kousen: Dana, can I comment on that? I did speak very quickly on that issue. There is a Groovy plug-in in there. I was actually very pleased to see that, because I was concerned.
I've been using the Groovy plug-in for a while, and I wasn't sure whether that was going to be in the catalog at all. It did take me a while to find it, because it was filed under an area that I wasn't expecting, but once I put it in the search box, then it showed up immediately.
The only thing about Grails is that there isn't really a dedicated Grails plug-in yet, and the Groovy plug-in is really moving towards something like that. So, when that becomes available, I'm sure it will be incorporated into Pulse.
By the way, another issue that is very useful is that when I am browsing inside Pulse, just looking around to see what sort of components have been added, it's interesting to see what turns out to be popular; things that I hadn't really anticipated.
For example, I have been using Subclipse for the Subversion plug-in for a couple of years now. In browsing into the Version Control category I see that there are various other Subversion plug-ins as well and also others coming down the line. So that was another capability that I didn't anticipate and found rather interesting.
Gardner: Todd, looking forward a little bit, it seems that this profile information, while useful in a tactical sense, might actually have some strategic value too.
I'm thinking about the metadata that might be available through a profile and a definition of what a developer wants to do from an activity or behavioral or a pattern base. Then, applying that to when they do a search; perhaps refining the search based on their profile at that time, or perhaps using the profile in regard to when they do testing, bills, other aspects of lifecycle management for development.
Have you taken this a step further, where we could take these profiles and use them in a more strategic fashion, or is that something you are looking at?
Williams: That's a great question, Dana. Of course, we have a very large product plan for Pulse. We've only had it out since November, but you're right. We do have a lot of profile information, so if we chose to mine that data, we could find some correlations between the tools that people use, like some of the buying websites do.
People who buy this product also like this one, and we could make ad hoc recommendations, for example. It seems like most people that use Subversion also use Ruby or something, and you just point them to new things in the catalog. It's kind of a low-level way to add some value. So there are certainly some things under consideration.
Gardner: Michael, what do you think of that, taking more profile information; metadata about behaviors, uses, pattern of work, and then applying that to some of the other larger lifecycle activities in development?
Cote: Things like that work out really well, when you have the proliferation of choice that we were talking about earlier, where the systems can always be gained and everything.
This thing is a small enough subset that it doesn't happen, but just seeing sheer quantity-wise and rating-wise what people are using, helps you evaluate. I am probably making this figure up, but if there are 10 different unified modeling language (UML) plug-ins for Eclipse, then you need to somehow narrow down to the ones that are going to work out well for you.
The mixture of the fast and best way to get to that is really just to see which one is being used the most, because chances are, if people are still using it actively, its going to be a popular one. People are pretty fast to dump plug-ins that don't work well for them.
There is a place to capture the metadata or the usage data that's going around with things. That's the kind of thing that usually developers only get a chance to figure out when they are face to face with someone at a conference or other sort of events that don't happen as frequently as you might want to, to simply figure out which plug-in you might want to use.
Gardner: Any time you can take personalization information and automate that or refine searches and activities is certainly a productivity improvement, and Pulse really strikes me as setting up the opportunity to do that.
Cote: Absolutely.
Gardner: Alright. Let's stat wrapping up a little bit. Ken, any last thoughts as a technical trainer about where you would like to see this sort of capability go?
Kousen: I'm not exactly sure where I will be able to take advantage of it. Let me rephrase that. I think the current Pulse configuration is already very useful, and I'm not sure what else I need in order to start trying to incorporate it into an environment.
The only other issue that I wind up having in a training environment is setting things like environment variables onto the operating system. If there is some way we can get that into Eclipse for example, or rather into Pulse, rather than having to do it on the operating system itself -- maybe through the tools or whatever -- then that would be helpful. But I don't know. Right now, I think the situation is pretty good. I can't think of anything else concrete that I would want to add right there.
Gardner: Okay. Todd, thoughts about what educators and trainers should be considering as they look at something like Pulse, and how to exploit it and leverage it.
Williams: One thing that came to my mind, from a student's perspective, is the integrated development environments (IDEs) that are available right now; even the various configurations of Eclipse, are really made for professionals. When you take something like MyEclipse, there is just so much in it.
We need the ability to actually strip down the IDE to only what is needed for a particular exercise. For example, you could set up a profile for the first exercise of the class with just a limited set of tools that a new student would need to get their hands on. It limits the confusion factor. When you do the next exercise, you could easily update the profile; add a few additional tools to it.
So, you have kind of a selected discovery of additional tools and capabilities that coincide with the level of expertise the students are developing, as they are going up the learning curve in a particular course. I was just wondering. Is that the kind of thing that now we have enabled through having a technology like Pulse, that makes delivery of that straightforward, versus what had to be done before.
Gardner: Just for those interested in perhaps getting started, Pulse uses its network. How do people access that, how do they find it, how do you get started?
Williams: Sure. The Pulse website is www.poweredbypulse.com. There is a little 5 MB installer that you download and start running. If anyone is out in academia, and they want to use Pulse in a setting for a course, please fill out the contact page on the Website. Let us know, and we will be glad to help you with that. We really want to see usage in academia grow. We think it’s very useful. It's a free service, so please let us know, and we will be glad to help.
Gardner: Terrific. I want to thank our panelists for helping us dig a little bit into training issues, and some of the solutions that are welling up in the market to address them. We have been talking with Michael Cote, he is an analyst at RedMonk. Thank you Michael.
Cote: Absolutely.
Gardner: Todd Williams, vice president of technology, Genuitec. Appreciate your input, Todd.
Williams: Thanks, Dana, I have enjoyed it.
Gardner: Ken Kousen, independent technical trainer, president of Kousen IT, Inc., and adjunct professor at Rensselaer. Appreciate your experience and input, Ken.
Kousen: Oh, you are welcome, no problem.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You have been listening to a sponsored BriefingsDirect podcast. Thanks for listening, and come back next time.
Listen to the podcast. Sponsor: Genuitec.
Transcript of BriefingsDirect podcast on Java training and education with Genuitec Pulse for Java and Eclipse. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.
Wednesday, August 13, 2008
Borland's Own ‘Journey' to Agile Development Forms Real-World Foundation for New Software Delivery Management Solutions
Transcript of BriefingsDirect podcast on Agile Development principles and practices with Borland Software.
Listen to the podcast. Sponsor: Borland Software.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today we present a sponsored podcast discussion about Agile software development.
We're going to be talking to a software executive from Borland Software about Borland's own Agile “journey.” They deployed Agile practices and enjoyed benefits from that, as well as gained many lessons learned, as they built out their latest application lifecycle management (ALM) products. [See product and solution rundowns.]
We're going to talk with Pete Morowski, the senior vice president of research and development (R&D) at Borland Software. Welcome to the show, Pete.
Peter Morowski: Thank you, Dana. It's good to be here.
Gardner: Before you get into Borland Software's journey, I want to get a level-set about Agile Development practices in general. Why is Agile development a good idea now? What is it about the atmosphere in the evolution of development that makes this timely?
Morowski: From the standpoint of software development, it's a realization that development is an empirical process, a process of discovery. Look at the late delivery cycles that traditional waterfall methodologies have brought upon us. Products have been delivered and end up on the shelf. The principles behind Agile right now allow teams to deliver on a much more frequent cycle and also to deliver more focused releases.
Gardner: There are also, I suppose, technical and business drivers: better quality, faster turnaround, more complexity, and, of course, distributed teams. What is it about the combination? Why is this important now in terms of some of these other technical business and even economic imperatives?
Morowski: With the advent of Web applications, businesses really expect a quicker turnaround time. In addition, when you look at cost structures, the time spent on features not used and other things are critical business inhibitors at this point.
Gardner: Let's help out some folks out who might not be that familiar with Agile and its associated process called Scrum. Tell us a little bit from an elevator-pitch perspective. What is Agile and what is Scrum?
Morowski: Agile really is a set of principles, and these principles are based on things like self-directed teams, using working code as a measure of progress, and also looking at software development in terms of iteration. What we mean by that is that when you look at traditional software development, we talked about things like design, code, and testing as actual phases in a development lifecycle. Within Agile, in an iteration, these are just activities that occur in each iteration.
Now, when you talk about Scrum, that is more of a process and a methodology. This is actually taking those Agile principles and then being more prescriptive on how to apply them to a software-development cycle.
In the case of Scrum, it's based upon a concept called a sprint, which is a two-to-four week iteration that the team plans for and then executes. In that two-to-four weeks, whatever they get done is considered completed during that sprint, and what work hadn't been completed goes into what they call "product backlog" for prioritization on what is done in the next sprint. You chain these several iterations together for a release.
The beauty of this is that now you have a way to induce change on the borders of those iterations. So, one of the things that's really advantageous to Agile is its ability to adapt the changing requirements.
Gardner: When I try to explain Agile to people, some of them come away thinking that it's an oxymoron or is conflicted because they say, "Okay, your goal is to do things better and faster, but you are telling people use fewer rules, use less structure, and have your teams be self-selecting." People see a conflict here. Why isn't that a conflict?
Morowski: I think it's a misnomer that self-directed teams and that type of thing mean that we can do whatever we want. What it's really about is that teams begin to take ownership for delivering the product. What happens is that, by allowing these teams to become self-directed, they own the schedule for delivery.
What happens is that you see some things like traditional breakdowns of roles, where they are looking at what work needs to be finished in a sprint, versus "Well, I am a developer. I don't do testing," or "I am a doc writer, and I can't contribute on requirements," and those types of things. It really builds a team, which makes it a much more efficient use of resources and processes, and you end up with better results than you do in a traditional methodology.
Gardner: It almost sounds like we're using market forces, whereby entrepreneurs or small startups tend to be more energized and focused than teams within a larger, centralized organization. Is that a fair characterization?
Morowski: Yeah, I think it is very fair.
Gardner: And, given that we're looking for this empirical learn-as-you-go, do what's right for you, I suppose that also means that one size does not fit all. So, Agile would probably look very different from organization to organization.
Morowski: It could. One thing we chose to do, though, was to really to set a benchmark process. So, when Borland first started developing in Agile, we had multiple locations, and each site was, in essence, developing its own culture around Agile. What I found was that we were getting into discussions about whose Agile was more pure and things like that, and so I decided to develop a Borland Agile culture. [See case study on Borland and Agile.]
We broke that up on geographic bases, where we started with one site, had one "ScrumMaster" and we built what we call the reference process. As we've grown, and our projects are getting more complex, the fact that we evolve from site-to-site based on the same process and the same terminology has allowed us to now choose more complex agile techniques like Scrum of Scrums or work across organizations, and have a common vocabulary, and that kind of common way of working.
Gardner: It also sounds like you are taking the best of what a centralized approach offers and the best of what a decentralized approach offers, in terms of incentive; take charge, and local ownership, and then making them co-exist.
Morowski: That's correct.
Gardner: All right, let's get specifically into Borland's situation. What is it about the way that Borland has been developing software, which is of course a core competency for a large independent software vendor (ISV) like yourselves, and it has been for 15-plus years … How difficult was it for you to come into this established organization and shake things up?
Morowski: Initially, it wasn't an issue because, like most organizations, when we went through and looked at it, there were a couple of grassroots efforts underway. From an Agile perspective, one of the things we did was to begin to leverage that activity and the successes that it had to use as a benchmark with other teams. As we grew and moved into other organizations that were not necessarily grassroots efforts, there were some challenges.
Gardner: So, it might be quite possible that lot of organizations that do development have people who are Agile-minded and perhaps even followers of Agile doing this already. Perhaps they should look for those and start there.
Morowski: I would recommend that you start with your grassroots efforts, establish your benchmark process, and then begin to move out from there.
One thing we clearly did was, once that we saw the benefits of doing this, we had a lot of executive sponsorship around this. I made it one of the goals for the year to expand our use of Agile within the organization, so that teams knew it was safe to go ahead and begin to look at it. In addition, because we had a reference implementation of it, it also gave teams a starting point to begin their experimentation. We also paid for our teams to undergo training and those types of things. We created an environment that encouraged transformation.
Gardner: Let's learn a little bit more about you, Pete. Tell us a little bit about your background and how you came into development and then into Agile?
Morowski: I've been in this business a little over 25 years now. I started in the defense and aerospace industries and then moved into commercial ISVs later in my career. I've been an executive at Novell. I've also been a CTO at IBM Tivoli, and prior to Borland, was the vice president of software at Dell.
Gardner: You've taken on this Agile project at Borland, and you've written a paper on the “Borland Agile Journey.” I've had the pleasure of reading it. I think it's a really nice read and I commend for you it.
Morowski: Oh, thank you.
Gardner: Tell us about this particular product set [Borland Software Delivery Management information] that Borland is coming out with. It's a product set about helping people develop software. Is there a commonality between some of the lessons you learned and then what you may have actually visited in terms of requirements for your products? [See demo and see launch video.]
Morowski: Oh, absolutely. One of the interesting things about the products that we are delivering is that one of them is a product for managing Agile development, especially in distributed teams and managing the requirements. So, we had the advantage of actually using the tools as we were developing them.
Now, we were also very cautious because you can get myopic about that type of thing, where we also using Agile principles, and we involved our customers in the process, as well. So we were getting kind of the best of both worlds.
Gardner: What makes software development different? In reading your paper, I was thinking about how these principles about self-empowerment and working quickly and then setting these boundaries -- "Okay, we're going to just work and do this for three weeks and then will revisit any changes," -- that might be something it would apply to almost any creative activity where a team is involved.
Is Agile something you think applies to any creative activity, a complex team-based activity, or is there something about it that really is specific and germane to software development?
Morowski: If you look at Agile principles, conceptually, they do apply to a lot of things. Anything in which you are going into a period of discovery, one of the key things is knowing what your goal or mission is. In the case of software, that's a requirement, and what you want the product to be.
But in any kind of empirically based endeavor, this would be something that you could apply. Now, when you get down to the actual Scrum process itself, it's the terminology, the measures, the metrics, and all those types of things are really tailored for software development.
Gardner: When I read your paper, I also came away with some interesting observations. You say, there is a difference between how development is supposed to work and how it actually works. It's sounds like many companies are living in denial or a certain level of dysfunction that they are not necessarily facing.
Morowski: It's one of the issues with laying a manufacturing process over something that's inherently an empirical process. In the end, all software R&D organizations or IT shops responsible for applications are responsible to the business for delivering results. And, in doing so, we all try to measure those things.
What I have observed over my career was the fact that there really existed two worlds. There is what I call the "management plane," and this is a plane of milestones, of phase transitions and a very orderly process and progress through the software development lifecycle.
Underneath it though, in reality, is a world of chaos. It's a world of rework, a world of discovery, in which the engineers, testers and frontline managers live. We traditionally use Gantt as a measure that is task-based. It requires a translation from the implementation world to the management world to show indications of progress. Any time you do a translation, there can be a loss of information, and that's why today software is such an experienced-based endeavor.
Gardner: And it's often been perceived as sort of a dark art. People don't appreciate or understand how it's done, and that those who do it should say, "Hey, leave me alone, get away from me. I'll come back with the results in three months."
Morowski: Exactly.
Gardner: But that doesn't necessarily or hasn't historically been the best approach.
Morowski: Absolutely not.
Gardner: Also, at times, you see them downplay process and say that doing good hiring probably is the biggest issue here. What's the relationship between hiring and what people, not always affectionately, refer to as human resources? What's the relationship between HR and Agile?
Morowski: Well, first of all, just getting back to a little bit on hiring thing. Hiring is important, regardless of what methodology you use, and I tend to stress that. I do contend there are different kinds of personalities and skill sets you are going to be looking for when you are building Agile teams, and it's important to highlight those.
It's very important that whoever comes onboard in Agile team is collaborative in nature. In traditional software environments, there are two roles that traditionally you may struggle with, and you have to look at them closely. One is the manager. If a manager is a micromanager-type, that's not going to work in an Agile environment.
And, the other thing, interestingly enough, is the chief architect role. What's interesting about that is that you would think you would fit in Agile very easily, but in a lot of traditional software organizations, all decisions of a technical nature on a project go through the chief architect. In an Agile world, it's much more collaborative and everybody contributes. So for some personalities, this would be a difficult change for them.
Gardner: So there is that grassroots element, and you have to be open to it.
Morowski: Right.
Gardner: What is it about the structures here? Again, for folks who might not be that familiar with Agile, tell us a little bit about some of the hierarchy.
Morowski: There are really two key roles. There is the ScrumMaster and the ScrumMaster runs what they call the daily stand-up. This is basically a meeting, where everybody on the team gets together on a daily basis and they answer three questions. "What did I get accomplished yesterday?" "What am I going to do today?" And "What's blocking me?"
Everybody goes around the room. It's a 15- minute meeting. You solve any particular problems, but you log things. The role of ScrumMaster is to run that meeting and to remove blocks to the team, and it's a very key role.
The second major role within Scrum is really the product owner, and this is the individual that's responsible for prioritizing the requirements or what we call the product backlog -- what is what is going to be done during the sprint, which features are going to be completed. Those are the two primary roles, and then from there everybody is pretty much a team member.
Gardner: When you decided to bring this into play at Borland, a very large, distributed organization, you didn't try to bite off too much. You didn't say, "We are going to transform the entire company and organization." You did this on more of an iterative basis. It seems that most people, when they do Agile, will probably follow similar path. They'll take a project basis and then say, "Now we need to expand this and make it holistic."
Many organizations, however, across all kinds of different management activities, can stumble at that transition from the project, or the tactical, into the holistic, or general, across an organization. What did you learn in making this transition from small to large scale at Borland?
Morowski: A couple things. One is that, as we rolled it out, let's say starting by site-by-site, we grew from teams-to-teams. The ScrumMasters worked very collaboratively to help each other out, because, in the end, they were responsible for delivering at the end of those sprints. That was a very positive effect.
As we moved out to distributed teams, there were a number of challenges, things like the daily stand-up, or if I have people in Singapore that are supporting a particular sprint, say, from the system testing point, that made things difficult. But, what I found is the team was pretty creative in involving those individuals, whether they recorded sprints, whether they shifted time zones, and they did this all on their own.
That was the absolute positive, one of the things that surprised me. It was an interesting discovery.
As we started to be more broad with the interaction with the non-Agile parts of the organization, this was a little bit more of a challenge, and I learned a couple of things. In doing any kind outsourcing, if you try to match a traditional, contractual base -- the statement of work (SOW)-type outsourcer -- with an agile team, that's going to present problems. The outsourcer is expecting very detailed specifications as a statement of work and that's just not produced during an agile or sprint/Scrum type of development activity.
The other thing is internally, and what I would say at the end of the pipe and at the beginning of the pipe, working with marketing and our new product introduction processes and support and getting sales out. One of the things that we found is that we started to have a capacity to release more often, but the organization, as a whole, had to adjust now to: A) provide market requirements to us in a different manner, and B) we had to adjust our process at the end to be able to accept more rapid releases.
Gardner: So in order to get the most out of Agile, it sounds like, for those organizations where software development is core competencies, important to their successes as a company, or as a government organization, or a public not-for-profit, that the edges of Agile start to blend into other departments. The whole competency around their organization can perhaps borrow some of these principles from development and extend them into the entire lifecycle.
Morowski: Yes, we no longer look at it as strictly an R&D thing anymore, just because of that. And, it's interesting. You know you are making progress from a development team perspective, when you are starting to output more than the organization can accept.
Gardner: Interesting. So, adjustments along the way, and that's again a principle of the approach.
All right. In this age of Agile and your Agile journey, you came away with three basic observations about the benefits. One was around self-directing teams; second around being able to manage change well; and, third, about how to do the relationship with the customer, in this case the customer being the folks who are interested in getting the software. Tell us about these three benefits and what you have learned?
Morowski: Well, we touched on the self-directing teams, and the key to that is one of the most important things as an executive is that you really have to take the lead and let your teams go and develop -- let them truly own their projects. There will be mistakes along the way, but once they do, it's an extremely powerful concept.
One of the great things about agile is that it's a very open and very visible methodology. During daily stand-ups, I can attend any daily stand-up and sit there and listen to what's going on. I can't contribute in those meetings, because that's run by the ScrumMaster. But, one of the times I was attending the daily stand-up, I knew the teams had progressed a great deal.
When they were looking at their remaining work backlog that they had for that particular sprint, and there were a couple of tests that need to be run that there was nobody assigned to. One of the developers had time, looked at that, and picked it up.
Now, normally, that would never happen, because we behave in a silo fashion. "I am an engineer." "I am a tester." It's an "I am a …" type of thing. But, when you really have a self-directing team, the team owns that schedule and it's very interested in making sure that they meet their commitments.
Gardner: I suppose that also fosters willingness of people to move in and out of role, without just saying, "Well, that's not my job …", but taking more group responsibility, and even as an individual.
Morowski: Absolutely correct, and that to me has been one of the more powerful things that I have personally observed.
Gardner: Change management has often been something that drives developers crazy. They hate when people come in and start changing requirements when they are in the middle of doing code or test. On the other hand, things don't stay the same, and change is part of everything in life and business, perhaps more so today than ever. How do you reconcile those two?
Morowski: Well, I think the reality is that there is going to be change during these development cycles, and so the question is what's the best way to handle it? If you look in a traditional waterfall methodology, you march along phase transitions. Even if you have iteration in place, if you discover a design or coding defect late in the game, you have to go backwards to a different phase and start going into the design or fixing the code. Then, you repeat the process again, and you continue to move along your space transition line.
The thing that's interesting is that with Agile you have an orderly way of injecting change. In other words, as a sprint completes and you've demonstrated the code -- and you demonstrate it after that three-week iteration -- if something has changed and you need to change the prioritization, you have a way to inject that change along that boundary, and then let the team go forward. That's what I always like to say, "We're always going forward in Agile."
Gardner: And how do the teams adjust to that?
Morowski: It's part of the process. The changes go into the backlog. The product owner looks at them and then prioritizes it based upon the complexity of the work and the timing and so on and so forth, and just how important that is. If it's important enough, it will go into the next iteration. The teams are used to doing that, because you are not, in essence, disrupting at a random point. They have already finished what work they were working on, and now there is a cleaner opportunity to inject that change.
Gardner: So, boundaries allow for those who want change to get it done without having to wait for a particularly long period of time or until the project is done. But, for those involved in the project, they have these sections where it's not going to become chaotic and they are not going to lose track of their overall process, because of this injection of change.
Morowski: No, as a matter of fact, the process encourages it.
Gardner: How about this, what you call customer relationships? It sound to me as thought it's just being transparent.
Morowski: It is. It's a different approach, in the sense that you are actually bringing in the customer as what I would call a partner in the development. They participate in sprint reviews, and sprint reviews at the end of a sprint, where you show the working code, what you have completed and so. Those are done on an every-three-week basis, and we involve our customers.
They also take early drops of the code and provide input into the product backlog on requests that they want, and things like that. It's proven to be very beneficial for us. The one thing is that, when you choose these customers to participate, it's important for them to be Agile, as well, and understand that, and they need to approach this as a partnership not just an opportunity to get their particular features or requirements in.
Gardner: And, that must also help keep expectations in line, right?
Morowski: Absolutely. What I have found is that the customers we have involved want to get used to our cycles and our delivery rhythm. They are less adamant about getting every feature on a list in a particular release, because they know it's a relatively short time before the next one comes around.
Gardner: When we describe these customers, would that, in many organizations, include bringing the marketing people in, and the salespeople. Can they get involved so that this becomes something that will enter the market as an agile activity, rather than having Agile happening on the development side, and then falling back into a waterfall mentality when it comes to the go-to-market activities?
Morowski: Yes, we do, and the transparency that's there actually helps build confidence in the rest of the organization on what we are delivering, because they see it as we progress along. It's not something that mysteriously shows up on their doorstep.
Gardner: It certainly sounds great in theory, and it sounds like you've been able to accomplish quite a bit in practice, but what about metrics of success? How have you been able to say, "it works?" Has Borland cut their cost, their time to development? Do they have better products? All of the above? How do we know we are succeeding?
Morowski: I'd say it's combination of all the above. The first thing is that by putting these teams together, they are much smaller teams than in traditional organizations. So, if you look at it, my teams are almost 30 percent smaller on the Agile side than they are on the traditional side.
Gardner: And what's accounting for that change?
Morowski: I think one, is the ownerships of the teams, and two, the breakdown of very specific roles.
Gardner: Would I be going out on a limb in saying you have eliminated the middle management factor?
Morowski: There is absolutely that as well. The other thing is the fact that we're delivering working code and involving with customers. We are developing fewer superfluous features. When a product goes out the door, it generally has the most important features that were entailed for this release. So, it really helps the prioritization process.
Gardner: Not too many cooks in the kitchen?
Morowski: Exactly.
Gardner: Cool! Tell us a little bit about what surprised you the most about this Agile journey of Borland.
Morowski: I think the power of the daily stand-up. I mean, yes, we got a lot of benefits, and yes, we had a number of successes, we were able to transition code from locations and things like that, but I owe that a lot to the daily stand-up. The thing that surprised me is how powerful it is each morning when everybody gets around the table and actually goes through what they've done, basically saying, "Have I lived up to my commitments? What I am committing to the team today? And then is there anything blocking?"
Generally speaking, a lot of developers tend to be quiet and not the most social. What this did is that it just didn't allow the few people who were social to have input on what was going on. This daily stand-up had people, everybody on the team, contributing, and it really changed the relationships in the team. It was just a very, very powerful thing.
Gardner: It sounds like balance among personality types, but that balance directed toward the real activity that is developing code.
Morowski: Absolutely.
Gardner: Interesting! Well, congratulations. I enjoyed reading your paper, and this certainly sounds like the future of development, I know that's what many people in the business think. We've been talking about Agile development practices and principles and how Borland Software has been undertaking an Agile journey itself, in a development project around development process tools and application lifecycle management products.
Back to those products. Is there anything about the synergy between doing it this way and then presenting products into the field that you think will help other people engage with Agile benefits?
Morowski: Are you talking about the products themselves?
Gardner: Yes.
Morowski: The products themselves, absolutely. We have a product coming out called Team Analytics. The key to this is that, while we talked about self-directed teams, we still have responsibilities to reporting to the business and how we are progressing.
Team Analytics gives us a view into the process, gives us the ability to go ahead and look at how the team is progressing, and those types of things, what features have been included or dropped, without having to go into the team and request that information. So that's a very powerful thing.
Gardner: Right. So, it's one thing to agree that visibility and transparency are good, but it's another to actually accomplish it in terms of complexity in large teams and hierarchy.
Morowski: Absolutely. This allows us to move to what I call a "monitored" from a "reported" kind of methodology of metrics. What I mean by that is, typically, at the senior vice president or vice president level, you really get to look at the state of your products once a month, in the sense that you have operations reviews or some kind of review cycle where all your teams come in and then they report the progress of what's going on.
With Team Analytics, you are able to actually look at that on a daily basis and see if anything’s changed over time. That way, you know where you need to spend your time and that's why we call it monitored, at this point.
Gardner: Super! Well, thank you for sharing your insights. I think there is a lot to be taken away here and learned.
We have been talking with Pete Morowski, the senior vice president of research and development for Borland Software. We were looking at Agile principles in the context of Borland's Agile journey.
Thanks, Pete.
Morowski: Thank you, Dana.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions, and you’ve been listening to a sponsored BriefingsDirect podcast.
Thanks for joining us and come back next time.
Listen to the podcast. Sponsor: Borland Software.
Transcript of BriefingsDirect podcast on Agile development principles with Borland Software. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.
Listen to the podcast. Sponsor: Borland Software.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today we present a sponsored podcast discussion about Agile software development.
We're going to be talking to a software executive from Borland Software about Borland's own Agile “journey.” They deployed Agile practices and enjoyed benefits from that, as well as gained many lessons learned, as they built out their latest application lifecycle management (ALM) products. [See product and solution rundowns.]
We're going to talk with Pete Morowski, the senior vice president of research and development (R&D) at Borland Software. Welcome to the show, Pete.
Peter Morowski: Thank you, Dana. It's good to be here.
Gardner: Before you get into Borland Software's journey, I want to get a level-set about Agile Development practices in general. Why is Agile development a good idea now? What is it about the atmosphere in the evolution of development that makes this timely?
Morowski: From the standpoint of software development, it's a realization that development is an empirical process, a process of discovery. Look at the late delivery cycles that traditional waterfall methodologies have brought upon us. Products have been delivered and end up on the shelf. The principles behind Agile right now allow teams to deliver on a much more frequent cycle and also to deliver more focused releases.
Gardner: There are also, I suppose, technical and business drivers: better quality, faster turnaround, more complexity, and, of course, distributed teams. What is it about the combination? Why is this important now in terms of some of these other technical business and even economic imperatives?
Morowski: With the advent of Web applications, businesses really expect a quicker turnaround time. In addition, when you look at cost structures, the time spent on features not used and other things are critical business inhibitors at this point.
Gardner: Let's help out some folks out who might not be that familiar with Agile and its associated process called Scrum. Tell us a little bit from an elevator-pitch perspective. What is Agile and what is Scrum?
Morowski: Agile really is a set of principles, and these principles are based on things like self-directed teams, using working code as a measure of progress, and also looking at software development in terms of iteration. What we mean by that is that when you look at traditional software development, we talked about things like design, code, and testing as actual phases in a development lifecycle. Within Agile, in an iteration, these are just activities that occur in each iteration.
Now, when you talk about Scrum, that is more of a process and a methodology. This is actually taking those Agile principles and then being more prescriptive on how to apply them to a software-development cycle.
In the case of Scrum, it's based upon a concept called a sprint, which is a two-to-four week iteration that the team plans for and then executes. In that two-to-four weeks, whatever they get done is considered completed during that sprint, and what work hadn't been completed goes into what they call "product backlog" for prioritization on what is done in the next sprint. You chain these several iterations together for a release.
The beauty of this is that now you have a way to induce change on the borders of those iterations. So, one of the things that's really advantageous to Agile is its ability to adapt the changing requirements.
Gardner: When I try to explain Agile to people, some of them come away thinking that it's an oxymoron or is conflicted because they say, "Okay, your goal is to do things better and faster, but you are telling people use fewer rules, use less structure, and have your teams be self-selecting." People see a conflict here. Why isn't that a conflict?
Morowski: I think it's a misnomer that self-directed teams and that type of thing mean that we can do whatever we want. What it's really about is that teams begin to take ownership for delivering the product. What happens is that, by allowing these teams to become self-directed, they own the schedule for delivery.
What happens is that you see some things like traditional breakdowns of roles, where they are looking at what work needs to be finished in a sprint, versus "Well, I am a developer. I don't do testing," or "I am a doc writer, and I can't contribute on requirements," and those types of things. It really builds a team, which makes it a much more efficient use of resources and processes, and you end up with better results than you do in a traditional methodology.
Gardner: It almost sounds like we're using market forces, whereby entrepreneurs or small startups tend to be more energized and focused than teams within a larger, centralized organization. Is that a fair characterization?
Morowski: Yeah, I think it is very fair.
Gardner: And, given that we're looking for this empirical learn-as-you-go, do what's right for you, I suppose that also means that one size does not fit all. So, Agile would probably look very different from organization to organization.
Morowski: It could. One thing we chose to do, though, was to really to set a benchmark process. So, when Borland first started developing in Agile, we had multiple locations, and each site was, in essence, developing its own culture around Agile. What I found was that we were getting into discussions about whose Agile was more pure and things like that, and so I decided to develop a Borland Agile culture. [See case study on Borland and Agile.]
We broke that up on geographic bases, where we started with one site, had one "ScrumMaster" and we built what we call the reference process. As we've grown, and our projects are getting more complex, the fact that we evolve from site-to-site based on the same process and the same terminology has allowed us to now choose more complex agile techniques like Scrum of Scrums or work across organizations, and have a common vocabulary, and that kind of common way of working.
Gardner: It also sounds like you are taking the best of what a centralized approach offers and the best of what a decentralized approach offers, in terms of incentive; take charge, and local ownership, and then making them co-exist.
Morowski: That's correct.
Gardner: All right, let's get specifically into Borland's situation. What is it about the way that Borland has been developing software, which is of course a core competency for a large independent software vendor (ISV) like yourselves, and it has been for 15-plus years … How difficult was it for you to come into this established organization and shake things up?
Morowski: Initially, it wasn't an issue because, like most organizations, when we went through and looked at it, there were a couple of grassroots efforts underway. From an Agile perspective, one of the things we did was to begin to leverage that activity and the successes that it had to use as a benchmark with other teams. As we grew and moved into other organizations that were not necessarily grassroots efforts, there were some challenges.
Gardner: So, it might be quite possible that lot of organizations that do development have people who are Agile-minded and perhaps even followers of Agile doing this already. Perhaps they should look for those and start there.
Morowski: I would recommend that you start with your grassroots efforts, establish your benchmark process, and then begin to move out from there.
One thing we clearly did was, once that we saw the benefits of doing this, we had a lot of executive sponsorship around this. I made it one of the goals for the year to expand our use of Agile within the organization, so that teams knew it was safe to go ahead and begin to look at it. In addition, because we had a reference implementation of it, it also gave teams a starting point to begin their experimentation. We also paid for our teams to undergo training and those types of things. We created an environment that encouraged transformation.
Gardner: Let's learn a little bit more about you, Pete. Tell us a little bit about your background and how you came into development and then into Agile?
Morowski: I've been in this business a little over 25 years now. I started in the defense and aerospace industries and then moved into commercial ISVs later in my career. I've been an executive at Novell. I've also been a CTO at IBM Tivoli, and prior to Borland, was the vice president of software at Dell.
Gardner: You've taken on this Agile project at Borland, and you've written a paper on the “Borland Agile Journey.” I've had the pleasure of reading it. I think it's a really nice read and I commend for you it.
Morowski: Oh, thank you.
Gardner: Tell us about this particular product set [Borland Software Delivery Management information] that Borland is coming out with. It's a product set about helping people develop software. Is there a commonality between some of the lessons you learned and then what you may have actually visited in terms of requirements for your products? [See demo and see launch video.]
Morowski: Oh, absolutely. One of the interesting things about the products that we are delivering is that one of them is a product for managing Agile development, especially in distributed teams and managing the requirements. So, we had the advantage of actually using the tools as we were developing them.
Now, we were also very cautious because you can get myopic about that type of thing, where we also using Agile principles, and we involved our customers in the process, as well. So we were getting kind of the best of both worlds.
Gardner: What makes software development different? In reading your paper, I was thinking about how these principles about self-empowerment and working quickly and then setting these boundaries -- "Okay, we're going to just work and do this for three weeks and then will revisit any changes," -- that might be something it would apply to almost any creative activity where a team is involved.
Is Agile something you think applies to any creative activity, a complex team-based activity, or is there something about it that really is specific and germane to software development?
Morowski: If you look at Agile principles, conceptually, they do apply to a lot of things. Anything in which you are going into a period of discovery, one of the key things is knowing what your goal or mission is. In the case of software, that's a requirement, and what you want the product to be.
But in any kind of empirically based endeavor, this would be something that you could apply. Now, when you get down to the actual Scrum process itself, it's the terminology, the measures, the metrics, and all those types of things are really tailored for software development.
Gardner: When I read your paper, I also came away with some interesting observations. You say, there is a difference between how development is supposed to work and how it actually works. It's sounds like many companies are living in denial or a certain level of dysfunction that they are not necessarily facing.
Morowski: It's one of the issues with laying a manufacturing process over something that's inherently an empirical process. In the end, all software R&D organizations or IT shops responsible for applications are responsible to the business for delivering results. And, in doing so, we all try to measure those things.
What I have observed over my career was the fact that there really existed two worlds. There is what I call the "management plane," and this is a plane of milestones, of phase transitions and a very orderly process and progress through the software development lifecycle.
Underneath it though, in reality, is a world of chaos. It's a world of rework, a world of discovery, in which the engineers, testers and frontline managers live. We traditionally use Gantt as a measure that is task-based. It requires a translation from the implementation world to the management world to show indications of progress. Any time you do a translation, there can be a loss of information, and that's why today software is such an experienced-based endeavor.
Gardner: And it's often been perceived as sort of a dark art. People don't appreciate or understand how it's done, and that those who do it should say, "Hey, leave me alone, get away from me. I'll come back with the results in three months."
Morowski: Exactly.
Gardner: But that doesn't necessarily or hasn't historically been the best approach.
Morowski: Absolutely not.
Gardner: Also, at times, you see them downplay process and say that doing good hiring probably is the biggest issue here. What's the relationship between hiring and what people, not always affectionately, refer to as human resources? What's the relationship between HR and Agile?
Morowski: Well, first of all, just getting back to a little bit on hiring thing. Hiring is important, regardless of what methodology you use, and I tend to stress that. I do contend there are different kinds of personalities and skill sets you are going to be looking for when you are building Agile teams, and it's important to highlight those.
It's very important that whoever comes onboard in Agile team is collaborative in nature. In traditional software environments, there are two roles that traditionally you may struggle with, and you have to look at them closely. One is the manager. If a manager is a micromanager-type, that's not going to work in an Agile environment.
And, the other thing, interestingly enough, is the chief architect role. What's interesting about that is that you would think you would fit in Agile very easily, but in a lot of traditional software organizations, all decisions of a technical nature on a project go through the chief architect. In an Agile world, it's much more collaborative and everybody contributes. So for some personalities, this would be a difficult change for them.
Gardner: So there is that grassroots element, and you have to be open to it.
Morowski: Right.
Gardner: What is it about the structures here? Again, for folks who might not be that familiar with Agile, tell us a little bit about some of the hierarchy.
Morowski: There are really two key roles. There is the ScrumMaster and the ScrumMaster runs what they call the daily stand-up. This is basically a meeting, where everybody on the team gets together on a daily basis and they answer three questions. "What did I get accomplished yesterday?" "What am I going to do today?" And "What's blocking me?"
Everybody goes around the room. It's a 15- minute meeting. You solve any particular problems, but you log things. The role of ScrumMaster is to run that meeting and to remove blocks to the team, and it's a very key role.
The second major role within Scrum is really the product owner, and this is the individual that's responsible for prioritizing the requirements or what we call the product backlog -- what is what is going to be done during the sprint, which features are going to be completed. Those are the two primary roles, and then from there everybody is pretty much a team member.
Gardner: When you decided to bring this into play at Borland, a very large, distributed organization, you didn't try to bite off too much. You didn't say, "We are going to transform the entire company and organization." You did this on more of an iterative basis. It seems that most people, when they do Agile, will probably follow similar path. They'll take a project basis and then say, "Now we need to expand this and make it holistic."
Many organizations, however, across all kinds of different management activities, can stumble at that transition from the project, or the tactical, into the holistic, or general, across an organization. What did you learn in making this transition from small to large scale at Borland?
Morowski: A couple things. One is that, as we rolled it out, let's say starting by site-by-site, we grew from teams-to-teams. The ScrumMasters worked very collaboratively to help each other out, because, in the end, they were responsible for delivering at the end of those sprints. That was a very positive effect.
As we moved out to distributed teams, there were a number of challenges, things like the daily stand-up, or if I have people in Singapore that are supporting a particular sprint, say, from the system testing point, that made things difficult. But, what I found is the team was pretty creative in involving those individuals, whether they recorded sprints, whether they shifted time zones, and they did this all on their own.
That was the absolute positive, one of the things that surprised me. It was an interesting discovery.
As we started to be more broad with the interaction with the non-Agile parts of the organization, this was a little bit more of a challenge, and I learned a couple of things. In doing any kind outsourcing, if you try to match a traditional, contractual base -- the statement of work (SOW)-type outsourcer -- with an agile team, that's going to present problems. The outsourcer is expecting very detailed specifications as a statement of work and that's just not produced during an agile or sprint/Scrum type of development activity.
The other thing is internally, and what I would say at the end of the pipe and at the beginning of the pipe, working with marketing and our new product introduction processes and support and getting sales out. One of the things that we found is that we started to have a capacity to release more often, but the organization, as a whole, had to adjust now to: A) provide market requirements to us in a different manner, and B) we had to adjust our process at the end to be able to accept more rapid releases.
Gardner: So in order to get the most out of Agile, it sounds like, for those organizations where software development is core competencies, important to their successes as a company, or as a government organization, or a public not-for-profit, that the edges of Agile start to blend into other departments. The whole competency around their organization can perhaps borrow some of these principles from development and extend them into the entire lifecycle.
Morowski: Yes, we no longer look at it as strictly an R&D thing anymore, just because of that. And, it's interesting. You know you are making progress from a development team perspective, when you are starting to output more than the organization can accept.
Gardner: Interesting. So, adjustments along the way, and that's again a principle of the approach.
All right. In this age of Agile and your Agile journey, you came away with three basic observations about the benefits. One was around self-directing teams; second around being able to manage change well; and, third, about how to do the relationship with the customer, in this case the customer being the folks who are interested in getting the software. Tell us about these three benefits and what you have learned?
Morowski: Well, we touched on the self-directing teams, and the key to that is one of the most important things as an executive is that you really have to take the lead and let your teams go and develop -- let them truly own their projects. There will be mistakes along the way, but once they do, it's an extremely powerful concept.
One of the great things about agile is that it's a very open and very visible methodology. During daily stand-ups, I can attend any daily stand-up and sit there and listen to what's going on. I can't contribute in those meetings, because that's run by the ScrumMaster. But, one of the times I was attending the daily stand-up, I knew the teams had progressed a great deal.
When they were looking at their remaining work backlog that they had for that particular sprint, and there were a couple of tests that need to be run that there was nobody assigned to. One of the developers had time, looked at that, and picked it up.
Now, normally, that would never happen, because we behave in a silo fashion. "I am an engineer." "I am a tester." It's an "I am a …" type of thing. But, when you really have a self-directing team, the team owns that schedule and it's very interested in making sure that they meet their commitments.
Gardner: I suppose that also fosters willingness of people to move in and out of role, without just saying, "Well, that's not my job …", but taking more group responsibility, and even as an individual.
Morowski: Absolutely correct, and that to me has been one of the more powerful things that I have personally observed.
Gardner: Change management has often been something that drives developers crazy. They hate when people come in and start changing requirements when they are in the middle of doing code or test. On the other hand, things don't stay the same, and change is part of everything in life and business, perhaps more so today than ever. How do you reconcile those two?
Morowski: Well, I think the reality is that there is going to be change during these development cycles, and so the question is what's the best way to handle it? If you look in a traditional waterfall methodology, you march along phase transitions. Even if you have iteration in place, if you discover a design or coding defect late in the game, you have to go backwards to a different phase and start going into the design or fixing the code. Then, you repeat the process again, and you continue to move along your space transition line.
The thing that's interesting is that with Agile you have an orderly way of injecting change. In other words, as a sprint completes and you've demonstrated the code -- and you demonstrate it after that three-week iteration -- if something has changed and you need to change the prioritization, you have a way to inject that change along that boundary, and then let the team go forward. That's what I always like to say, "We're always going forward in Agile."
Gardner: And how do the teams adjust to that?
Morowski: It's part of the process. The changes go into the backlog. The product owner looks at them and then prioritizes it based upon the complexity of the work and the timing and so on and so forth, and just how important that is. If it's important enough, it will go into the next iteration. The teams are used to doing that, because you are not, in essence, disrupting at a random point. They have already finished what work they were working on, and now there is a cleaner opportunity to inject that change.
Gardner: So, boundaries allow for those who want change to get it done without having to wait for a particularly long period of time or until the project is done. But, for those involved in the project, they have these sections where it's not going to become chaotic and they are not going to lose track of their overall process, because of this injection of change.
Morowski: No, as a matter of fact, the process encourages it.
Gardner: How about this, what you call customer relationships? It sound to me as thought it's just being transparent.
Morowski: It is. It's a different approach, in the sense that you are actually bringing in the customer as what I would call a partner in the development. They participate in sprint reviews, and sprint reviews at the end of a sprint, where you show the working code, what you have completed and so. Those are done on an every-three-week basis, and we involve our customers.
They also take early drops of the code and provide input into the product backlog on requests that they want, and things like that. It's proven to be very beneficial for us. The one thing is that, when you choose these customers to participate, it's important for them to be Agile, as well, and understand that, and they need to approach this as a partnership not just an opportunity to get their particular features or requirements in.
Gardner: And, that must also help keep expectations in line, right?
Morowski: Absolutely. What I have found is that the customers we have involved want to get used to our cycles and our delivery rhythm. They are less adamant about getting every feature on a list in a particular release, because they know it's a relatively short time before the next one comes around.
Gardner: When we describe these customers, would that, in many organizations, include bringing the marketing people in, and the salespeople. Can they get involved so that this becomes something that will enter the market as an agile activity, rather than having Agile happening on the development side, and then falling back into a waterfall mentality when it comes to the go-to-market activities?
Morowski: Yes, we do, and the transparency that's there actually helps build confidence in the rest of the organization on what we are delivering, because they see it as we progress along. It's not something that mysteriously shows up on their doorstep.
Gardner: It certainly sounds great in theory, and it sounds like you've been able to accomplish quite a bit in practice, but what about metrics of success? How have you been able to say, "it works?" Has Borland cut their cost, their time to development? Do they have better products? All of the above? How do we know we are succeeding?
Morowski: I'd say it's combination of all the above. The first thing is that by putting these teams together, they are much smaller teams than in traditional organizations. So, if you look at it, my teams are almost 30 percent smaller on the Agile side than they are on the traditional side.
Gardner: And what's accounting for that change?
Morowski: I think one, is the ownerships of the teams, and two, the breakdown of very specific roles.
Gardner: Would I be going out on a limb in saying you have eliminated the middle management factor?
Morowski: There is absolutely that as well. The other thing is the fact that we're delivering working code and involving with customers. We are developing fewer superfluous features. When a product goes out the door, it generally has the most important features that were entailed for this release. So, it really helps the prioritization process.
Gardner: Not too many cooks in the kitchen?
Morowski: Exactly.
Gardner: Cool! Tell us a little bit about what surprised you the most about this Agile journey of Borland.
Morowski: I think the power of the daily stand-up. I mean, yes, we got a lot of benefits, and yes, we had a number of successes, we were able to transition code from locations and things like that, but I owe that a lot to the daily stand-up. The thing that surprised me is how powerful it is each morning when everybody gets around the table and actually goes through what they've done, basically saying, "Have I lived up to my commitments? What I am committing to the team today? And then is there anything blocking?"
Generally speaking, a lot of developers tend to be quiet and not the most social. What this did is that it just didn't allow the few people who were social to have input on what was going on. This daily stand-up had people, everybody on the team, contributing, and it really changed the relationships in the team. It was just a very, very powerful thing.
Gardner: It sounds like balance among personality types, but that balance directed toward the real activity that is developing code.
Morowski: Absolutely.
Gardner: Interesting! Well, congratulations. I enjoyed reading your paper, and this certainly sounds like the future of development, I know that's what many people in the business think. We've been talking about Agile development practices and principles and how Borland Software has been undertaking an Agile journey itself, in a development project around development process tools and application lifecycle management products.
Back to those products. Is there anything about the synergy between doing it this way and then presenting products into the field that you think will help other people engage with Agile benefits?
Morowski: Are you talking about the products themselves?
Gardner: Yes.
Morowski: The products themselves, absolutely. We have a product coming out called Team Analytics. The key to this is that, while we talked about self-directed teams, we still have responsibilities to reporting to the business and how we are progressing.
Team Analytics gives us a view into the process, gives us the ability to go ahead and look at how the team is progressing, and those types of things, what features have been included or dropped, without having to go into the team and request that information. So that's a very powerful thing.
Gardner: Right. So, it's one thing to agree that visibility and transparency are good, but it's another to actually accomplish it in terms of complexity in large teams and hierarchy.
Morowski: Absolutely. This allows us to move to what I call a "monitored" from a "reported" kind of methodology of metrics. What I mean by that is, typically, at the senior vice president or vice president level, you really get to look at the state of your products once a month, in the sense that you have operations reviews or some kind of review cycle where all your teams come in and then they report the progress of what's going on.
With Team Analytics, you are able to actually look at that on a daily basis and see if anything’s changed over time. That way, you know where you need to spend your time and that's why we call it monitored, at this point.
Gardner: Super! Well, thank you for sharing your insights. I think there is a lot to be taken away here and learned.
We have been talking with Pete Morowski, the senior vice president of research and development for Borland Software. We were looking at Agile principles in the context of Borland's Agile journey.
Thanks, Pete.
Morowski: Thank you, Dana.
Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions, and you’ve been listening to a sponsored BriefingsDirect podcast.
Thanks for joining us and come back next time.
Listen to the podcast. Sponsor: Borland Software.
Transcript of BriefingsDirect podcast on Agile development principles with Borland Software. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.
Subscribe to:
Posts (Atom)
