ITSM Automation and Intelligence Gains Deliver Self-Service to More Users

Transcript of a discussion on how automation, self service, and analytics are combining to allow IT help desks to do more for less.

Listen to the podcast. Find it on iTunes. Get the mobile app. Download the transcript. Sponsor: Hewlett Packard Enterprise.

Dana Gardner: Hello, and welcome to the next edition of the HPE Voice of the Customer Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing discussion on IT innovation and how it’s making an impact on people’s lives.

Gardner

Our next IT support thought leadership discussion highlights how automation, self-service and big data analytics are combining to allow IT help desks to do more for less.

We will see how automation and ITSM-driven insights endow help desk personnel with more knowledge and provide a single point of support for end users, regardless of their needs while still catering to their preferred method of help.

Here to share the latest on how IT support is advancing in the era of bring your own device (BYOD), cloud, and tight budgets, are three experts, David Blackeby, Program Solution Owner for Cloud Services at Sopra Steria, based in the UK. Welcome, David.

David Blackeby: Good morning.

Gardner: We're also here with Diana Wosik, Group Program Manager at Sopra Steria, based in Poland. Welcome, Diana.

Wosik

Diana Wosik: Good morning.

Gardner: And we're here with Mark Laird, Group Technical Architect at Sopra Steria, based in the UK. Welcome, Mark.

Mark Laird: Good morning.

Gardner: Let’s start at a high level and talk about how support has changed, and why enabling self-service is so important nowadays. Mark, why is self-service such an important issue when it comes to IT help desk?

Laird: For us, there are probably a number of issues. We have a range across our customer base, from millennials, who are used to dealing with websites, mobile, tablets, who really don’t want to call a call center, and don’t want to end up talking to somebody on the phone, through to the legacy users who are much more used to picking up the phone, asking for help, and talking through a problem.

So they're looking for a more human approach, human interaction, versus the millennials who want to fix it themselves, want to do it quickly, and really don’t want to talk to somebody about it. That’s introducing a range of problems and challenges.

Gardner: It sounds as if you need to deliver support in a spectrum of ways, but perhaps with a common core to that support function.

Underlying answer

Laird: The underlying answer to the problem, whatever the problem is, is likely to be the same. If you have a log-on issue, it will be a password reset or an account issue. It’s how you get that information out to the person who has the challenge.

Laird

If it’s a person on the phone, it's easy enough to talk them through it. But if you have somebody who is coming through a self-service portal, you have to provide them with that same information. So yes, at times, you connect a single call, a single database, and send your knowledge environment to a range of callers.

Gardner: David, we're being called on here to deliver support across the spectrum of modalities, methods, or even latency, but at the same time, many of the world governments are asking for austerity and savings in their budgets for IT. How are we able to reconcile this need for more variety and the delivery of help desk services, but cutting costs at the same time? Is there any way to reconcile them?

Blackeby: It’s part of the core challenge in the current world with austerity, where both our public and private customers are looking at how they can do more for less money.

IT has continuing cost pressures to reduce cost and overhead of providing IT.  At the same time, we talk about new methods of self-service, different types of platforms and different types of devices and this multi-channel effect that costs time, effort and money to invest in these technologies.

Blackeby

That’s the underlying driver for how it comes down to the service provider to do that. The only way we can do that is looking at industrializing that service delivery and automating processes, moving activities that may have previously been done by Level 2 and Level 3 resources. We're looking at how we can move those to cheaper or lower-cost resources, such as a service desk, or in an ideal world, remove them entirely from the cost chain and drive the automation. So the activity increases the speed and the agility while reducing the cost of delivering the service.

Gardner: Diana, another variable in the mix here is the increased use of mobile devices, of fluidity of the user in terms of their geography, their location, even the time of day that they might be working, and of course there is a plethora of devices, if you want to bring your own device organization. How is mobility affecting this equation for a more complex approach to help desk?

Wosik: Mobility is very important nowadays, because everybody uses mobile devices, every single day. We need to ensure a single point of contact, so they all can approach their help desk at any time they need, and they need the availability 24×7 for that.

Gardner: So, we've established that we have a need for more variability, addressing more types of help from more types of users. Tell me a bit more, Mark, about automation and self-service and how they support one another? What is it about automating processes that endows the user with more access to help, but then maybe that same feedback loop between the user and the support infrastructure can be brought to bear on future issues?

Laird: Automation is doing the same thing in a repeated, controlled fashion. Whether it’s a password reset or the delivery of a service or a server, what you're doing is scripting. You're putting into a workflow a process that a user can call on. Whether that user is an end user, an end customer, or in fact one of the operations team, it allows them to do that fairly standard process in a repeated quality controlled fashion.

And that can allow lower cost, potentially, as David said, bringing the tasks from maybe a qualified Level 3 expensive support person into an operations center, or in fact, maybe on to the self-service portal, where you're not having to give access to systems to end users, but you are allowing them to run a script.

Double benefit

Gardner: David, perhaps you could help me understand why self-service is a benefit to both the receiver of the help, the end user, as well as the organization. What is it about self-service that refines process and benefits the deliverer of the help, but at the same time, gives more speed or perhaps options to the receiver of the help?


Blackeby: Essentially it supports both sides of the equation. From an end user perspective, it’s that instant gratification, I can go into a centralized portal. I can do my search or raise my request and I can be instantly satisfied with the response. I could be presented with a knowledge article that tells me how to fix my particular issue.

If I'm requesting a new service to be delivered through orchestration in the back end, I can make my request, and the orchestration comes in and drives the automated delivery of that service to me. So it increases the agility for the user and it reduces delays.

From the other side of the equation, looking at it from a service provider’s perspective, the more work the user can do themselves takes cost away from us as a service provider.

Historically, a user would have called the service desk, so as a part of that conversation you need to understand who the user is to provide them the service. Make sure it’s a service that they are potentially allowed to have and sort of help through the process. That means that we need a body to answer the phone, and the amount of time that we spend on a typical call from the user drives the cost from a support center perspective.

That reduces the handling time by our agents and by the people who are delivering them the service.

Even if you have a scenario where a user using the portal today, and still need ultimately a human interaction to deliver that service, we already know who they are, and will have asked relevant questions upfront which means we don’t have to ask the questions later on down the line when we try to deliver a service. That reduces the handling time by our agents and by the people who are delivering them the service.

Gardner: Before we dig into the how you do this, now that we have established why it's an important new aspect of helpdesk, Diana, perhaps you can tell us a little bit about Sopra Steria, the organization, and to what degree they are supporting help desks in your markets?

Wosik: I can give you a good example of how it works in Poland and how the automation helps us out regarding the functionality of help desk.

We apply quite a few solutions, like virtual machine (VM) provisioning that has been automatically provisions the machines aligned to customer needs. There is a monitoring tool that is automated. So not only we monitor whatever is going on, but we're also able to answer the needs very quickly, thanks to our automation services.

And then there's the thing regarding the automatic deployment of our releases. Whenever there's a new release of the system, we don’t need a bunch of people who are going to work on it. We can also deploy it very quickly in production, and that helps us to bring the solution as quickly as possible to our customer.

Higher-level view

Gardner: Could you give us a higher-level view of Sopra Steria, the organization, and to what degree help desk support is part of a larger portfolio of services?

Laird: We're a European IT company. We run IT for a wide range of European customers. We deliver services. We write software. We do business process outsourcing. Essentially, if there's a computer involved in there somewhere, that’s what we do.

We have a presence in 27 countries across Europe, in India, and then smaller offices in Singapore, Hong Kong, and China. We have 36,500 staff, and an annual turnover of about 3.5 billion euros. So, we're a reasonably large company, one of the top 10 European IT companies.

For us, the service desk is the single point of contact. For all of our customers, that is their point of contact with us, whether it’s through the Global Delivery Center in Poland, where we're offering French, German, English, small amounts of Spanish and Italian, or through some of the in-country service desks, such as the ones we have in France and the UK. So that is our single point of contact and it’s of key importance to us.

Blackeby: Just to follow on from that, the key piece of that is that it’s an intelligent service desk as opposed to a help desk. It’s really about having the phones manned by intelligent people who are able to both try and fix or resolve issues straight away, as opposed to just logging a call, creating a ticket, and passing it off to someone else.
 
Gardner: How is it that we're providing those individuals on the front line with better knowledge? Are they getting more tools? Are they getting more data? Is this really just correlating a single point of access to the existing data? Is it all of the above? How do we empower those people to do this difficult help desk job better?

Blackeby: In the same way that we try to have a single point of entry for users, for a portal, it’s really the same piece for our support staff as well.

While there are many systems that underpin our service delivery, the key element we try to strive for is that the operators have a single place to work. It’s very much thorough the integration of various systems and data sources into a centralized repository, so that the person that’s trying to act on a ticket, request, or other activity has everything they need in one place, so they can immediately see what the issue is, see what the request is, and then deliver the service to that end user.

Gardner: It strikes me that whether it’s a help desk’s person or the end user, the more they use this, the more the data can be collected, the more knowledge can be harnessed from the interactions, and therefore brought back through a feedback loop into the next level of support.

Is the cost savings on this ultimately about you're better able to understand the market because of the self-service, because of these portal approaches? Is that a big part of it?

Key items

Blackeby: It feeds into that. If you're looking at industrializing or automating, you're really looking for repeatable activities that are done time and time again. The data helps to support that. It identifies suitable candidates that are high volume, high throughput transactions that are really the key things that you want to focus on in terms of introducing automation into the environment, or automation into task elements in a given process. So, over time, it’s pretty much what we are doing.

As Mark mentioned, we're a managed service provider (MSP), providing the services across many customers. So, a lot of the economies of scale we get are best practices that we apply in one account or particular scenarios or issues that we see in one, we can see correlations in other customer accounts as well. So we can bring those efficiencies and bring that investment we make and automation through our back office processes to benefit multiple customers.

Wosik: What is very well known right now is big data and smart analytics that will help us to gather all the information from our customers, so the more tickets and the more incidents are logged, the more information you can gather as well. This is gathered and analyzed. This is when we can provide more accurate and quicker answers to our customers. It’s something that has really impacted our quality of service.

Gardner: Let’s look also back to the systems, when we think about gathering information, more and more big data gathered from logs and other output data from the systems themselves, from the platforms. How are you at Sopra Steria managing the knowledge gathering from your systems and then applying that into this other knowledge base about the activities on your help desk and from the self-help portal?

What is very well known right now is big data and smart analytics that will help us to gather all the information from our customers, so the more tickets and the more incidents are logged, the more information you can gather as well.

Laird: We're looking at some of the new technologies around smart analytics and big data, but we're starting with some of the simpler approaches, which as David alluded to and as Diana mentioned earlier, are just the simple high-volume transactions, the things that we do on a regular basis that are maybe quality issues or maybe they are just time consuming, but those are the key ones we're after.

Then, over the next three to six months, as we move into some of the newer technologies around smart analytics, for example, we'll be taking some of the incidents and things coming into service desk, into the service management system, and looking at those and doing problem management on them.

Have we suddenly got an influx of incidents around our exchange platform? Is that actually indicating that there is an underlying problem or an underlying system error that we need to fix?

It’s starting to link all the various systems, whether it’s the business service monitoring system to the back end that the operations teams are using, or the service management platforms at the front that our service desk people are using, pulling all those together, tying them in with, for example, the configuration management platform, so that people are seeing the same information, both from a front-end user impacting view, or from a back-end infrastructure and service view.

Gardner: And I should think that would also help in more agility to do root-cause analysis and making it faster to time for resolution.

Automate and fix

Laird: Exactly. That back goes back to when we fix problems, close incidents, and if there's a resolution in there, doing the analysis on them to identify common fixes. If an incident comes in or a particular type of incident comes in and we always do the same thing to it, we can automate that. We can actually either get the service desk or help desk people access to that quick fix or just automate it right at the start, so when that issue occurs, we automate and fix.

In some cases, that’s moving out of the customer’s view completely. We're fixing it almost before there's an impact.

Gardner: We've talked a bit about making these help desk approaches better from the end-user perspective, empowering the personnel in the help desk organization itself, and finding some new technologies and analysis benefits to propel that forward, but I would like to go back to the issue of cost.

How are we wringing out more cost from this process, perhaps things like identifying automation and what’s called shift left, better or earlier in the process. So, where are we targeting to get the most results when it comes to cost reduction in all of this?

Blackeby: It really talks about how people do transactions, what things are continually occurring that have a high amount of touch points to them. Some of that comes out through time.

These days, more and more commonly, we can use software distribution, or automated software push tools, that don’t require human interaction at all.

One of the challenges we have when we take on a new customer is that you don’t have the excellent benefit of hindsight around how the organization works and what their common problems are. So, as we take on a new customer or a new contract, we have the ability to go and talk to their existing service provider or their in-house person. A lot of that comes out over time.

There are some standard things that we can recognize, because we have similar customers in similar marketplaces or industries and things that we would expect to get from the outset, and by looking at things like password reset tools and things like that are common and applicable across all types of clients.

Then, it’s a case of looking at your volumetrics over time, your repeatable activities, incidents and requests, identifying how can we drive the agility and improve the service levels that we're delivering, and at the same time, reduce cost.

Take a simple thing like software deployment to users machines, historically, that might have been a call to the service desk. They might have dispatched a desk-side engineer or used remote control to be able to connect with a user’s device to go and install the software.

These days, more and more commonly, we can use software distribution, or automated software push tools, that don’t require human interaction at all. We can automatically deploy software to the user.

Zero-touch environment

That moves into that zero-touch type of environment. Through a portal request, we can manage the workflow around any approval activities. Then once fully approved, through the orchestration at the back-end, we can interface by software deployment solution to automate the delivery of that software to that endpoint device.

And we support many different types of devices now. We've seen more and more cases where not only are we talking about physical desktops or laptops, but also around how we manage mobile devices and tablet type devices as well, using mobility and mobile device management solutions.

Gardner: Let’s look at some of these solutions in practice. Sopra Steria has been doing this for some time and across a large marketplace. Do you have any examples that demonstrate when you can do this well that you get those benefits of self-help, common core data, more knowledgeable help desk, reduce costs, all at the same time?

It probably took two or three days to code the solution, but we're saving a significant amount of time every day.

Laird: One of the solutions we looked at in Poland, certainly around automation, was a really simple challenge that the operations team had as part of our Polish operation. Every morning, backups from a particular customer was taking them in the region of one hour to produce a backup report, look at the backup that had failed, re-run backups as appropriate, and then if backups had failed maybe consistently for a couple of days, escalating that out to support team.

We automated the whole thing. It’s all automated using HPE Operations Orchestration. The whole process now takes one of the team about five minutes in the morning, and it’s really a case of checking the output from the system.

So, we've saved somewhere in the region of just under an hour everyday for one person. It probably took two or three days to code the solution, but we're saving a significant amount of time every day. We're getting a much better quality report, and we're able to pass that information out to our second-line and third-line teams earlier in the day, it gives them much more time to fix things.

One of the things that we've looked at now is automating the re-run of backups overnight. Rather than letting them go to maybe two or three days, they're fixed overnight, and we run them within the backup window. It's improving quality to the customer and a having significant impact on savings to the operations team.

Gardner: You mentioned the use of the HPE tools. Are there any other HPE platforms or approaches that are helping you bring in this common data. We talked about the analysis earlier that also helps in this equation of doing more with less.

HPE partner

Laird: We're an HPE partner. We have been for over 10 years now, and we have quite a range of HPE tools across the portfolio, whether that’s from things like the Application Lifecycle Manager, through to HPE Service Manager.

We also have solutions like OMi doing things like event correlation, where we have events coming in from the monitoring solutions, whether that’s from HPE SiteScope or Operations Manager or from third party tools, like SCCM and some of the Nagios tools.

OMi is correlating those events and passing through to the service desk and the operations center the ones that actually need to be looked at. We're filtering out more than 50 percent, 60 percent of the alerts. It reduces our cost. We're filtering those alerts out at a much earlier point in the chain, and with that, we're only raising incidents for ones that actually need to be escalated up to the teams.

We're using tools and technology, to keep costs down and reduce the costs as far as we can.

One of the challenges that are coming more to the forefront these days is probably the adoption of cloud services. It’s a disruptive influence on traditional IT and how IT is delivered.

Gardner: So as we think about being able to future-proof the support services, and by that I mean being able to adapt to a millennial audience, more distribution points, more types of help desk and automation, and that single portal, we also need to be thinking about being backwards compatible. Some organizations do want more of that human touch, the interactions, and perhaps some of the government organizations are interested in that as well.

What is it about the future direction of your services at Sopra Steria, some of the tools and technologies that you are employing from HPE, that allows you to feel confident about being both future proof and backwards compatible for your support?

Blackeby: One of the challenges that are coming more to the forefront these days is probably the adoption of cloud services. It’s a disruptive influence on traditional IT and how IT is delivered.

It’s a challenge for us the service providers to adapt to these. You're talking about environments that can be built in minutes, bringing a whole new way of working, very fluid environments with auto-scaling where the number of resources that we are supporting and managing is growing and shrinking dynamically over time. So that’s really had a big sort of impact on how we deliver service.

We've recognized this and are looking at how we transform the service delivery. We're becoming more reliant on the data that supports the service. So it’s very much around how we manage what’s out there, with a heavy reliance on things like configuration management systems, and discovery of IT resources.

As Mark said, there are things like event correlation, looking at patterns, trends and events so that we can increase the agility and really manage much higher volumes of applications, of servers and of users with a smaller number of people or with the same number of people.

Gardner: It is very exciting a lot is going on.

Tools and technologies

Blackeby: As a ratio you might have a scenario of a support person looking after an average 40 servers to now having to deal with realms of managing, so there are a 100-plus servers, but it’s only through the deployment of the tools and technologies that we can do that.

But at the same time, we still have a large legacy estate and legacy clients and we still need to support. So it’s really looking at how come we engineer our processes so that irrespective of what we are talking about legacy physical server workloads or perhaps on premise virtualized workloads as well as things that might be spun up inside Amazon Web Services or in Microsoft Azure public cloud environments that we provide that consistent level of service and service delivery irrespective of where the service is located or in which format it is delivered back to the customer or users.

Gardner: When I speak to developer organizations and IT production organizations operations, they're seeing a compression and a large degree of collaboration between development and operations. Thus, the DevOps trend.

But at the same time, we still have a large legacy estate and legacy clients and we still need to support.

But when I listen to you, I'm hearing also a compression between operations and help desk in such a way that it benefits the entire IT process in a more automated and the more software-defined and the more data that’s made available, the tighter that compression seems to get. Am I perhaps describing seeing this idea of help desk, support and operations becoming more collaborative, more tightly aligned?

Laird: The whole concept of the operations team being hidden away in a back room and the service desk being the public face is changing. They're becoming much more tightly aligned. Things that the operations team is doing have an almost immediate impact on what the service desk is looking at, and the service desk needs to have access to really all the information the operations team has got.

When the user is on the phone and has a problem with a service, it’s good if the service desk can actually say, "Yes, we know there's a problem and we know what the problem is. We have an estimated fix time of 15 minutes." That gives the user the warm feeling that you're in control and you know what you're doing.

Gardner: I am afraid we will have to leave it there. We've been discussing how automation, self-service and analytics are combining to allow IT help desks to do more for less. And we’ve seen how automation and ITSM-driven insights endow help desk personnel with more knowledge and provide a single point of support for end users regardless of their needs, whether it’s self-service or more of the traditional way of reaching support.

So, join me please in thanking our guests, David Blackeby, Program Solutions Owner for Cloud Services at Sopra Steria. Thanks so much, David.

Blackeby: Thank you.

Gardner: And we’ve also been joined by Diana Wosik, Group Program Manager at Sopra Steria in Poland. Thank you so much, Diana.

Wosik: Thanks to you.

Gardner: And also thanks to Mark Laird, Group Technical Architect at Sopra Steria in the UK. Thank you, Mark.

Laird: Thank you, Dana.

Gardner: And a big thank you to our audience as well for joining us for this IT-support thought leadership discussion.

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of Hewlett Packard Enterprise-sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Get the mobile app. Download the transcript. Sponsor: Hewlett Packard Enterprise.

Transcript of a discussion on how automation, self service, and analytics are combining to allow IT helpdesks to do more for less. Copyright Interarbor Solutions, LLC, 2005-2016. All rights reserved.

You may also be interested in:

• How Etsy uses big data for ecommerce to put buyers and sellers in the best light
• IoT plus big data analytics translate into better services management at Auckland Transport
• How HPE’s internal DevOps paved the way for speed in global software delivery
• Extreme Apps approach to analysis makes on-site retail experience king again
• How New York Genome Center Manages the Massive Data Generated from DNA Sequencing
• The UNIX evolution: A history of innovation reaches an unprecedented 20-year milestone
• Redmonk analysts on best navigating the tricky path to DevOps adoption
• DevOps by design--A practical guide to effectively ushering DevOps into any organization
• Need for Fast Analytics in Healthcare Spurs Sogeti Converged Solutions Partnership Model
• HPE's composable infrastructure sets stage for hybrid market brokering role
• Nottingham Trent University Elevates Big Data's role to Improving Student Retention in Higher Education
• Forrester analyst Kurt Bittner on the inevitability of DevOps

Master IT Support Providers Chris and Greg Tinker's Take on How Integrated Technical Support is the Future

Transcript of a podcast discussion on new methods for rapid-response IT support on mission critical applications and systems.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. View the blog.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you're listening to BriefingsDirect.

Today, we present a podcast discussion on why IT customer support is so important and why industry changes are forcing an integration and empowerment effect for how helpdesks respond and perform.

We're here with two lauded IT Master Technologists from HP to learn more about what makes good customer support tick. Part of the solution comes from providing a more centralized, efficient, and powerful means of getting all the systems involved working, and all the knowledge necessary to come together to quickly get people back in action and keep them there. But, it also involves getting disparate parties and vendors across an IT ecosystem to work together in new ways. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

These two technologists, who happen to be identical twins, were chosen via a sweepstakes hosted by HP to identify favorite customer support personnel. We will learn why they gained such recognition and uncover their recommendations for how IT support should be done better now and later in a rapidly changing future of increasingly hybrid and cloud modeled computing.

Please join me now in welcoming our guests. We're here with Chris Tinker and Greg Tinker, both HP Master Technologists. Welcome to you, Chris.

Chris Tinker: Hi, Dana.

Gardner: And also to you, Greg.

Greg Tinker: Thank you, Dana.

Gardner: Let me congratulate you on this award. This was I think a worldwide pool, or at least a very large group of people that you were chosen from. So, congratulations on that.

Greg Tinker: Yes, Dana, thank you very much.

Gardner: Did this come as a surprise? How did you feel when you learned about it?

Greg Tinker: It was an honor, I can say that, and we are very grateful for that. Our customer installed base, as well as our peers and the management team, put our names into this situation. It was a great honor.

Gardner: Yes.

Chris Tinker: And it was a surprise.

Gardner: Just so we could fill this out a bit, in addition to the quiz and sweepstakes, there was a philanthropic element as well. Every time folks voted, a $10 donation was made to CARE, a leading humanitarian organization that fights global poverty. Is that right?

Greg Tinker: That's correct. For each vote that was cast, HP donated $10 to the humanitarian organization Care, to max out at a $100,000. They met that goal in just a few days. It was quite astonishing.

Gardner: Great. Now, it's kind of ironic from my perspective, because I'm thinking that some of the most unpopular people can sometimes be the IT support, because people are in a really difficult situation when they encounter them, but you guys won the popularity contest for an unpopular task. How does that feel?

Greg Tinker: It's definitely an honor. It's our livelihood, but it’s definitely rewarding.

Chris Tinker: Very rewarding.

Their darkest hour

Gardner: You deal with people when they are, in some cases, their darkest hour. They're under pressure. There's something that's gone wrong. They're calling you. So, you're not just there in a technical sense, which of course is important, but there must be a human dynamic to this as well. How does that work?

Chris Tinker: We become their confidant. We foster a relationship there between the two parties. For us, it's very exhilarating. It's the ultimate test. You want to build both the technical and business, but also the interpersonal relationship, because you have to weigh in on so many levels, not just technical. That’s a critical component, but not the only component.

Gardner: Anything to add to that, Greg?

Greg Tinker: No, Chris actually summed it up quite nicely. He and I both have a passion for what we do and we really thrive in the heat of the moment.

He and I both have a passion for what we do and we really thrive in the heat of the moment.

Gardner: All right. So what does it take to be a good IT support person nowadays? Let me start with you Chris?

Chris Tinker: It’s simply not enough to be a technical guru -- not in today's industry. You have to have a good understanding of technology, yes, but you also have to understand the tools and realize that technology is simply a tool for business outcomes. If you're listening to the business, understanding what their concerns and their challenges are, then you can apply that understanding to their technical situation to essentially work for a solution.

Gardner: Greg, how about for you? What do you think makes a good IT support person?

Greg Tinker: I second Chris's sentiment on that, and I'll add this. Chris and I study, almost on a daily basis, to stay ahead of the technology curve. Chris and I both do a lot in SCSI I/O control logic, with respect to the kernel structure of HP-UX as well as Linux, which is our playground, if you will.

And, it takes what I would call firm foundation to be able to provide that strong wealth of knowledge to be the customer's confidant. You can't be an expert at one point anymore. You can't be a network expert only. You have to understand the entire gamut of the business, so that you can understand the customer's technical problem.

Gardner: It's not enough to go to them and say, "Well, that's really not part of our technical expertise. You'll have to go somewhere else." People don't want to hear that. They want that one hand to shake, right?

Greg Tinker: That's correct, and today the customer expects the technical master technologist, like my brother and I, not just to know the one thing they're asking about, because that question is going to quickly turn. For example, I am having an Oracle performance issue, the customer thinks it may be disk related, but when you dig into it, you find out that it's actually an ODBC call, a networking issue. So, you have to be quite proficient at a multitude of technologies and have a lot of depth and breadth.

Gardner: How did you both get involved with this? Did one get into it first and the other follow? What's the story behind how you ended up here?

Lengthy road

Greg Tinker: It was quite a lengthy road. Chris and I actually started off going in one direction, and we agreed many years ago in school that one of us would go one direction and the other in another, and see who was enjoying the industry better. Chris joined HP and fell in love with it. He and I have a very strong Linux background. Then, I jumped ship and went with my brother Chris, and we have been with HP ever since, and have loved it dearly.

Chris Tinker: That's a great point. We look at IT support as a ladder and we just climbed that ladder. We started in mission-critical support and found it to be exhilarating. With mission-critical support you're talking about enterprise-class corporations. We're not talking about consumer products. We're talking about an entire corporation's business running on an IT solution and how we're engaged in that process.

Unfortunately, in our line of work, we do see customers, where the technology did not go as planned, predicted, or expected and it's up to us to essentially figure out what the expectations are with technology and ascertain whether or not the technology can deliver that. That's how we moved through support.

We started off as mission-critical support specialists. We became architects, designing solutions for corporations and found out that we were very good at escalations and that's where we are today.

Gardner: You've mentioned exhilarating a couple of times. Maybe you could provide us a memorable example of why that's the case. Is there some event that you were involved with in this capacity that comes to mind that illustrates that sense of exhilaration? Let me start with you, Greg.

When I talk about exhilarating, we're talking about C-level execs and everybody else staring at you with hands on the keyboard to figure out what's causing this panic situation.

Greg Tinker: Well, I can't give customer names out, but I will stick to one particular incident. It was a dire-strait moment, where a customer deployed a particular non-intrusive patch. They didn't think anything of it, and it actually caused a catastrophic kernel panic inside their infrastructure and shut down their entire enterprise. Once that condition was met, they couldn't boot the enterprise back up, and then it became a pointing game as to what was the fault, was it x, y, or z?

That's when my brother and I got engaged in this to find that one smoking gun that was causing the environment to panic. And, all eyes were on us. When I talk about exhilarating, we're talking about C-level execs and everybody else staring at you with hands on the keyboard to figure out what's causing this panic situation.

That’s where Chris and I really thrive. We were able to isolate the condition in probably about an hour-and-a-half and pull out that component, the offender, and get the enterprise back rolling again.

Chris Tinker: Not to speak light of the customer situation, but it was a fun moment -- and I say fun in air quotes -- because you have the C-level execs standing over your shoulder, literally watching what you are doing. They're sweating because they've been down for so much time. I should state here that it wasn't the HP technology or HP solution that was at fault. It was a third-party interoperability issue that had gone down and caused that interruption.

But, we did isolate it and we did figure out what it was. We talked to that vendor, partnered with them, and got the solution in place in very short order.

Gardner: I imagine that, even though typically these vendors don't always have all of their ducks aligned, when it comes to this sort of a mission-critical situation, they're probably thankful that there's someone there trying to corral this. So, I imagine the cooperation is pretty high in these circumstances.

Stakes are high

Chris Tinker: Yeah, the stakes are high at this level. You are talking about, not only the corporation, the customer, but you are also talking about the vendors, whether it be HP or third party, and we are partnering with all these vendors. Everybody has got a stake in the game. Essentially, their reputation is on the line.

So we partner, regardless. As we don’t want to be thrown under the bus, we don’t throw anybody else under the bus. We partner. We come together as one throat to choke or one hand to shake, however you want to look at it. But, essentially, we all have the same thing in common, the customer’s wellbeing.

Greg Tinker: I'll second Chris’ sentiment on that, in the sense that when we're engaged at our level, it's no longer a finger-pointing game. It's a partnership, regardless of who the customer is. If it's HP gear, so be it. If it's somebody else’s gear, and we see where the problem is at, we don't point the finger. We ask the customer to get their vendor on the bridge with us and we work as a team to get the business restored, because that’s priority one.

Chris Tinker: That’s HP technical support. That’s what we thrive at. That’s one of our charters. Our management has dictated that they want team effort, global effort.

Gardner: I suppose you can always deconstruct fault afterward, and the point is to get people up and running ASAP.

Greg Tinker: That’s right.

Chris Tinker: That’s exactly right. Root cause is a nice to have, business online is better.

Catchphrases change. Today it's cloud computing, but cloud computing has been around for a long time. We just didn’t refer to it as cloud computing.

Gardner: Right. How long have you guys been doing this? How long has this been your profession and your passion?

Chris Tinker: Thirteen years now.

Greg Tinker: Twelve for me.

Gardner: Okay, 12 and 13 years. What's changed over that period of time? It seems as if complexity just keeps rolling higher and higher, with more unintended consequences as a result of that. What would you characterize, Chris, as what's evolved or changed most in the past dozen years or so?

Chris Tinker: Catchphrases change. Today it's cloud computing, but cloud computing has been around for a long time. We just didn’t refer to it as cloud computing. Shared infrastructure of course is what we called it.

Virtualization today is becoming a big ticket item, where in years past, big iron was the thing that was a catchphrase. Big iron was very large computers. We still have big iron in storage, that’s true. We still have that big footprint, big powerhouse, that consumes a lot of power, but that’s a necessity of the storage platform.

The big thing for today is converged infrastructure. These are terms you wouldn’t have heard years ago, where we are trying to converge multiple type of protocols, physical media under one medium, networking, Fibre Channel, which of course is your storage network, TCP/IP network, going across the same physical piece of media. These are things that are changing, and of course with that comes extreme amount of complexity, especially when it comes into the actual engine that drives this.

Gardner: Additional thoughts, Greg? What's changed in your perception?

Big iron

Greg Tinker: As Chris stated, the key phrase of yesteryear was big iron. I want a big behemoth machine that can outdo mainframe. If you look back to 1999 and 2000, what you were looking for in the open system world was something to compete with Big Blue.

Today it's virtualization and blades. Everybody used to say -- probably about mid-2005 -- "I want a pizza box. I want a new blade." We no longer call those blades. Those are called pizza boxes now. Today, the concept is all about blades. If you can't make the thing 3 inches tall and 1 inch wide, there is something wrong.

Gardner: You've been describing how things have changed technically. How have things changed in terms of the customer requirements and/or the customer culture? That is to say, what are their expectations or perceptions? Let's start with you Chris.

Chris Tinker: Expectation is more for less. They want more computing power. They want more IT for less cost, which I think that’s been true since day one, but today, of course, that "more for less" just means more computing power. The footprint of the servers has changed.

And two, the support model has changed. Keep in mind, we're in support, and we're seeing a trend with these concepts where customers are having all these physical servers and the support contracts on all these servers are being consolidated down to one physical server with virtual instances.

The support model of yesteryear doesn’t always fit the support model that they should have today.

The support model of yesteryear doesn’t always fit the support model that they should have today.

Greg Tinker: What Chris is talking about there is consolidation efforts. Customers used to have 500 servers. Today, -- I want to exaggerate my point here -- we have it on a virtualization of one or two physical machines that are behemoth and it's virtualized 500 guests.

Though that model works right for consolidating the cost effort of the infrastructure, so your capital cost is less, the problem now becomes the support model. Customers tend to reduce the support as well, because it's less infrastructure. But, keep in mind, most customers kind of forget a lot of times that they've put all their eggs into the basket and that basket needs a lot of protection.

So now you have your entire enterprise running on one or two pieces of physical hardware that is a grossly complex with not only the virtual servers, but the virtual Ethernet modules, the Fibre Channel model concepts are all now basically one concept to run every protocol type, whether you are running infiniband, Gigabit Ethernet, Fibre Channel, etc., the complexity requires a great deal of support.

When a customer calls up and says, "We've made a change in our environment and my server has crashed, the physical server went down, or has lost access to its storage or network," you're not just affecting that one physical server, but you're affecting hundreds. So, the support model today is quick.

Chris Tinker: To add to Greg’s point, a compartmentalization of yesteryear was, "I have physical servers in racks and I will go to another row with a different rack. It has more servers there." So, your compartmentalization, your isolated zones, were in the physical data center, where today your isolated compartmentalized zones are within the same chassis.

Gardner: It sounds to me that there is a higher risk profile. Is that a fair characterization?

Hardware redundancy

Greg Tinker: That would be a fair characterization. There is a higher risk on the hardware end in the sense that you still have hardware redundancy, of course, but you're fully dependent upon cluster technology and complexity.

Let's talk about the chassis. The chassis concept of our blade infrastructure, and this is true for most vendors, is that you are redundant there. But, if you want to be redundant at the hardware layer, you've got to have yet another chassis. In order to get that redundancy across the chassis components, you have to have a virtualization software on top of it, adding more complexity, which becomes a real need for a powerful support base.

Chris Tinker: A good solution design for business risk assessments are still a critical component to your solution design.

Gardner: I'm going to guess that over the past several years in the tradeoff for cost and risk, people probably favor the cost side a bit. So, that means the people in your position are the backstop. "I'll assume more risk and I'll have some cost benefits, but in order for me to survive, I'm going to need a more capable IT support function." Is that a fair assessment?

The new light today is that customers are focused more on the higher end support models, meaning four-hour call to repair.

Greg Tinker: That’s what the trend is becoming. The trend is, "We're going to reduce our cost in the CAPEX and reduce our cost in the infrastructure. We're going to consolidate and virtualize that concept, and we are going to look at our support strategy in a different light." That’s what most customers think.

Gardner: What is that new light?

Greg Tinker: The new light today is that customers are focused more on the higher end support models, meaning four-hour call to repair, where it used to be 24-hour or 48-hour support models, where we were not in a huge rush. If we had a disk drive failure, we had plenty of time, because we had full redundancy, whatever. So we had plenty of time to fix those components.

Today, with all this consolidation effort, it becomes a real critical need when you have a failing component, whether it be hardware or software, to get that component addressed urgently. You don’t really have the time.

Chris Tinker: That’s a great point. Looking at that standard support model, you had so many physical servers and your business was essentially interlaced with these systems. You could handle an outage, whether software or hardware condition. It wasn't as strategic or as strong as today’s virtualized environments, where you would have much heavier business impact.

To Greg’s point, this inter-support model used to work with some of these virtualized environments. I am not saying all virtualized environments, but some of these virtualized environments. With four-hour call-to-repair, you can imagine in four hours what’s required. The technologists who answer the phone first have to address the business concerns to figure out what the business impact is and understand what the problem is.

Once we ascertain what’s causing that problem and the problem has been defined, we have to figure out what’s going wrong with the technology in order to bring it back online.

Business assessment

All that has to be done within four hours on some of our most critical contracts. Of course, that’s the most advanced contract. There are many stages between that one and all the way down to standard support. There are all levels in between, and that customized support model has to be a business assessment.

Gardner: So, we have these trends around increased complexity, reduced time to repair or meantime to emulate your issues. We also have a higher level of concentration of risk and an impetus to cut cost, and you guys are dropped in the middle of that.

What does this mean for your role? It sounds like you need to be good technically. You need to be almost Professional Services as well as helpdesk and support. You need to have those good interpersonal skills, a background in architecture, a background in a variety of different technologies. Help me understand what it is that you think comes together that allow somebody to do what you do?

Greg Tinker: I think the biggest thing I would say is having strong technical background. Having in-depth knowledge of C is a good idea, knowing the kernel structure. That way when you have a failure in a component, software or hardware, you have a clear understanding in the stack as to where the problem most likely resides. You need to have a good idea of where to focus.

"I'm having a set-sock-opt error in the TCP protocol stack." You know you don’t have to look at the Fibre Channel stack. Granted, I'm making that way too simple on purpose. My point is that you have to have a very clear understanding of where the stuff resides.

The very first thing you do is not start looking at logs. You start listening to the customer’s problem and having that relationship

Chris Tinker: It's having an understanding of the actual layers, and in computer technology it's understanding all about the layers of the technology, whether it be the hardware layer or the upper layer stack. If they describe a problem to you as X, it's being able to understand where would that fall, what layer would that fall in. And, that’s going to expedite your ability to troubleshoot that problem.

But, to Greg’s point, that goes back to listening -- listening to the problem, listening to the customer's situation. The very first thing you do is not start looking at logs. You start listening to the customer’s problem and having that relationship. One of the key components here is ownership, letting the customer know that I am engaged now, I own this, I'll work with you, and we will get this solved. That gives them the confidence and the reassurance that there is somebody that’s going to work with them. That’s what HP Technical Support is all about -- having that ownership.

Gardner: There have also been some shifts over the past dozen years or so in the degree to which remote support is possible and your ability to get inside and get that information. Maybe we could take a moment to learn more about what tools have been brought to bear to help you with this, when you get that phone call. When you're dealing with that customer in their moment of need, their darkest hour, you also have a bit more of an arsenal. You have some arrows in your quiver. Maybe you could explain what you think are the most powerful ones and why they work well.

HP virtual room

Chris Tinker: The HP Virtual Room (HPVR). If you go to rooms.hp.com, it’s a good example. As you just mentioned, yesteryear it was, "Hey, send me the logs. Send me the examples. Send me some data, and I'll parse through it and figure it out." You had to wait for data to come in and then start parsing those logs, parsing that data, and building your hypothesis of what might be the problem.

Now, imagine if I were able to take that in real time. So, Greg, talk about real time.

Greg Tinker: Real time is key in today’s technology world. Nobody wants to wait. Take your phone for example. Can you stand it when you have pressed the email button and your phone takes more than three seconds to load it up? Everybody gets annoyed when it's slow. Well, the same is true in technology services support.

When customers call in, they expect immediate response. By the time it gets to our level, where Chris and I sit and our team resides inside the support model, the customer is in dire straits. We use the Virtual Room technology. It's similar to WebEx.

There are a lot of similarities out there. Different vendors have different tools. We use the HP Virtual Room toolset and we can jump onto any machine in the world, anywhere in the world, at a moment’s notice. We can do crash analysis on a Linux kernel crash in real time on a customer’s machine. The same with HP-UX, Solaris, AIX, name your favorite.

We can look at these stack traces and actually find the most likely component that compromises the infrastructure. We can find it, isolate it, and remedy it.

We can look at these stack traces and actually find the most likely component that compromises the infrastructure. We can find it, isolate it, and remedy it.

Chris Tinker: Not only is it just us troubleshooting, but it's bringing to bear our peers. It's team work, a two-heads-are-better-than-one mentality. Greg even lived that first. At the end of the day, you've got 2, 4, or 20 people on the phone. You can imagine all of those people sharing the same desktop at the same time to try to look at a problem. You get all these different levels of expertise.

You're able to take all these talents and focus them on one scenario. So, now with four-hour call to repair, how is that even possible? It's possible when we have to bring these people and partner with these people. They could be not only HP employees and HP technical support. That goes back to vendors and those relationships. We bring those vendors into the same Virtual Room, showing them where we're seeing the problem and asking what we need to do to solve this.

Gardner: That puts you in the role of being the conductor in an orchestra in a sense. That’s another skill set as well, getting that leadership and the ability to get people to line up and focus on a common problem. Does that come up more nowadays?

Chris Tinker: We have many hats to wear. It goes back to our prior point that being a technical guru is not the only critical component to being able to execute at this level.

Greg Tinker: It's knowing one’s limitations. As powerful as Chris and I are in the technology world, we have limitations like anyone would. That’s why it's a team effort. Using tools like the Virtual Room, we can look at a situation and have a good idea of where the problem may be.

Leadership role

If we don’t have that skill set, in a moment’s notice we can get one of our team members to jump into the room with us, look at the desktop, look at the situation, and assess it with us. So, it's a leadership role that we hold in our organization, in the massive technology world of HP, to go out and grab those experts that you need and bring them to bear to the situation.

Chris Tinker: Dana, to your point, it's not enough just to know the technology that you're responsible for supporting. For example, you’re tasked with having to know third-party vendor technology, but you are also tasked with having to understand the technologies like HP Virtual Room.

For example, Greg mentioned WebEx, there are many technologies out there, tools that we use that HP doesn’t create and doesn’t support, but the industry as a whole utilizes on a daily basis. I'm sure you're using one right now that’s either a freeware or a public license.

Greg Tinker: Take Outlook for example. That’s a tool. Today, everybody is expected to know Outlook. If you find someone that doesn't know it, you then question their ability. Everybody would. I'm using that as an example, but a lot of people take these types of tools we use today for granted.

Gardner: While we are on the subject of tools, what's coming next? If I were to design these types of tools, you would be the guys I would go to, to get my list of requirements. What are you asking for? What would you like to see come next in order for you to be able to do your jobs better?

The hard one to fix is "My application is not running the way I want it to, Fix it."

Chris Tinker: The mind meld or The Borg.

Gardner: Reading minds, that’s a good one. More practical.

Greg Tinker: Now, there are some tools that are being leveraged daily inside HP as well as outside. HP Storage Essentials being one. The biggest thing we see today is storage. The growth rate of storage is enormous. And the biggest problems customers run into are performance and capacity.

Capacity is the easy one, right? I am 100 percent full in my file system. I just need more. That's the easy one to fix.

The hard one to fix is "My application is not running the way I want it to, Fix it." Those are the difficult ones. We have to have a lot of tools to help us understand what the load conditions are, because it's no longer the yesteryear scenario of a Superdome, HP Rack, one big behemoth machine, four terabytes of memory, 400 CPUs, loading up one storage array. That's no longer the case.

We have grid computing structures of 600+ nodes running a multitude of different things -- SAP, Oracle, Informix, Exchange, etc. All of these different load-bearing concepts are coming into one monolithic storage array. It can become quite daunting to understand what's causing that load condition, and we have a lot of tools today that are helping us ascertain the root of those problems faster.

Chris Tinker: We have become the bleeding edge of technology. Essentially, it's software that hasn't been released. It's tools which are not actually production ready, and we use these tools as well, and some tools we can’t even speak about.

Business realities

But, these are tools that will be in the enterprise eventually. They will be out in the world eventually. You asked earlier what we see coming down the road? Imagination is essentially one of the only things in technology. In today's world, there are other factors of course. Business realities temper the development of technology, but it's going to be very exciting to see what technology is being developed and what's coming next.

Gardner: While we're looking at what's coming next, you mentioned that level of interest in applications not performing, a very general sort of problem at the surface. It seems to me that the definition of application is shifting. As we look at more hybrid computing models, we look at people who will be compositing from a variety of services, all perhaps coming from a variety of sources. The business process needs to be supported, but the constituent parts now are even more scattered, harder to identify.

It seems as if folks who are in your role are going to have an even more important play here when it comes to these distributed and cloud and hybrid types of applications. Any thought about what you would be needing and what to expect if that's the future?

Chris Tinker: Well, with performance, your key challenge is understanding what tools we use, what metrics we look at. With databases, there are databases tools like AWR with Oracle. When should I be looking at AWR, as opposed to the operating system performance metrics, as opposed to the storage array or network performance?

It's having this very large breadth of technology expertise. It's being able to understand first what tool I use to look at performance.

It goes back to what Greg said earlier. It's having this very large breadth of technology expertise. It's being able to understand first what tool I use to look at performance. Then, of course, you have to go back to the business. You have to ask the business owners, the P&L owners, "What is your expectation? What is actually your business challenge?"

Maybe it's a batch job. Maybe it's a report they want to run at month end. Maybe they want to run a month-end processing for their business accounting, calculate payroll.The business has to be able to define what it is they are going after. Their challenge is being able to align the technology to deliver on that challenge.

Gardner: I wonder if you might have just some last advice for those listening to the podcast as to how they on the consumption side might help folks like you on the services and support delivery side do your job better? What advice do you have for them in order to have a better outcome? Any thoughts on that, Chris?

Chris Tinker: Yeah, it's being able to articulate the actual problem at hand, and the challenge that you have with your technology, because keep in mind that technology, IT, is nothing more than a tool that allows us to have business outcomes. So it's nothing more than a tool that the business utilizes for their requirements.

Then, to have metrics around their environment. They have to have a baseline. They have to have an understanding of what the technology has been doing.

Trending is key

Greg Tinker: Trending is key in a lot of these new virtualized consolidated environments. You need to have a baseline, as Chris stated. We need to have the performance characteristics. Your logging and ESX is about as common as sliced bread in a grocery store. ESX environments are very common and thought of very highly. I enjoy them. They are very nice.

Customers tend to start moving towards ESXi, which is fine, but ESXi doesn't log. It does log but you only get like a two hour history. The point is that customers take that logging for granted. You have to have your logging enabled and you must keep at least a six month trend.

So you don't keep all your logs and your service forever, but a six month trend is very helpful when you have a mysterious problem show up. Then, we can compare yesterday to today and see what differences have shown up in the environment.

Gardner: It comes down to data, having the data at your disposal.

Chris Tinker: Not just data, but having a baseline. We get a lot of calls where customers have no idea of what the environment was doing before. They say, "We're having a problem now. Our users are complaining." We ask, "How did it used to run? How long did this job used to take? Did it use to take 2 hours, and now it takes 20 hours?" A lot of times, they simply do not know.

I wish customers would yield to knowing that logging is critical. You don't have to keep it forever, but keep it for a strategic period of time. Six months is a good number.

I wish customers would yield to knowing that logging is critical. You don't have to keep it forever, but keep it for a strategic period of time. Six months is a good number.

Gardner: So as we look at the benefits from a cost and performance angle of concentrating and converging, you might increase your risk profile and become more dependent on folks like Chris and Greg, but having that data and having an understanding of your baseline can help reduce that risk significantly. That's good advice.

Terrific. I want to thank you two for your input and, again, congratulations on being designated favorites at something that's probably, as I say, not a popular role. So to be popular in an unpopular position really speaks well of you.

We've been listening to a podcast discussion on how IT customer support is growing in importance and why the industry changes are flipped, forcing more work towards reducing that risk, but with an emphasis on the people at the front line on your support services.

So thanks to Chris Tinker. I really enjoyed your thoughts.

Chris Tinker: Thank you, Dana.

Greg Tinker: Dana, thank you again for having us. I would like to add one more comment. For those of your listeners that are willing to come out to the HP DISCOVER Event in Las Vegas, Chris and I have multiple publications and we are giving multiple advanced session discussions on internal I/O control logics at HP DISCOVER Event in Las Vegas, June 6-10. So, if any of your listeners wish to come out and meet us firsthand, we would love to see them.

HP also has a site where you can connect with HP Technology Services experts. We encourage your readers to engage with HP directly.

Gardner: Thanks to you Greg. We have been, as I say, discussing the support life and the trends, and both of you gentlemen are HP Master Technologists. So thanks again.

Greg Tinker: Thank you so much.

Chris Tinker: Thank you.

Gardner: This is Dana Gardner, Principal Analyst at Interarbor Solutions. I've been your host and moderator and you've been listening to a BriefingsDirect podcast. Thanks for listening and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. View the blog.

Transcript of a podcast discussion on new methods for rapid-response IT support on mission critical applications and systems. Copyright Interarbor Solutions, LLC, 2005-2011. All rights reserved.

You may also be interested in:

• HP Premier Services Closes Gap Between Single Point of Accountability and Software Sprawl
• HP's Kevin Bury on How Cloud and SaaS Will Help Pave the Way to Increased Efficiency in IT Budgets for 2011 and beyond
• HP's Instant-On Enterprise Takes Aim at Shifting Needs of Business and Government