How an Agile Focus for Enterprise Architects Builds Competitive Advantage in the Digital Transformation Age

Transcript of a panel discussion on how Enterprise Architects should embrace agile approaches to build more competitive advantage for their companies.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Our next business trends discussion explores the reinforcing nature of Enterprise Architecture (EA) and agile methods.

Gardner

We’ll now learn how Enterprise Architects can embrace agile approaches to build competitive advantages for their companies. To learn more about retraining and rethinking for EA in the Digital Transformation (DT) era, we are now joined by Ryan Schmierer, Director of Operations at Sparx Services North America. Welcome, Ryan.

Ryan Schmierer: Thanks, Dana.

Gardner: We are also joined by Chris Armstrong, President at Sparx Services North America. Welcome, Chris.

Chris Armstrong: How are you, Dana?

Gardner: I’m great, thanks. Ryan, what's happening in business now that’s forcing a new emphasis for Enterprise Architects? Why should Enterprise Architects do things any differently than they have in the past?

Schmierer: The biggest thing happening in the industry right now is around DT. We been hearing about DT for the last couple of years and most companies have embarked on some sort of a DT initiative, modernizing their business processes.

Schmierer

But now companies are looking beyond the initial transformation and asking, “What’s next?” We are seeing them focus on real-time, data-driven decision-making, with the ultimate goal of enterprise business agility -- the capability for the enterprise to be aware of its environments, respond to changes, and adapt quickly.

For Enterprise Architects, that means learning how to be agile both in the work they do as individuals and how they approach architecture for their organizations. It’s not about making architectures that will last forever, but architectures that are nimble, agile, and adapt to change.

Gardner: Ryan, we have heard the word, agile, used in a structured way when it comes to software development -- Agile methodologies, for example. Are we talking about the same thing? How are they related?

Agile, adaptive enterprise advances 

Schmierer: It’s the same concept. The idea is that you want to deliver results quickly, learn from what works, adapt, change, and evolve. It’s the same approach used in software development over the last few years. Look at how you develop software that delivers value quickly. We are now applying those same concepts in other contexts.

First is at the enterprise level. We look at how the business evolves quickly, learn from mistakes, and adapt the changes back into the environment.

Second, in the architecture domain, instead of waiting months or quarters to develop an architecture, vision, and roadmap, how do we start small, iterate, deliver quickly, accelerate time-to-value, and refine it as we go?

Gardner: Many businesses want DT, but far fewer of them seem to know how to get there. How does the role of the Enterprise Architect fit into helping companies attain DT?

The core job responsibility for Enterprise Architects is to be an extension of the company leadership and its executives. They need to look at where a company is trying to go ... and develop a roadmap on how to get there.

Schmierer: The core job responsibility for Enterprise Architects is to be an extension of company leadership and its executives. They need to look at where a company is trying to go, all the different pieces that need to be addressed to get there, establish a future-state vision, and then develop a roadmap on how to get there.

This is what company leadership is trying to do. The EA is there to help them figure out how to do that. As the executives look outward and forward, the Enterprise Architect figures out how to deliver on the vision.

Gardner: Chris, tools and frameworks are only part of the solution. It’s also about the people and the process. There's the need for training and best practices. How should people attain this emphasis for EA in that holistic definition?

Change is good 

Armstrong: We want to take a step back and look at how Ryan was describing the elevation of value propositions and best practices that seem to be working for agile solution delivery. How might that work for delivering continual, regular value? One of the major attributes, in our experience, of the goodness of any architecture, is based on how well it responds to change.

In some ways, agile and EA are synonyms. If you’re doing good Enterprise Architecture, you must be agile because responding to change is one of those quality attributes. That’s a part of the traditional approach of architecture – to be concerned with the interoperability and integration.

As it relates to the techniques, tools, and frameworks we want to exploit -- the experiences that we have had in the past – we try to push those forward into more of an operating model for Enterprise Architects and how they engage with the rest of the organization.

Learn About Agile Architecture
At The Open Group July Denver Event

So not starting from scratch, but trying to embrace the concept of reuse, particularly reuse of knowledge and information. It’s a good best practice, obviously. That's why in 2019 you certainly don't want to be inventing your own architecture method or your own architecture framework, even though there may be various reasons to adapt them to your environment.

Starting with things like the TOGAF® Framework, particularly its Architecture Development Method (ADM) and reference models -- those are there for individuals or vertical industries to accelerate the adding of value.

The challenge I've seen for a lot of architecture teams is they get sucked into the methodology and the framework, the semantics and concepts, and spend a lot of time trying to figure out how to do things with the tools. What we want to think about is how to enable the architecture profession in the same way we enable other people do their jobs -- with instant-on service offerings, using modern common platforms, and the industry frameworks that are already out there.

We are seeing people more focused on not just what the framework is but helping to apply it to close that feedback loop. The TOGAF standard, a standard of The Open Group, makes perfect sense, but people often struggle with, “Well, how do I make this real in my organization?”

Partnering with organizations that have had that kind of experience helps close that gap and accelerates the use in a valuable fashion. It’s pretty important.

Gardner: It’s ironic that I've heard of recent instances where Enterprise Architects are being laid off. But it sounds increasingly like the role is a keystone to DT. What's the mismatch there, Chris? Why do we see in some cases the EA position being undervalued, even though it seems critical?

EA here to stay 

Armstrong: You have identified something that has happened multiple times. Pendulum swings happen in our industry, particularly when there is a lot of change going on. People are getting a little conservative. We’ve seen this before in the context of fiscal downturns in economic climates.

But to me, it really points to the irony of what we perceive in the architecture profession based on successes that we have had. Enterprise Architecture is an essential part of running your business. But if executives don't believe that and have not experienced that then it’s not surprising when there's an opportunity to make changes in investment priorities that Enterprise Architecture might not be at the top of the list.

We need to be mindful of where we are in time with the architecture profession. A lot of organizations struggle with the glass ceiling of Enterprise Architecture. It’s something we have encountered pretty regularly, where executives are, “I really don’t get what this EA thing is, and what's in it for me? Why should I give you my support and resources?”

Learn About Agile Architecture
At The Open Group July Denver Event

But what’s interesting about that, of course, is if you take a step back you don’t see executives saying the same thing about human resources or accounting. Not to suggest that they aren’t thinking about ways to optimize those as a core competency or as strategic. We still do have an issue with acceptance of enterprise architecture based on the educational and developmental experiences a lot of executives have had.

We’re very hopeful that that trend is going to be moving in a different direction, particularly as relates to new master’s programs and doctorate programs, for example, in the Enterprise Architecture field. Those elevate and legitimize Enterprise Architecture as a profession. When people are going through an MBA program, they will have heard of enterprise architecture as an essential part of delivering upon strategy.

Gardner: Ryan, looking at what prevents companies from attaining DT, what are the major challenges? What’s holding up enterprises from getting used to real-time data, gaining agility, and using intelligence about how they do things?

Schmierer: There are a couple of things going on. One of them ties back to what Chris was just talking about -- the role of Enterprise Architects, and the role of architects in general. DT requires a shift in the relationship between business and IT. With DT, business functions and IT functions become entirely and holistically integrated and inseparable.

When there are no separate IT processes and no businesses process -- there are just processes because the two are intertwined. As we use more real-time data and as we leverage Enterprise Architecture, how do we move beyond the traditional relationship between business and IT? How do we look at such functions as data management and data architecture? How do we bring them into an integrated conversation with the folks who were part of the business and IT teams of the past?

A good example of how companies can do this comes in a recent release from The Open Group, the Digital Practitioner Body of Knowledge™ (DPBoK™). It says that there's a core skill set that is general and describes what it means to be such a practitioner in the digital era, regardless of your job role or focus. It says we need to classify job roles more holistically and that everyone needs to have both a business mindset and a set of technical skills. We need to bring those together, and that's really important.

As we look at what's holding up DT we need to take functions that were once considered centralized assets like EA and data management and bring them into the forefront. ... Enterprise Architects need to be living in the present.

As we look at what's holding up DT -- taking the next step to real-time data, broadening the scope of DT – we need to take functions that were once considered centralized assets, like EA and data management, and bring them into the forefront, and say, “You know what? You’re part of the digital transmission story as well. You’re key to bringing us along to the next stage of this journey, which is looking at how to optimize, bring in the data, and use it more effectively. How do we leverage technology in new ways?”

The second thing we need to improve is the mindset. It’s particularly an issue with Enterprise Architects right now. And it is that Enterprise Architects -- and everyone in digital professions -- need to be living in the present.

You asked why some EAs are getting laid off. Why is that? Think about how they approach their job in terms of the questions that would be asked in a performance review.

Those might be, “What have you done for me over the years?” If your answer focuses on what you did in the past, you are probably going to get laid off. What you did in the past is great, but the company is operating in the present.

What’s your grand idea for the future? Some ideal situation? Well, that’s probably going to get you shoved in a corner some place and probably eventually laid off because companies don't know what the future is going to bring. They may have some idea of where they want to get to, but they can’t articulate a 5- to 10-year vision because the environment changes so quickly.

What have you done for me lately? That’s a favorite thing to ask in performance-review discussions. You got your paycheck because you did your job over the last six months. That’s what companies care about, and yet that’s not what Enterprise Architects should be supporting.

Instead, the EA emphasis should be what can you do for the business over the next few months? Focus on the present and the near-term future.

That’s what gets Enterprise Architects a seat at the table. That’s what gets the entire organization, and all the job functions, contributing to DT. It helps them become aligned to delivering near-term value. If you are entirely focused on delivering near-term value, you’ve achieved business agility.

Gardner: Chris, because nothing stays the same for very long, we are seeing a lot more use of cloud services. We’re seeing composability and automation. It seems like we are shifting from building to assembly.

Doesn’t that fit in well with what EAs do, focusing on the assembly and the structure around automation? That’s an abstraction above putting in IT systems and configuring them.

Reuse to remain competitive 

Armstrong: It’s ironic that the profession that’s often been coming up with the concepts and thought-leadership around reuse struggles a with how to internalize that within their organizations. EAs have been pretty successful at the implementation of reuse on an operating level, with code libraries, open-source, cloud, and SaaS.

There is no reason to invent a new method or framework. There are plenty of them out there. Better to figure out how to exploit those to competitive advantage and focus on understanding the business organization, strategy, culture, and vision -- and deliver value in the context of those.

For example, one of the common best practices in Enterprise Architecture is to create things called reference architectures, basically patterns that represent best practices, many of which can be created from existing content. If you are doing cloud or microservices, elevate that up to different types of business models. There’s a lot of good content out there from standards organizations that give organizations a good place to start.

Learn About Agile Architecture
At The Open Group July Denver Event

But one of the things that we've observed is a lot of architecture communities tend to focus on building -- as you were saying -- those reference architectures, and don't focus as much on making sure the organization knows that content exists, has been used, and has made a difference.

We have a great opportunity to connect the dots among different communities that are often not working together. We can provide that architectural leadership to pull it together and deliver great results and positive behaviors.

Gardner: Chris, tell us about Sparx Services North America. What do you all do, and how you are related to and work in conjunction with The Open Group?

Armstrong: Sparx Services is focused on helping end-user organizations be successful with Enterprise Architecture and related professions such as solution architecture and solution delivery, and systems engineering. We do that by taking advantage of the frameworks and best practices that standards organizations like The Open Group create, helping make those standards real, practical, and pragmatic for end-user organizations. We provide guidance on how to adapt and tailor them and provide support while they use those frameworks for doing real work.

And we provide a feedback loop to The Open Group to help understand what kinds of questions end-user organizations are asking. We look for opportunities for improving existing standards, areas where we might want to invest in new standards, and to accelerate the use of Enterprise Architecture best practices.

Gardner: Ryan, moving onto what's working and what's helping foster better DT, tell us what's working. In a practical sense, how is EA making those shorter-term business benefits happen?

One day at a time 

Schmierer: That’s a great question. We have talked about some of the challenges. It’s important to focus on the right path as well. So, what's working that an enterprise architect can do today in order to foster DT?

Number one, embrace agile approaches and an agile mindset in both architecture development (how you do your job) and the solutions you develop for your organizations. A good way to test whether you are approaching architecture in an agile way is the first iteration in the architecture. Can you go through the entire process of the Architecture Development Method (ADM) on a cocktail napkin in the time it takes you to have a drink with your boss? If so, great. It means you are focused on that first simple iteration and then able to build from there.

Number two, solve problems today with the components you have today. Don’t just look to the future. Look at what you have now and how you can create the most value possible out of those. Tomorrow the environment is going to change, and you can focus on tomorrow's problems and tomorrow’s challenges tomorrow. So today’s problems today.

Third, look beyond your current DT initiative and what’s going on today, and talk to your leaders. Talk to your business clients about where they need to go in the future. That goal is enterprise business agility, which is helping the company become more nimble. DT is the first step, then start looking at steps two and three.

Architects need to understand technology better, such things as new cloud services, IoT, edge computing, ML, and AI. These are going to have disruptive effects on your businesses. You need to understand them to be a trusted advisor to your organization.

Fourth, Architects need to understand technology better, such things as fast-moving, emerging technology like new cloud services, Internet of Things (IoT), edge computing, machine learning (ML), and artificial intelligence (AI) -- these are more than just buzz words and initiatives. They are real technology advancements. They are going to have disruptive effects on your businesses and the solutions to support those businesses. You need to understand the technologies; you need to start playing with them so you can truly be a trusted advisor to your organization about how to apply those technologies in business context.

Gardner: Chris, we hear a lot about AI and ML these days. How do you expect Enterprise Architects to help organizations leverage AI and ML to get to that DT? It seems really essential to me to become more data driven and analytics driven and then to re-purpose to reuse those analytics over and over again to attain an ongoing journey of efficiency and automation.

Better business outcomes 

Armstrong: We are now working with our partners to figure out how to best use AI and ML to help run the business, to do better product development, to gain a 360-degree view of the customer, and so forth.

It’s one of those weird things where we see the shoemaker’s children not having any shoes because they are so busy making shoes for everybody else. There is a real opportunity, when we look at some of the infrastructure that’s required to support the agile enterprise, to exploit those same technologies to help us do our jobs in enterprise architecture.

It is an emerging part of the profession. We and others are beginning to do some research on that, but when I think of how much time we and our clients have spent on the nuts and bolts collection of data and normalization of data, it sure seems like there is a real opportunity to leverage these emerging technologies for the benefit of the architecture practice. Then, again, the architects can be more focused on building relationships with people, understanding the strategy in less time, and figuring out where the data is and what the data means.

Obviously humans still need to be involved, but I think there is a great opportunity to eat your own dog food, as it were, and see if we can exploit those learning tools for the benefit of the architecture community and its consumers.

Gardner: Chris, do we have concrete examples of this at work, where EAs have elevated themselves and exposed their value for business outcomes? What’s possible when you do this right?

Armstrong: A lot of organizations are working things from the bottoms up, and that often starts in IT operations and then moves to solution delivery. That’s where there has been a lot of good progress, in improved methods and techniques such as scaled agile and DevOps.

But a lot of organizations struggle to elevate it higher. The DPBoK™  from The Open Group provides a lot of guidance to help organizations navigate that journey, particularly getting to the fourth level of the learning progression, which is at the enterprise level. That’s where Enterprise Architecture becomes essential. It’s great to develop software fast, but that’s not the whole point of agile solution delivery. It should be about building the right software the right way to meet the right kind of requirements -- and do that as rapidly as possible.

We need an umbrella over different release trains, for example, to make sure the organization as a whole is marching forward. We have been working with a number of Fortune 100 companies that have made good progress at the operational implementation levels. They nonetheless now are finding that particularly trying, to connect to business architecture.

There have been some great advancements from the Business Architecture Guild and that’s been influencing the TOGAF framework, to connect the dots across those agile communities so that the learnings of a particular release train or the strategy of the enterprise is clearly understood and delivered to all of those different communities.

Gardner: Ryan, looking to the future, what should organizations be doing with the Enterprise Architect role and function?

EA evolution across environments 

Schmierer: The next steps don’t just apply to Enterprise Architects but really to all types of architects. So look at the job role and how your job role needs to evolve over the next few years. How do you need to approach it differently than you have in the past?

For example, we are seeing Enterprise Architects increasingly focus on issues like security, risk, reuse, and integration with partner ecosystems. How do you integrate with other companies and work in the broader environments?

We are seeing Business Architects who have been deeply engaged in DT discussions over the last couple of years start looking forward and shifting the role to focus on how we light up real-time decision-making capabilities. Solution Architects are shifting from building and designing components to designing assembly and designing the end systems that are often built out of third-party components instead of things that were built in-house.

Look at the job role and understand that the core need hasn’t changed. Companies need Enterprise Architects and Business Architects and Solution Architects more than ever right now to get them where they need to be. But the people serving those roles need to do that in a new way -- and that’s focused on the future, what the business needs are over the next 6 to 18 months, and that’s different than what they have done in past.

Gardner: Where can organizations and individuals go to learn more about Agile Architecture as well as what The Open Group and Sparx Services are offering?

Schmierer: The Open Group has some great resources available. We have a July event in Denver focused on Agile Architecture, where they will discuss some of the latest thoughts coming out of The Open Group Architecture Forum, Digital Practitioners Work Group, and more. It’s a great opportunity to learn about those things, network with others, and discuss how other companies are approaching these problems. I definitely point them there.

Learn About Agile Architecture
At The Open Group July Denver Event

I mentioned the DPBoK™. This is a recent release from The Open Group, looking at the future of IT and the roles for architects. There’s some great, forward-looking thinking in there. I encourage folks to take a look at that, provide feedback, and get involved in that discussion.

And then Sparx Services North America, we are here to help architects be more effective and add value to their organizations, be it through tools, training, consulting, best practices, and standards. We are here to help, so feel free to reach out at our website. We are happy to talk with you and see how we might be able to help.

Gardner: I’m afraid we’ll have to leave it there. You have been listening to a sponsored BriefingsDirect discussion on reinforcing the relationship between Enterprise Architecture and agile businesses. And we have learned how Enterprise Architects should embrace new approaches and digital practitioner, leading-edge thinking to build competitive advantages for their companies.

So a big thank you to our guests, Ryan Schmierer, Director of Operations at Sparx Services North America. Thank you so much, Ryan.

Schmierer: Thank you, Dana.

Gardner: And thank you, too, to Chris Armstrong, President at Sparx Services North America.

Armstrong: You are more than welcome, Dana. 

Gardner: And a big thank you as well to our audience for joining this BriefingsDirect agile business innovation discussion. I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your host throughout this series of BriefingsDirect discussions sponsored by The Open Group.

Thanks again for listening, please pass this along to your IT community, and do come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Transcript of a panel discussion how Enterprise Architects should embrace agile approaches to build more competitive advantage for their companies. Copyright Interarbor Solutions, LLC and The Open Group, 2005-2019. All rights reserved.

You may also be interested in:

• Where the rubber meets the road: How users see the IT4IT standard building competitive business advantage
• How the data science profession is growing in value and impact across the business world
• The Open Group panel explores ways to help smart cities initiatives overcome public sector obstacles
• The Open Group digital practitioner effort eases the people path to digital business transformation
• How The Open Group Healthcare Forum and Health Enterprise Reference Architecture cures process and IT ills
• Why government agencies could lead the way in demanding inter-public cloud interoperability and standardization
• Panel explores how the IT4IT Reference Architecture acts as a digital business enabler
• The UNIX evolution: A history of innovation reaches an unprecedented 20-year milestone
• The Open Group president, Steve Nunn, on the inaugural TOGAF User Group and new role of EA in business transformation

Confluence of Global Trends Ups Ante for Improved IT Governance to Prevent Costly Business 'Glitches'

Transcript of a sponsored BriefingDirect podcast on the growing danger from faulty software and how to overcome it.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: WebLayers.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on the nature of, and some possible solutions for, a growing parade of enterprise-scale glitches. The headlines these days are full of big, embarrassing corporate and government "gotchas."

These complex snafus cost a ton of money, severely damage a company’s reputation, and most importantly, can hurt or even kill people.

From global auto recalls to bank failures, and the cyber crime that can uproot the private information from millions of users, the scale and damage that technology-accelerated glitches can inflict on businesses and individuals has probably never been higher. So what is at the root?

Is it a technology run amok problem, or a complexity spinning out of control issue -- and why is it seemingly worse now?

A new book is coming out this summer that explores the relationship between glitches and technology, specifically the role of software use and development in the era of cloud computing.

It turns out the role and impact of governance over people, process, and technology comes up again and again in the new book.

We have with us here today the author of the book as well as a software expert from IBM to delve into the causes and effects of glitches and how governance relates to the problem and fixes.

Please join me in welcoming our guests, Jeff Papows, President and CEO of WebLayers, and the author of Glitch: The Hidden Impact of Faulty Software. Welcome to the show, Jeff.

Jeff Papows: Thanks, Dana. Thanks for having us on.

Gardner: We're also here with Kerrie Holley, IBM fellow and Chief Technology Officer for IBM’s SOA Center of Excellence. Welcome to the show, Kerrie.

Kerrie Holley: Thank you, very much.

Gardner: Jeff, let me start with you. Now, the general trends around these complex issues are affecting business and probably affecting just about everyone’s lives. How do these seem to be something that’s different? Is there an inflection point? Is there something different now that 20 years ago in terms of the intersection of business with technology?

Papows: There is. I’ve done a lot of research in the past 10 months and what we're actually seeing is the confluence of three primary factors that are creating an information technology perfect storm of sorts. Some of these are obvious, but it’s the convergence of the three that’s creating problems on the scale that you are describing here.

The first is a loss of intellectual capital. For the first time in our careers -- the three of us have all been at this for a long time now -- we saw, between 2000 and 2007, the first drop in computer science graduates. That's the other side of the dot-com implosion.

Mainframe adoption patterns

While it’s not always popular or glamorous to talk about, 70 percent of the world’s critical infrastructure still runs on IBM mainframes. Yet, the focus of most of our new computer science graduates and early life professionals is on Java, XML, and the open and more modern development languages.

For the first time in our lifetimes and careers, the preponderance of that COBOL-based analytical community is retiring and/or -- God forbid -- aging and dying. That’s created a significant problem, concurrent with a time where the merger and consolidation activity -- the other side of the recession of 2008 -- have created this massive complexity in these giant mash-ups and critical back-office systems. For example, the mergers between Bank of America and Countrywide, and on and on.

The third factor is just the sheer ubiquity of the technological complexity curve. It’s the magnitude of technology that’s now part of our social fabric, whether it’s literally one million transistors that now exist for every human being on the planet or the six billion network devices that exist in the world today, all of which are accessing the same critical, in many cases, back-office structures.

It's reached the point, Dana, from a consumer standpoint, where 60 percent of the value of our automobiles now consists of networked electronic components -- not the drive trains, engines, and the other things. Look at the recent glitches you have seen at places like Toyota.

You take those three meta-level factors and put them together and we're making the morning broadcast news cycles now on a daily basis with, as you said, more and more of these embarrassing things coming to light. They're not just inconvenient, but there are monumental economic consequences -- and we're killing people.

Gardner: Kerrie Holley, we've looked at some of these issues -- society issues, organizational issues, and the technology behind them -- but technology has also been part of the solution or the ability to scale and manage and automate. I think service oriented architecture (SOA) has a major impact on that.

So, are we at a point where the ability of technology to keep up with the rate of growth is out of whack? What do you sense is behind some of this and why hasn't the technology been there to fix it along the way?

Holley: Jeff brought up some excellent points, which are spot-on. The other thing that we see is that we've had this growth of distributed computing. The easy stuff we've actually accomplished already.

If we look at a lot of what businesses are trying to accomplish today, whether it’s a new business model, differentiation, or whatever they're trying to do compete, what we are finding is that the complexity of that solution is pretty significant.

It's something that we obviously can do. If we look at a lot of technologies that are out in the market place, unfortunately, in many cases they are siloed. They repair or they help with a part of the problem, but perhaps they're not holistic in dealing with the whole life-cycle that is necessary to create some of this value.

Secondly -- this is a point-in-time statement -- we're seeing rapid improvements in the technology to solve this. With Jeff's company and other organizations, we are seeing that today. It hasn’t caught up, but I think it will. In summary, Jeff brought up several points in terms of the fact that we have ubiquitous devices and a tremendous amount of computing power. We have programming available to the masses. We have eight-year-olds, grandmothers, and everyone in between, writing software.

Connecting devices

We have a tremendous need to connect mobile devices and front-ends. We have 3D Internet. We just have an explosion of technologies that we have to integrate. Along with that comes some of the challenges in terms of how we make this agile, and how we make it such that it doesn't break. How do we make sure that we actually get the value propositions that we see? Clearly, SOA is a part of the solution, but it's certainly not the end-all in terms of how we repair and how we get better.

Gardner: One of the things that intrigues me about SOA is the emphasis on governance. To get the best out of a distributed services-orientation, you need to think at the very beginning and throughout the process about how to manage, automate, and reuse, as well as the feedback loops into the process -- all on an ongoing basis.

It strikes me that if that works for SOA, it probably also works for management and organizations, and it works for the relationship between workers and customers. Let me take this back to you, Jeff. Is governance also in catch-up mode? Do we have a sense of how to govern the technology, but not necessarily the process? Is that what's behind some of it?

Papows: You're right, Dana. There's a cultural maturation process here. Let's look at a couple of the broad economic planks that have affected how we got here, because I've been in the software industry for 30 years now. Remember that the average computer scientist, at least in North America, on average, makes 32 percent more than the mean average in the U.S. economy. And, software, computer services and infrastructure has accounted for about 37 percent of the growth in the gross domestic product in the United States and Asia in the last decade.

So the economic impact and success of our industry almost can’t be overstated. Because of that, we've grown up for decades now where we just threw more and more bodies at the problem, as
the technological curve grew.

All that means is automating those best practices and turning them inward, so that we’re governing ourselves as an industry the way that we would automate or govern many things.

There was always this never-ending economic rosy horizon, where you would just add more IT professionals and you would acquire and you’d merge systems, but rarely would you render
portions of those workforces redundant.

In 2008, the economic malaise that we’re managing our way through changed all of that. Now, the only way out of this complexity curve that we’ve created, to use Kerrie's terms, is turning the innovation that has been the hallmark of our industry back on ourselves.

That means automating and codifying all of the best practices and human capital that’s been in-place and learning for decades in the form of active policy management and inference engines in what we typically think of as SOA and design-time governance.

Really, all that means is automating those best practices and turning them inward, so that we’re governing ourselves as an industry in the same way that we would automate or govern many things. But now it’s no longer a "nice to have." I would argue that it’s critical, because the complexity curve and the economics have crossed and there is no way to put this genie back in the bottle. There is no way to go backward.

Gardner: Kerrie, any thoughts about what’s perhaps now a critical role for governance, perhaps governance up and down the technology spectrum, design time, runtime, but also governance in terms of how the people and processes come together?

Holley: Absolutely. One of the nice things that the attention to SOA has brought to our marketplace is the recognition that we do need to focus on governance. I don’t know of a single client who’s got an SOA implementation who has not, as a minimum, thought about governance. They may not be doing everything they want to do or should be doing, but governance is clearly on the attention span of everyone in terms of recognizing that it needs to be done.

So, when we look at governance and when we look at it around SOA, IT governance is something that we’ve had for a long time. SOA governance is a subset, you could say. It complements, but at the same time, it focuses our attention on, what some of the deltas have brought to the marketplace that require improved governance.

Services lifecycles

That governance is not only around the technology. It’s not only around the life-cycle of services. It’s not only around the use of addressing processes and addressing application development. Governance also focuses on the convergence that’s required between business and IT.

The synergistic relationship that we seek will be promoted through the use of governance. Change management specifically brings about a pretty significant focus, meaning that there will be a focus on the part of the business and the IT organizations and teams to bring about the results that are sought.

Examples of problems

Gardner: Jeff, in your book you identify some examples. Are there any that really stand out I that we can trace back to some root cause in the software lifecycle?

Papows: There are, and it’s unfortunate. The ones that make the greatest memory points and often the national headlines, characteristically are the ones that affect the consumer broadly as opposed to the corporate ones.

Obviously, Toyota is in the headlines everyday now. Actually, there was another news cycle recently about Toyota’s Lexus vehicles. The new models apparently have a glitch in the software that controls the balance system.

The ones that make the greatest memory points and often the national headlines, characteristically are the ones that affect the consumer broadly as opposed to the corporate ones.

One of the most heartbreaking things in the research for the book was on software that controls the radiation devices in our hospitals for cancer treatment. I ran across a bunch of research where, because of some software glitches and policy problems in terms of the way those updates were distributed, people with fairly nominal cancers received massive overdoses in radiation.

The medical professionals running these machines -- like much of our culture, because something is computerized -- just assume that it’s infallible. Because of the problems in governance or lack of governance policy, people were being over-radiated. Instead of targeting small tumors in a very targeted way, people’s entire upper torsos, and unfortunately, in one case, head and neck were targeted.

There are lots of examples like that in the book that may not be as ubiquitous as Toyota, but there are many cases of widespread health, power, energy, and security risks as a consequence of the lack of policy management or governance that Kerrie was speaking to just a few minutes ago.

Gardner: Well, these examples certainly are very poignant and clearly something to avoid. I wonder if these are also perhaps just the tip of the iceberg. In addition to things that are problematic at a critical level, is there also a productivity hit? Are large aspects of work in process not nearly as optimal as they could be or are plagued by mistakes that drag down the process?

I want to take this over to Kerrie. IBM has its Smarter Planet approach. I think they're talking about the issue that we're just not nearly as efficient as we could be. What makes the headlines are these terrible issues, but what we're really talking about is a tremendous amount of waste. Aren’t we?

Things we could do better

Holley: We are. That’s exactly what inefficiency is. It speaks to a lot of waste and a lot of things we could do better. A lot of what we’ve been talking about from a Smarter Planet standpoint is actually the exact issues that Jeff has talked about, which is that the world is getting more instrumented. There are more sensors. There is a convergence of a lot of different technology, SOA, business process management, mobile computing, and cloud computing.

Clearly, on one end of the spectrum, it’s increasing the complexity. On the other end of the spectrum, it’s adding tremendous value to businesses, but it mandates this attention to governance.

Gardner: Jeff, in your book do you offer up some advice or solutions about what companies ought to be doing in this governance arena to deal with these glitches?

Papows: We do. We talk about what I call the IT Governance Manifesto, for lack of another catchy phrase. I make the argument that it’s almost reached the point now where we need to lobby for legislation that requires more stringent reporting of software glitches in cases where there is human health and life at stake. Or, alternately, that we impose fines upon individuals or organizations responsible for cover-ups that put people at risk. Or, we simply require a level of IT governance at organizations that produce products that directly affect productivity and quality of life issues.

Kerrie said this really well, Dana. Remember that about 70 percent of our computer scientists in a given year are basically contending with maintaining the existing application inventories that run all of our financial transactions in core sub-systems and topologies. So, 70 percent of our human capital is there to basically keep the stuff that’s in place running.

So, 70 percent of our human capital is there to basically keep the stuff that’s in place running.

Concurrently, we have this smarter planet, where we’ve got billions of RFID tags in motion and 64-bit microprocessors have reached a price point where they are making the way into our dishwashers. We’ve got this plethora of hand-held devices and applications that’s exploding.

All of that is against the backdrop of this more difficult economy, where we can’t just hire more people without automation. We haven't a prayer keeping our noses about water here.

So, God forbid that we ask the federal government, which moves at a dinosaur’s pace relative to Internet speed, to intercede and insist on some of the stuff. But, if we don’t police our own industry, if we don’t get more serious about this governance, whether it’s IBM or WebLayers or some other technological help, we run the risk of seeing the headlines we’re seeing today become completely ubiquitous.

Gardner: Kerrie, I understand that you’re also penning a book, and it’s focused on SOA. First, could you tell us about it, but then are there any aspects of it that address this issue of governance, maybe from a self-help perspective and of not waiting for some legislation or external direction on it.

Holley: The book that’s going to be out later this year is 100 SOA Questions: Asked and Answered. What my co-author [Ali Arsanjani] and I are trying to accomplish in the book, which distinguishes us from other SOA books in the marketplace, is based on thousands of questions that we’ve experienced over the decade in hundreds of projects where we’ve had first-hand roles in as consultants, architects, and developers. We provide the audience with a hands-on, prescriptive understanding of some of the more difficult questions, and not just have platitudes as answers, but really give the reader an answer they can act on.

We’ve organized the content in a way that you can go by domain. If you’re a business stakeholder, you can go to particular areas. That gets back to your question, because business clearly has a big role to play here. The convergence or the relationship between business and IT has a big role to play.

You can go directly into those sections. We do talk about governance. The book is not about governance, but a good percentage of the questions are on governance. What we try to do is help organizations, clients, practitioners, and executives understand what works what doesn’t work.

Always a choice

One of the examples, a small example, is that we always have a choice when we do a project. We can do it in multitude of ways, but we have a lot of evidence that when governance is not applied, when it’s not automated, when it’s not thought about upfront, the expense on the back-end side is enormous. That expense could be the cost of not having the agility that you foresaw.

The expense could be not having the cost reduction that you foresaw. The expense could be the defects that Jeff has spoken about -- the glitches. There is a tremendous downside to not focusing on governance on the front-side, not looking at it in the beginning. The book really tries to ask and answer the toughest SOA questions that we’ve seen in the marketplace over the last decade.

Gardner: We’ll certainly look forward to that. Back to you Jeff. When we think about governance, it has a bit of a siloed history itself. There's the old form of management, the red-light, green-light approach to IT management. We’ve seen design-time governance, but it seems to be somewhat divorced from, even on a different plane than, runtime or operational governance.

What needs to happen in order to make governance more holistic, more end-to-end?

Papows: It’s a good question, Dana. It’s like everything else in our industry. We’re sometimes our own worst enemy and we get hung up on language, and God forbid, we create yet another acronym headache.

There's an old expression, "Everybody wants governance, but nobody wants to be governed." We run the risk, and I think we’ve tripped over it several times, where we get to the point where developers don’t want to be slowed down. There is this Big Brother-connotation at times to governance. We’ve got to explore a different cultural approach to it.

Governance, whether it’s design time or run time, is really about automating and codifying best practices.

Governance, whether it’s design-time or run-time, is really about automating and codifying best practices, and it’s not done generically as was once taught. It can be, in my experience, very specific. The things we see Ford Motor Co. doing are very different. They're germane to their IT culture and organization, and very different than what we see the Bank of America do, as an example.

To Kerrie’s point about the cost of a lack of automated best practices, if we can use the new verb, it isn’t always quantitative. Look at the brand damage to a bank when they shut customers out of their ATM network, the other side of turning the switch when they merged back-office systems. Look at the number of people whose automated payment systems and whatnot were knocked out of kilter.

The brand damage affecting major corporations is a consequence of having these inane debates about whether SOA is alive or dead, whether you need design-time governance or run-time governance. What you need is a way to automate what you are doing, so that your best practices are enforced throughout the development lifecycle.

Kerrie answered your question well when he said it really is about waste. It’s not just about wasted human capital or wasted productivity or cycles. It’s about wasted go-to-market opportunity. Remember, we're now living in the era of market-facing systems. For almost every major business enterprise, our digital footprint is directly accessible in the marketplace, whether it’s an ATM network or a hand-held device. The line between our back-office infrastructure and our consumer experience is being obliterated.

I'd argue that rather than making distinctions between design and run-time governance, companies simply, one way or another, need to automate their best practices. The business mandates of the corporations need to be reflected in an automated way that makes it manageable across the information technology life-cycle -- or you exist at your own peril.

Gardner: Kerrie, any thoughts on this concept of governance and how we make it more ubiquitous and more enforced as the pain and the problems grow evident? The solution at a high level seems pretty clear. It seems to be the implementation where we stumble.

Governance mindset

Holley: You hit it on the head, and Jeff made the point as well. A lot of people think governance is onerous, that it’s a structure that forces people to do things a certain way. They look at it as rigid, inflexible, unforgiving. They think it just gets in the way.

That’s a mindset that people find themselves in, and it’s a reason not to do something. But when you think about the goals that you're seeking, most goals have something to do with efficiency, lower cost, customers, and making the company more agile. When you think about this, pretty much everybody in the marketplace knows that you don’t get those goals for free. There is some cultural change that’s often necessary to bring those goals about, some organizational change.

There's automation. You don’t start with automation. You actually start with the problem, the processes, and picking the right tool. But, automation has to be a part of that solution. One end of the spectrum, we’ve got to address this mindset that governance gets in the way, that it’s overhead, and that it’s unnecessary.

We know that organizations that are very successful, that are achieving many of their goals, when we peel the onion back, we see them focused on governance. One advice that we all know is that you shouldn’t boil the ocean, that you should do incremental change. We also need to do this in governance.

We need to have these incremental successes, where we are focused on automation holistically and looking at the life-cycle, not just looking at the part-of-the-problem space.

Looking for automation as a way out of the hole that has been created is a consequence of the industry’s own success.

Gardner: Jeff, it sounds like governance needs a makeover. Is there an opportunity? You are going to be discussing this book at the IBM Impact Conference 2010, their SOA conference? Is this a good opportunity? You have a lot of IT executive and software executives from the variety of enterprises on hand, but what would you tell them in terms of how to make governance a bit more attractive?

Papows: We all need to say, "I am a computer science professional. We have reached a point in the complexity curve where I no longer scale." You have to start with an admission of fact. And the reality is that the demands placed on today's IT organizations, the magnitude of the existing infrastructure that needs to continue to be cared for, the magnitude of application demands for new systems and access points from all of this new technology, simply is not going to correlate without a completely different highly automated approach.

Kerrie is right. You can't boil the ocean and you can’t do it at once, but you have to start with an honest self-assessment that, as an industry, we can't continue to go forward at the rate and pace that we have grown, given everything we know and that we see, without finally eating our own cooking.

Looking for automation as a way out of the hole that has been created is a consequence of the industry’s own success. We didn't get here because we failed to be fair to all of those developers in the audience. They're going to listen to this and say, "Why am I the bad guy?" They're not the bad guys.

The reality is, as I said, that we're responsible for the greatest percentage of growth in the gross domestic product. We're responsible for the greatest percentage workforce productivity. We've changed the way civilization lives and works. We've dealt with a quantum leap -- and the texture of human existence is a consequence of this technology.

It's time that we simply admit that we need to turn back on ourselves in order to continue to manage this or we, literally, I believe, are on the precipice of that digital equivalent of a Pearl Harbor, and the economic and productivity consequences of failing are extreme.

Gardner: Well, we'll have to leave it there. We're about out of time. We've been discussing how glitches in business have highlighted a possible breakdown in the continuity of technology and that governance is an important factor in making technology continue on its productivity curve, without falling at some degree under its own weight.

I want to thank our guests. We have been joined today by Jeff Papows, President and CEO of WebLayers, and the author of the new book, Glitch: The Hidden Impact of Faulty Software. Thank you so much, Jeff.

Papows: Thank you, Dana, and thank you, Kerrie.

Gardner: And, we have been joined also by Kerrie Holley, an IBM Fellow as well as the CTO for IBM’s SOA Center of Excellence. Thanks for your input, and we will look forward to your book as well.

Holley: Thank you, Dana, and thank you, Jeff.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. Thanks for listening and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: WebLayers.

Transcript of a sponsored BriefingDirect podcast on the growing danger from faulty software and how to overcome it. Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.