The Dangers

of Technical
Debt (And What
CIOs Should Do
About It)
 The Real Costs of
Technical Debt
Imagine losing $54 million in an afternoon1.

That happened to Southwest Airlines in

July 2016, when a faulty network router
triggered a system-wide outage that
included its website and reservations
system. The backup systems that were
made for a situation like this faltered too,
turning what could have been a technical
hiccup into a week-long disaster. In the
end, more than 2,300 flights were forced
to be canceled or delayed. This has
happened to Southwest before. A year
earlier, another outage, caused by a
software glitch, resulted in 800 canceled
flights2.

Other airlines, too, are no stranger to

system outages like this. Delta, Jetblue,
and United have seen their fair share of IT
failures, often with losses in the hundreds
of millions. What do these airlines have in
common? Technical debt. Lots of it.

Of course, every organization has

technical debt. As the pace of technology
accelerates, so does the rate of
obsolescence. Question is: how much debt
are you carrying and what does it cost to
service that debt?

1
“Southwest Outage, Canceled Flights Cost an Estimated
$54M” Chicago Tribune. http://www.chicagotribune.com/
business/ct-southwest-outage-cancelled-flights-20160811-
story.html

2
“Outdated Technology Likely Culprit in Southwest Airlines
Outage”. NBC News. https://www.nbcnews.com/business/
travel/outdated-technology-likely-culprit-southwest-airlines-
outage-n443176

2 thinkwgroup.com
 Some organizations don’t recognize the debt they’re carrying
because the costs are insidious and multifaceted. They can show
up in the form of frequent service outages, which results in lost sales
The Real and poor customer service. They show up as a lack of agility, as
Costs of seen when the business waits months for what seem to be simple
application development enhancements that ultimately don’t
Technical
work as expected. These kinds of symptoms are caused by
Debt overcomplexity, driven by poor architectural practices. They’re
implemented over years of system add-ons, followed by more
add-ons to the add-ons. Instead of simplifying, the IT department
continues to invest in more tools and people. No one person can
explain how the system works end-to-end or what is the root cause
of major problems. The ultimate technical debt payment is
finding out the hard way that the disaster recovery plan doesn’t
work. Oops!

The interest that technical debt incurs shows up in the form of

complexity. The more complex your systems, the higher your interest
rate. And if the interest rate gets high enough the debt becomes
unserviceable—that’s when the CIO losses his or her job. But soon,
a new person comes in and exposes the technical debt problems,
everyone feigns surprise, and a large sum of capital is approved
to retire the debt, along with a catchy project name. Therein lies
the problem with technical debt: it is accrued based on seemingly
sound management decisions made over a long period of time.

thinkwgroup.com 3
The Real It happened when the decision was made not to convert that
system after an acquisition because it was going to cost too much,
Costs of so you ran two systems instead of one. Add that to the decision to
Technical stretch out the life of the operating systems that are now deemed
“unsupported” by the vendor, which also means unpatchable to
Debt remediate security flaws. It happened when the decision was made
to defer systems maintenance in favor of delivering new business
requirements. These decisions looked good at the time, but they
each contributed to the increased complexity of the system—and
the interest rate that you pay on the debt.

So how do you avoid getting in over your head in debt? The

answer is simple. Keep your systems as simple as possible and keep
complexity at bay.

How do you do that? Start by recognizing and tracking the amount

of complexity in your environment. All new projects will have an
impact on the overall complexity. Adding on to a system with
more functionality or size typically adds complexity. Replacing
architecturally non-conforming systems with ones that conform
reduce complexity. You can measure this by including a Complexity
Factor (CF) in your business case analysis as a kind of variable
interest rate for your debt. Some projects will make your interest
rate go up; others can make it go down. Recognize what impact a
project is going have on the overall environment. No one project is
going to have a huge impact on your CF. The point is, if you only
approve initiatives that make the CF increase without approving
ones to make it decrease then you’ll eventually get so far in debt
that you can’t service the interest.

Work with the business to agree on a methodology that calculates

CF and consider including that percentage as an added cost to
project financing, which then goes into a fund to finance negative
CF projects. We would recommend the methodology described
later in this paper as a starting point.

Even if you don’t use the CF to collect funds to finance remediation

projects, use the CF to communicate to the business and leadership
if you are heading into a danger zone that will eventually have
consequences. Build awareness that will create a new level of
scrutiny on all projects being approved.

4 thinkwgroup.com The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup
The Real What if your technical debt is already at a critical level? Conduct
a Technical Debt Assessment (TDA). A TDA is best done by a skilled
Costs of external organization who has no perceived biases and can look at
Technical things objectively. It’s necessary to drag all the skeletons out of the
closet and assess them for risk and cost of remediation. Only then
Debt can you develop a prioritized plan that will get your organization
back to a healthy debt position.

For Southwest, things may be looking up. Since the outage, the
airline has invested heavily in a technological overhaul that
includes a new reservations system3. One that the airline started
rolling out in May 2017 and will continue to transition to in the next
three years. Hopefully, they’ll keep track of their CF and justify
necessary remediation investments so that they can avoid the next
big outage.

3
“Southwest Airlines’ Digital Transformation Takes Off”. Fast Company. https://www.fast
company.com/3065045/southwest-airlines-digital-transformation-takes-off

thinkwgroup.com 5
 Assessing
Complexity
It is interesting to note that when we
examine the complexity of legacy
systems holistically, we come to see
opposing factors at work. For example,
the simpler the user interface, the more
complex the technology backend is
bound to be. We can expect the
software supporting the Apple iPhone
to be quite sophisticated, given that it is
a device universally recognized for its
highly intuitive interface.

For this paper, we shall not refer to this

relationship as complexity, but use the
more accurate term: sophistication.
As long as all the components within a
sophisticated solution conform to an
approved architecture and are aligned
to the needs of the business, then we do
not consider that solution to be complex.
Our definition of complexity is based
on the amount of non-conforming
architectural components within a given
solution. The greater the non-conformity,
the greater the Complexity Factor. Our
experience is that as the number of
architectural exceptions grows within a
system, so do the negative symptoms of
complexity that we described earlier.

6 thinkwgroup.com
 Based on our definition, we will discuss a Complexity Factor rating
methodology that deals with a heuristic, common-sense approach
Assessing to assessing the complexity of technical debt in relation to the
Complexity business it supports. We will do this by examining the four dimensions
of complexity that are directly impacted as a result of your
architectural practices:

1. Operational complexity
This is measured by the amount of difficulty you
currently face in making your system operate as
expected. The greater the difficulty, the more
complex in your system.

2. Support complexity
This is measured by your ability to respond to daily
problems. Even if a system is operating successfully,
it will need continued care to ensure users can use it
as designed.

3. Integration and technology complexity

This is measured by the maturity and consistency
of your solutions’ architecture. No system exists in
isolation. Eventually, they will need to communicate
with other systems to support a larger value chain.

4. Functional and usage complexity

This is measured by your system’s ease of use. Even
the best system will not perform well if users face
tremendous difficulty using it or if its functions do not
meet actual business needs.

The Complexity Factor self-assessment in the following section is

designed to be applied to any area or function. With it, you can
assess your entire system as a whole or apply it to a sub-system such
as payroll, purchasing, or manufacturing. While it’s designed to
provide a very high-level analysis, the results will allow you to
determine whether there’s a need to engage outside help to do
a deeper dive.

The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup thinkwgroup.com 7
 The Complexity Factor:
A Quick Start Self-Assessment
Dimension A B C
Operational complexity

Does the system consistently meet availability SLAs? Usually Sometimes Rarely

What portion of the operational budget is used to perform upgrades,

25% or less 26%–50% More than 50%
fixes, or workarounds?

Are there multiple systems supporting the same business functions? No Some Yes

How many components are still under vendor support? Most Some Few

What is your ability to promptly detect and protect against security With
Best practice Not good
breaches? deficiencies

Support complexity

How would you define your recovery processes? Best practice Unreliable Ad-hoc

Is it easy to find skilled resources for the system? Yes Somewhat Difficult

What is the availability of system documentation? Good Partial Lacking

How often do break-fix services meet SLA? Usually Sometimes Rarely

How often are proactive alerts and logs for analysis available? Generally Some Not available

Integration and technology complexity

How often is data is entered manually between systems? Some Regularly Typically

What proportion of your systems are centrally located and managed Spead
Most Several
(cloud/data center)? around

How often is file sharing used as primary means of data exchange? Rarely Frequently Primarily

Some local/
What interfaces do your applications have (web, Windows, Citrix)? Mostly web Mix
Citrix

What proportion of your applications and systems are aligned with

Significant Some Few
modern architecture standards?

Functional and usage complexity

Do problem tickets reported for system issues represent an undue

No Some Yes
precentage of total?

How reliable is your data? Always Usually No

How many manual processes does it take to perform tasks? Few Some Several

How many shadow systems does it take to perform functions? None Some Several

Less than Less than More than

How much user training does the system require for use?
a day a week a week

Total

8 thinkwgroup.com The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup
 Understanding
Your Complexity
Factor (CF)
Score

Once you have answered all the questions, add up the number
of checked boxes for each column A, B, and C. Then, multiply
the sum of column B by 3 and the sum of column C by 5. Finally,
add up all three values.

The Complexity Factor is thus calculated as follows:

CF = A + B*3 + C*5

The resulting CF score will be a number between 20 and 100.

If your CF score falls between 40 and 60, it is time to start thinking

about suitable system replacements that will keep complexity at
bay. With a score between 20 and 40, you should pat yourself
on the back and invite your team to celebrate. If you scored a
20, you’re in an elite class with few peers. Even if you achieve a
strong score, it’s a good idea to see if any one of the complexity
dimensions exceeded 15.

An imbalance between the dimensions is a weakness even if the

total CF score is good. For example, it may be that operations
alone contributed a CF of 20, which would be an indication to
consider a deeper dive and some outside help.

If your score is higher than 60, you are definitely having problems
and it’s time to do something about it.

The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup thinkwgroup.com 9
 Understanding
Your Complexity
Factor (CF)
Score

Quick-Start Self Assessment

Technical Debt Assessment Continuum and Attributes

0 20 40 60 80 100

Optimized Planned Manageable Challenged Chaos

Thoroughly Well-defined Architectural Ad-hoc or Poorly defined

defined architectural standards exist dated architecture
architecture standards architectural or one
Current standards created
architecture as part of a
project life
cycle

Strong Good Some Weak No

Governance governance governance governance governance governance

Future-state Future-state Evidence Ad-hoc Future-state

Future-state architecture architecture of some future-state architecture
architecture plans refreshed plans in place future-state architecture plans absent
annually architecture planning

Waiver Waiver limits Waivers Ad-hoc No waiver No waiver

management established formally waivers management management
and enforced reviewed

SLA SLAs typically SLA SLAs generally Constant SLA Poor SLA
management exceeded adherence met exceptions adherence

Consistent, Most often Reasonable Ad-hoc Consistently

Functional rapid delivery timely delivery delivery of delivery missing
delivery of new of new new of new delivery
functionality functionality functionality functionaliy targets

10 thinkwgroup.com The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup
Four Principles
for Avoiding
Technical Debt
The best way to avoid accumulating
technical debt is to not incur debt in the
first place. In the end, it behooves you as
a technology leader to ensure that no
avoidable technical complexity is
introduced into your projects.

Always keep in mind that the most

important variables in apportioning
complexity are business priorities and
technology capabilities. The trick is to
strike the proper balance in the
deliverable. A good technical delivery
will endeavor to place the bulk of
complexity inside the solution black box,
making the interaction with the software
simpler and more user-friendly, but not
at the cost of creating a solution that
cannot be implemented or be supported.

thinkwgroup.com 11
Four Use the Complexity Factor to gauge your overall debt levels
and apply the following four principles to keep it under control:
Principles
for Avoiding
Technical
Have a quality-first focus
Debt
01 When under pressure to deliver major functionality
quickly, never adopt a philosophy of “implement now
and fix later.” Later never comes. Don’t sacrifice quality
for speed.

Establish a well-understood architecture

02 Your architecture sets standards and filters out unnecessary

options that would otherwise add complexity. An
established architecture helps to get to solutions faster,
which speeds up delivery. Establish governance around
the architecture to ensure everyone follows the rules.

Adopt industry best practices

03 You don’t need to re-invent the wheel. There are all kinds
of industry best practices out there such as ITIL. When in
doubt, seek help from an expert.

Leapfrog

04 If there’s too much technical debt, take the leapfrog

approach. You probably don’t have an architecture or
aren’t following it. This is particularly true if your total CF
score is higher than 80. Create a green field environment
based on a future state architecture. Build the new
environment and convert the applications over one by
one. Let the old environment shrink down to nothing, just
like your technical debt.

12 thinkwgroup.com The Dangers of Technical Debt (And What CIOs Should Do About It) i COPYRIGHT © 2018 WGroup
Four As we’ve seen with the outages in the airline industry, the costs
of technical debt can be severe enough to cause a company
Principles to lose its competitive advantage and customers. Although
for Avoiding every organization pays for technical debt one way or another,
prevention will always be less costly.
Technical
Debt

Do you need a Technical Debt Assessment?

1. Are you experiencing a high number of outages?

2. Does it take too long to deliver new business

functionality?

3. Do the estimates for new projects shock you?

4. Are your trouble tickets trending upwards?

If you answered yes to any of the questions above, you

may want to perform a Technical Risk Assessment.

thinkwgroup.com 13
 About the Authors

William Mummery,
Principal
For more than 30 years, Bill has managed
large, complex IT environments by
developing pragmatic IT strategies that
are aligned with business needs. He is
adept at driving mission-critical programs
and possesses deep expertise in high-
availability systems and operations. His
industry experience spans healthcare,
financial services, government, insurance,
banking, and more.

Israel del Rio,

Principal
Israel is a leading IT executive and
technologist with more than 30 years’
experience. He has held CIO and CTO
roles, led global IT teams, and delivered
innovative solutions that have reduced
costs by the millions. His specialties include
IT strategy, IT management, and SOA.

14 thinkwgroup.com
Don’t Let Technical Debt
Hold You Back

Visit us at thinkwgroup.com
or give us a call at (610) 854-2700
to see what we can do for you.

About WGroup
WGroup is a management consulting
firm with a peer-to-peer approach to IT
optimization and transformation. The team
is comprised of consultants with over two
decades’ experience as former C-suite
executives and IT leaders. The firm boasts
a clientele that includes many Fortune
1000 companies across a wide range
of industries. WGroup is known for an
outcome-driven, service provider-agnostic
approach that optimizes IT operations and
minimizes costs.
OPTIMIZE IT

thinkwgroup.com
TEL: (610) 854-2700