Defining the Decision Factors for Managing Defects: A Technical Debt Perspective
Will Snipes, Brian Robinson
ABB Corporate Research
Industrial Software Systems
Raleigh, NC, USA
{will.snipes, brian.p.robinson}@us.abb.com
AbstractMaking a decision about whether to fix or defer
fixing a defect is important to software projects. Deferring
defects accumulates a technical debt that burdens the software
team and customer with a less than optimal solution. The
decision to defer fixing a defect is made by Software Change
Control Boards (CCBs) based on a set of decision factors. In
this paper, we evaluated the set of decision factors used by two
CCBs at ABB in the context of technical debt management.
The aim was to determine how a model of cost and benefits of
incurring technical debt could be part of the CCB decision
process. We identified the cost categories and decision factors
for fixing and deferring defects as a result of interviews with
CCB members. We found that the decision factors could
incorporate the financial aspects when using the technical debt
metaphor. We identify opportunities for further research to
integrate technical debt concepts with the decision factors
towards better long term outcomes.
Keywords-Technical debt; defects; defect costs; decision
factors
I. INTRODUCTION
Software development teams have the constant challenge
of delivering new feature requests while addressing defects
from current and past software releases. During a release
cycle, the volume of new change requests and defect reports
from test activities can exceed the volume that can be
addressed by development resulting in a backlog of requests
that must be managed.
Software change management, such as that practiced by a
Change Control Board (CCB)[1], is designed to help the
organization balance the competing demands of schedule,
new feature and defect change requests on the development
resources. Jones describes a CCB in [2] as a group of
managers, client representatives, and technical personnel
who decide which change to accept and which to reject.
When change requests, including changes to fix defects,
come to a CCB, they prioritize these requests and decide
when the changes will be implemented in a software product
release. As a result, change requests with higher priority are
addressed in the release they were proposed, while other
requests may be deferred. Over time the deferred change
requests (such as those to fix defects) accumulate and thus
the product incurs a form of technical debt. The technical
debt consists of change requests and defects that are deferred
to a later release. Defect debt consists of defects that the
978-1-4673-1749-8/12/$31.00
c 2012 IEEE
Yuepu Guo, Carolyn Seaman
Department of Information Systems
University of Maryland Baltimore County
Baltimore, MD, USA
{yuepu.guo, cseaman}@umbc.edu
CCB agrees should be fixed, but due to competing priorities
and limited resources have to be deferred to a later time.
Decisions made by the CCB to defer addressing defects
can accumulate a significant amount of technical debt for a
product. Tate expresses in [3] that a short term outlook on
maintaining technical debt can lead to a pivot point where
development teams spend so much time addressing defect
backlog that they cannot devote adequate time to deliver new
feature requests. Tate describes how the team may not be
aware of the burden because they can manage to barely keep
up with new requests but that those teams are in a short or
long death spiral leading to the point where their product
development becomes unsustainable.
As the technical debt metaphor suggests, deferring
defects may incur higher maintenance cost in the future, as if
paying interest for the defect debt. For example, using the
previously established criteria, each defect is evaluated for
cost to fix now versus cost to fix later. In the case of a low
severity defect representing a potential usability problem, it
may cost much less to fix the defect now than to process the
potential customer calls on the helpline as they struggle to
understand how to use the system. Thus a cost benefit
analysis of this scenario would support the decision to fix the
defect now even though it is low severity. From an
economics perspective, the goal of managing defect debt is
to minimize the overall long-run cost. Therefore, the quality
of a decision criterion in terms of defect debt is judged by
how well it can balance the short-term benefit, e.g. earlier
release date, with the long-term cost.
Currently Change Control Boards decide when to fix a
defect based on a set of defined criteria. The CCB decision
criteria involve consideration of a set of factors, e.g. what is
the risk of introducing a fix to the system? and how much
effort is needed to fix the defect? These criteria have a
significant influence over how much technical debt is
incurred from known defects. A given set of criteria may
allow a CCB to accumulate a larger volume of defect debt
than is prudent for the long term sustainability of the
product. Extending the above example, a criterion to only
fix defects with certain severity levels leads to an
accumulation of low-severity defects over time.
The goal of this study is to evaluate, in the context of
technical debt management, the decision factors that a CCB
considers when determining when to fix a defect. By
understanding the decision factors and the cost categories we
aim to incorporate a model of technical debt into the decision
54 MTD 2012, Zurich, Switzerland
criteria. In this study we identified various types of costs
related to fixing a defect, or deferring a defect and fixing it
later. Through interviews of CCB members, we identified a
set of factors that affect the decision on when to fix a defect.
Findings of this study include the set of factors influencing
the decision about when to fix a defect ranked by importance
and their relationship to the decision outcome. Based on the
findings, we propose further research to address gaps in the
decision criteria and enhance the criteria through cost-benefit
analysis. By introducing the debt concept into the decision
criteria, we expect the CCB to put more weight on the cost-
effectiveness of their decisions, thus ensuring the long term
maintainability and profitability of the software product.
In section II we review the relevant literature on technical
debt, defect management, cost analysis, and software
process. In section III the study design describes the
research questions, the approach to conducting the case study
with the target participants, and the data analysis method. In
section IV we review the demographics and findings of the
case study, and section V discusses conclusions and ideas for
future work.
II. RELATED WORK
There are multiple dimensions to classifying technical
debt. McConnell categorized technical debt as unintentional
and intentional debt [4]. The intentional debt was further
broken down to short term debt and long term debt. Based on
this classification, Fowler created a technical debt quadrant,
which consists of two dimensions deliberate/inadvertent
and reckless/prudent [5]. Each quadrant represents a
particular type of debt that takes the combination of the
values from each dimension. This type of classification is
helpful in finding the causes of technical debt, thus
contributing to technical debt identification. Technical debt
can also be classified in terms of the phase it occurs in
software lifecycle design debt, testing debt, defect debt,
etc. [6]. This type of classification ties technical debt to the
software development process. It indicates different forms of
technical debt may have different characteristics and hence
need different measures and approaches for management.
For example, the difference between expected code coverage
of the test suites and their actual coverage can be used to
measure testing debt. Our focus was on how defect debt is
created deliberately during the testing and maintenance
phase of the lifecycle by prudence oriented management
decisions of the timing for fixing an identified defect.
Since the ultimate goal of managing technical debt is to
facilitate decision making, it is necessary that all types of
technical debt have uniform measurement so that they can be
easily monetized and are comparable to one another [7].
Inspired by the metaphor, technical debt can be measured by
principal and interest. Principal refers to the cost of
completing a software maintenance task at present, while the
interest is the extra cost required to complete it in the future
if the task is postponed, plus the cost of other work that is
required due to the presence of the debt. As a particular type
of technical debt, defect debt is measured by the same
metrics. The principal of defect debt refers to the cost of
55
fixing defects at present. The interest of defect debt is the
cost associated with deferring the defects so they can be
fixed in the future plus the costs managing or working
around the presence of these defects until they are fixed.
Software cost estimation approaches would be required to
approximate the present and future value of technical debt.
The efficacy of the cost estimation approach will affect the
quality of technical debt measurement and eventually the
quality of decisions in technical debt management.
Therefore, the interviews in this study include questions
about the maturity of their cost estimation approaches.
Comparing technical debt to debt in its common sense,
such as financial debt, from which the debt metaphor
originates, technical debt has its unique characteristic the
interest of technical debt may or may not be incurred,
depending on whether changes will be requested in the
future. Therefore, the interest part in principal-interest
measurement of technical debt has been broken down to two
sub metrics interest amount and interest probability, the
latter of which captures the uncertainty of interest payment
[8]. In defect debt, the uncertainty is manifested as
contingency of the costs related to fixing defects. Therefore,
we investigated in this study not only the types of costs
incurred in the course of fixing defect, but also the
conditions under which the costs are incurred. Due to the
involved uncertainty, technical debt can be treated as a
particular software risk [9]. Thus risk management
approaches such as risk exposure analysis described by
Boehm in [10] can be applied to evaluate technical debt
impact and address technical debt management problem.
This motivated us to study whether and to how risk has been
used in the CCB decisions about when to fix a defect.
III. METHODOLOGY
This study consisted of two stages. In the first stage, we
reviewed the history of the defects in the selected subject
systems. The outcome from this stage was a set of cost
categories related to defect fixing, the conditions under
which the costs were incurred and some defect debt instances
that were then used for the interviews on the defect decision
making process. In the second stage, we interviewed the
CCB members who play different roles in the maintenance
of the subject systems. Through the interviews we identified
other costs related to fixing defects, which were missed in
the first stage, and combined them with the previously
identified cost categories. Then we identified the factors
affecting the decisions about when to fix a defect. We also
determined the impact of the decisions in terms of which
categories of costs tended to go up, and which tended to go
down, as a result of a decision to defer fixing a defect. Figure
1 illustrates the structure and steps we followed to conduct
this study. Stage one was a study of the defect records used
to identify the defect instances that where the fix was
deferred and the impact of the defer decision on the future
release. Stage two on the right hand side shows how the
interviews of the CCB members with a qualitative
questionnaire identified the cost categories for fixing and
deferring defects and the decision factors used by the CCB to
choose to fix or defer fixing defects.

Research Questions
Defect CCB
History Members
Decision Cost
Impact Categories
Interview
Review
Defect Decision
Instances Factors
Stage 1 Stage 2
Figure 1. Defect Debt Study Process.
A. Research Questions
To begin the study we formed key research questions that
address the goal of evaluating the decision process in the
context of technical debt management.
What are the cost categories associated with defect
fixes and defect debt? (1)
What factors affect decisions about incurring/paying
off defect debt? (2)
How do costs change with different defect fixing
strategies? (3)
RQ 1 determines the cost categories that will form the
costs and benefits for evaluating the technical debt equation.
RQ 2 identifies the factors the CCB considers when deciding
when to fix a defect. These factors have significant
influence over the decision and the relative level of influence
for each factor is captured in the study. RQ 3 investigates
the relationship between the cost categories and the decision
made about the defect. The cost change depends on both the
decision outcome and the decision factors that lead to the
outcome.
B. Data Collection
In the first stage, we collected information about defects
including registration date, description, severity, responsible
person, change history, handling process, disposal date, etc.,
from a defect database. A set of query criteria were
configured to meet the goal of identifying defects that the
CCB classified as should fix but that also were deferred to
a later release for fixing. The defect history record indicates
that a defect was postponed to a later release regardless of
the current open or closed status. The estimated effort to fix
was used to determine the amount of the principal of the
technical debt represented in this deferred defect.
In the second stage of this study we conducted interviews
with 7 CCB team members to discover their decision criteria
for fixing or deferring defects. The study used a semi-
structured interview strategy. The interviews involved two
researchers. One mainly served as an interviewer and the
56
other was mainly responsible for taking notes, but they
shared the two roles. After each interview, the notes taken
by the two researchers were compared and then combined to
form the answers to the interview questions. Then the
interview questionnaire and the notes, together with follow-
up questions on the interviewees responses were sent to the
interviewee. A formal follow-up interview session was also
arranged for two interviewees.
C. Interview Questions
Based on the research questions in Section II A, we
developed the interview questions. The interview
questionnaire is composed of four sections. The first section
contains the questions about the profile of the interviewees
including: What are your current title and responsibilities,
How long have you been involved in developing or
maintaining the system?
In the second section we ask questions about the profile
of the subject project or system including: what is the goal
of the project?, what is the biggest release or project in
terms of duration or effort? The questions in the first two
sections construct the context where defect debt is incurred
and thus help us understand how the decisions on when to fix
a defect are made.
Questions in the third section are regarding defect
properties and the costs associated with fixing a defect.
Some questions directly correspond to the research question
1, while others can be used to derive the answers for the
research question. To discover the maturity of the estimation
methods, we ask What is the way you usually estimate the
time and effort for handling defects?, and Was there a big
difference between actual cost of defect fixing and the
estimated cost?
To address RQ 3 determining how costs change with
different fix strategies, we ask What exceptions might
happen in the defect fixing process and how do you handle
the exceptions? We then ask the participant a directly about
RQ 1 with the question What are the costs and cost
categories related to fixing a defect? Then to determine the
effort for each category, we ask What percent of total cost
of handling a defect does each category contribute to?
With these questions, we expected to uncover the hidden
costs by exploring all possible activities that are performed
after a defect is detected. The final question taps into the
relations and change patterns of different cost categories in
the process of fixing a defect by asking If a defect is found
by customers, what additional costs does it have?
In the fourth section we framed a set of questions to elicit
how a decision is made about when to fix a defect and the
impact of the decision. We directly address RQ 2 by asking
what factors are considered when you make a decision
about when to fix a defect? and how are these factors
weighted? We also asked questions on a specific decision
path such as how do you decide to defer a defect? In
addition to these general questions, we asked the
interviewees to give examples of deferred defects, or we
provided the examples selected in the first stage of the study.
Based on concrete defect debt instances, we asked whether
the defects should be deferred in their opinion and why they
were deferred. Thereby we could elicit the actual decision
making process about deferring defects. This section also
contains questions on the impact of the decision. We ask
has a decision to defer a defect ever had any positive or
negative impact on the project, and if so, what was the
impact? At the end of this section we asked two questions
that needed to be answered in a retrospective way. The two
questions were have you made decisions on deferring defect
fixes? If so, was it worthwhile to defer defect fixes? and if
you were given the chance to do the project again, would
you make different decisions? Using these questions, we
expected to gather lessons on how defect decisions increase
or decrease the costs and cost categories for fixing or
deferring a defect. These questions provided insight into the
current decision making process and thus contributed to the
improvement of the process.
The entire interview questionnaire was refined several
times, e.g. merging or separating questions and adjusting the
order of the questions, before we implemented the interview.
To assure the quality of the interview, we reviewed the
questionnaire with an expert, who has rich experience in
software change control, and made changes he suggested.
D. Data Analysis
Since the data gathered in the first stage of this study
were used as input for the second stage, the data analysis
focused on the information we collected through the
interviews. Data collected in the second stage included the
notes taken during the interviews and responses to the
follow-up questions. Since most of the interview questions
were open-ended and responses were free form text, data
collected in this study were qualitative in nature. Therefore,
these data were analyzed using qualitative methods including
coding and constant comparison [11]. Based on the research
questions, we developed a coding scheme that contains a set
of codes and sub-codes, each of which represents a concept
or an aspect of our object of study. Figure 2 shows the codes
and how they are related to the research questions. Within
the Defect Cost code the Type is the cost category
discussed in section 4.A., the Condition code refers to
conditions under which the cost is incurred or changed, and
Change code is related to how the cost changes with a
decision to defer or fix. Within the Defect Decision code
the Factors refers to the criteria used to decide to fix or
defer a defect, the Process is related to how the decision is
made, and the Impact code refers to how the decision
affects the cost.
Coding Scheme Research Questions
Defect Cost
Type (1)
Condition
Change
Defect Decision (2)
Factors
Process (3)
Impact
Figure 2. Mapping Between Code Scheme and Research Questions.
57
Data analysis started with coding the interview notes and
the responses to the follow-up questions using the coding
scheme, i.e., assigning the codes to pieces of text that fit the
definition of the codes. Then relationships among the codes
were identified and confirmed using the grouped text as the
chain of evidence. For example, a cost change is part of the
impact of a decision. Finally these codes and relations among
them are collectively used to answer the research questions.
Using this analysis approach we identified a set of decision
factors employed in the current decision making process and
change patterns of various types of costs related to fixing a
defect. The results from the data analysis are presented in the
next section.
IV. RESULTS
We studied defect data from 5 past releases of two
products in an ABB control system. The products are
representative significant programs of greater than 1M
Source Lines of Code.
We selected a set of defects that were at some point
identified as deferred in the defect tracking system history.
Of the deferred defect set, 10% were identified as deferred
for a reason other than having a low impact severity. The set
of defects with medium or high impact severity that were
deferred were investigated in detail and discussed with
interview participants to confirm the decision criteria.
The interview portion of the study enlisted 7 participants
who were current members of the Change Control Board for
each product. The average experience of the participants
was 7 years with the company and 4 years in their current
role. The job roles of the interviewees included product
manager, test manager, and project manager. Interviews
were conducted via teleconference with two interviewers
talking with one participant in all interview sessions. Each
interview session lasted about an hour.
The interviews used a pre-defined set of questions that
included questions on what the cost categories for fixing
defect were and what the decision criteria were about when
to fix a defect, as described in section III.C. Results of the
interviews are discussed in the following two sections.
A. Defect Cost Categories
There is a formal process for handling defects in ABB.
Usually a defect will first be reported and confirmed once it
is detected. Then the maintenance team needs to diagnose
the defect to find the causes, come up with solutions,
implement the change and then validate the change (testing).
Since cost is incurred in any link of this activity chain, there
are various types of costs associated with handling defects.
These costs largely fall into six categories investigation,
modification, workaround, customer support, patch and
validation cost.
Investigation cost refers to the cost of diagnosing a
reported problem, verifying that it is a defect, finding the
causes and providing possible solutions. A developer
conducts the investigation by performing causal analysis and
replicating the defect in the development environment (if it is
a customer reported issue). The developer defines one or
more fix proposals to repair the defect, describes the risk of
making the change(s) and the affected test scenarios and incurred when a decision is made to defer the defect. The
whether a system level validation step is required. risk of a customer requesting a patch increases when a defect
Investigation cost is estimated by interviewees at percentages is deferred. The cost of validation decreases with a deferred
between 50-70% of the cost of fixing a defect. defect because the test of the workaround requires less effort
Modification cost refers to the cost of implementing the than that of the fix. There are other types of costs, such as
proposed changes to fix a defect. This includes the cost to CCB meetings, but they are minor costs compared to those
implement the code and verify the fix with inspections, unit listed in Table I.
testing, and functional testing in the development
environment. It does not include costs of system validation
that are reported separately. Modification cost is estimated TABLE I. CHANGE PATTERNS OF DEFECT COSTS
by interviewees as 10-15% of the cost of fixing a defect. This
Cost Category Fixing Deferring Condition
is small because most of the work to develop the fix is
Investigation
performed during the Investigation, thus Modification is
simply implementing and retesting the selected fix design Modification
alternative.
Workaround
Workaround cost refers to the cost of providing
workarounds to bypass a known failure trigger when the Customer support Customer Request
underlying defect is not fixed right away. The cost for a
Patch Customer Request
workaround includes documentation and verification system
test environment. Workarounds may be suggested by any Validation
team member who works on the defect and must be : Incurred, : Increase, : Decrease
approved by the product manager as acceptable to the
customer. Table II shows the cost categories cited by participants
Customer support cost refers to the cost of providing with the percentage of respondents rating for the cost level of
support for a customer who encounters a problem due to a the category. The question was asked as a percentage, but
defect in the system. A change or a workaround may require the respondents answered with a range that we classified into
reeducation of system users and support cost may increase the three levels shown. We classified cost as high when
until users become familiar with the new operating participants gave answers in the range of 50-70%; Medium is
procedure. 20-30% range and low is 10-15% range. 71% of the
Patch cost refers to the cost of providing a temporary fix participants gave Investigation as a cost category and all
for the post-release defects to affected customers usually cited it as a high cost. 45% of participants gave the Patch
upon their request. The patch cost may vary across systems cost category with 14% indicating it was a high cost and
where some systems have binary patch capabilities that 29% indicating a medium cost. The Validation category
reduce the risk and scope of the change while others require ranked as medium by 57% of participants. The Customer
delivering an entire build. The latter have significantly Support category was given by 14% who indicated it was a
higher costs for this type of repair because it includes low percentage of the cost. The modification cost was given
validation and deployment of the patch build. by 57% of participants who all considered it as low. 14% of
Validation cost refers to the cost of testing in the context participants included project management cost as a low
of the complete system. This type of testing requires a percentage. The cost for a work around was identified by
separate team with specialized lab equipment for simulating 14% of participants as a medium cost.
a customer operational environment for the entire system.
Validation cost is 20-30% of the cost for fixing a defect TABLE II. PARTICIPANT CITING OF COST CATEGORIES
though this cost may be spread across a set of defects
delivered into a maintenance release. Cost Category High Medium Low
Compared to fixing a defect at present, deferring the Investigation 71% 0% 0%
defects may eliminate some categories of costs, but it may
Patch 14% 29% 0%
incur new categories of cost. Table I lists different types of
costs associated with defect fixing, the conditions under Validation 0% 57% 0%
which they are incurred and how they change if the defect is Customer Support 0% 0% 14%
deferred. Even within the same cost category, the cost may
increase or decrease with the decision to defer a defect. For Modification 0% 0% 57%
example, the cost for investigation increases with a deferral Project Management 0% 0% 14%
decision according the respondents because the investigation
Workaround 0% 0% 14%
must be repeated by the person who fixes the defect in a
subsequent release. The cost to modify stays the same as
long as the same solution is used. We incur a cost of The main cost in handling a defect comes from
customer support when a defect is deferred because it is investigation through which the cause and proposed fixes for
more likely for the customer to complain when the encounter the defect are identified by the developer. Validation cost
the failure mode for the defect. The cost for a workaround is takes the second place. Modification cost is relatively small

58
compared to investigation and validation cost. In most cases
deferring a defect will reduce the cost if no customer detects
the defect. When a customer finds a defect, however, the
total cost significantly increases if a patch must be issued for
a deferred defect. This case is discussed further in the
review of the decision factors in the next section.
B. Defect Decision
As previously stated, the decision factors and criteria for
when to fix a defect heavily influence the accumulation of
technical debt in the form of deferred defects. According to
the respondents, the key factors in order of importance are:
Severity
Existence of a workaround
Urgency of fix required by customer
Effort to implement the fix
Risk of the proposed fix
Scope of testing required
The severity factor explores the capabilities that are
affected by the defect that will concern the customer. The
extent of the impact may range from a minor irritation to a
critical impediment to normal system operations. The
importance of the function affected is the primary concern
for this factor.
The existence of a workaround makes it possible to defer
the fix to a later release. A developer or product manager
may identify a workaround in the process of investigating the
defect. The product manager approves the workaround as
acceptable and the team validates workaround in a simulated
operational environment.
Occasionally the customer will request that a particular
defect be addressed urgently. This criterion trumps others
and triggers the development validation and release of a
patch build to correct the defect.
The effort required to fix the defect is weighed by the
CCB against available resources and schedule and the
relative priority of this defect to fix within these constraints.
The developer estimates the effort and time required to
implement a fix and test the fix. The test organization gives
an estimate of effort and time to validate the fix.
The risk evaluation of the proposed fix evaluates the
extent of code and functionality that will be changed. This
also gives information to determine the scope of verification
and validation required to ensure the fix is correct. The
developer assesses the risk of the proposed fix solution for
the defect. The evaluation includes affected software
components and functions, classes within those components,
and their methods.
The scope of testing required considers the impacted
requirements and functions to determine the scope of
selected test cases and whether system regression needs to be
run on the resulting fix. The impact of the fix to non-
functional requirements is also considered particularly for
the areas of performance, scalability, usability, and
maintainability.
Questions that are considered but are extraneous to the
decision of whether to fix or defer include determining the
root cause of the problem, and identifying the updates
59
required to architecture models, design documentation, and
user documentation. The final extraneous question is how the
problem could have been identified earlier in the life-cycle.
V. DISCUSSION
Through this study we identified various types of costs
associated with fixing a defect. We also found the conditions
under which the costs are incurred and their change patterns.
Findings of this study indicate that it is not certain that
deferring a defect will increase or decrease the overall cost.
In other words, we cannot simply conclude, using only the
results of this study, that incurring defect debt will have
positive or negative impact on the project. However, the
results show that the total cost will increase significantly if
the customer requests a patch for a deferred defect. In
addition, it is true in most cases that deferring a defect will
reduce the cost if no customer detects the defect. Therefore
whether a defect will be detected by a customer drives the
cost to benefit ratio of fixing a defect or deferring the fix.
The defect decision making process has been studied
from several perspectives but, until now, not the technical
debt perspective. When we look at this problem from a
technical debt perspective, we can identify some
improvements to the decision factors provided by the CCB
members. The defect factors and decision making process
are mainly based on severity analysis and risk. Although cost
and schedule were considered in the analysis, a rigorous
cost-benefit analysis is not typically used. Therefore it is
possible that incorporation of cost-benefit analysis could lead
to better decisions for long term product profitability and
customer satisfaction.
To include technical debt as part of the CCB decision
process we need to develop a cost-benefit analysis that
incorporates the cost factors identified by the CCB
participants. With a cost-benefit analysis model each cost
category becomes part of either the cost or benefit side of the
equation. Based on the identified cost categories we build
the cost benefit analysis model with the following variables.
The variable P includes the cost categories of Investigation,
Modification, and Validation representing the cost to pay off
the debt at present. P is thus the principal of the debt in the
cost-benefit ratio. Let Iw represent the portion of the interest
cost for defining a workaround. Let Ic be the interest cost for
customer support of the workaround. Let Ip represent the
patch cost and Ipr represent the probability that the customer
will request a patch for the defect. The value of Ip * Ipr is the
expected value for the patch cost similar to the expected
value of the cost to patch the defect if required. Let Ifr
represent the probability that the deferred defect is eventually
fixed (the principal repaid). Thus P * Ifr is the expected
value of the cost to pay off the debt.
Thus the cost-benefit ratio r is given in the following
equation:
=
+ + +
Figure 3. Equation for Cost Benefit Analysis Ratio
 When the cost benefit ratio is < 1 it is to the advantage of
long term cost to fix the defect now (pay the principal).
When it is >1 there is a financial advantage to defer fixing
and pay the interest. This is a simplistic equation because it
does not consider the effects of time on factors such as Ic and
Ipr, nor does it consider the opportunity cost of other work
the developers could be producing rather than fixing defects.
Outside the boundary of a suitable equation for a specific
defect, there is the concept of the impact of accumulated
defect debt on customer satisfaction and the ability of the
company to sell their product in the marketplace.
We define the cost benefit analysis ratio as an additional
factor in the decision to fix or defer a defect. The cost
benefit analysis ratio replaces factors of the existence of a
workaround and the effort to fix with the cost benefit
analysis that incorporates these factors. Other factors remain
to consider the other aspects related to severity, customer
priority, and risk separately.
We discussed implementing this model with the CCB
participants. The CCB found the model interesting and
potentially useful for their decision making process. We
identified impediments to implementing the model including
the key impediment of estimation of the cost factors that are
part of the interest equation. CCB members find it difficult
to estimate the patch risk factor and customer support cost
because these relate to whether the customers will encounter
the defect. A secondary impediment is developing tool
support for the cost benefit analysis that integrates with their
normal workflow.
Another key question is what should be the
organizations policy with regard to incurring defect debt. In
the finance world there is the concept of a debt to income
ratio that tells lenders the ability of borrower to service and
retire the debt. Given that technical debt exists in software,
what is the appropriate income factor to use in the
corollary ratio? Is a policy that drives the product towards
zero debt the best for all stakeholders?
In this study we determined the cost factors for fixing or
deferring defects and the current decision factors used by the
CCB. We determined that these decision factors could
incorporate a financial model of cost benefit analysis for the
fix vs. defer fix decision. We can use the metaphor of
technical debt to define an economic model of a decision to
fix or defer fixing a defect. A cost model can balance the
60
costs of implementing and testing an identified fix against
costs of testing a workaround and future support costs and
re-investigation costs. We plan further research to evaluate
the use of a cost model combined with the current decision
factors in CCB decision making. Addressing the
impediments and key questions is also the target of future
work on technical debt management.
ACKNOWLEDGMENT
The authors would like to thank the participants in this
study from ABB and the support of ABB Corporate
Research for this work. Guo and Seamans work is
sponsored by the US National Science Foundation grant
#0916699.
REFERENCES
[1] "IEEE standard glossary of software engineering terminology," Tech.
Rep., 1990.
[2] C. Jones, "Strategies for managing requirements creep," Computer,
vol. 29, no. 6, pp. 92-94, Jun. 1996.
[3] K. Tate, Sustainable software development : an agile perspective, A.
Cockburn and J. Highsmith, Eds. Addison-Wesley, 2006.
[4] S. McConnell. (2007). 10x Software Development. Available:
http://forums.construx.com/blogs/stevemcc/archive/2007/11/01/techni
cal-debt-2.aspx
[5] M. Fowler. (2009). Technical Debt Quadrant. Available:
http://www.martinfowler.com/bliki/TechnicalDebtQuadrant.html
[6] J. Rothman. (2006). An Incremental Technique to Pay Off Testing
Technical Debt. Available:
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectT
ype=COL&ObjectId=11011&tth=DYN&tt=siteemail&iDyn=2
[7] N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim, P. Kruchten, E. Lim,
A. MacCormack, R. Nord, I. Ozkaya, R. Sangwan, C. Seaman, K.
Sullivan, N. Zazwork, Managing Technical Debt in software-reliant
Systems, a, Proceedings of the 18th FSE/SDP Workshop on Future of
Software Engineering Research, 47-52, 2010
[8] C. Seaman and Y. Guo, Measuring and Monitoring Technical Debt,
Advances in Computers, vol. 82, pp. 25-46, 2011.
[9] Y. Guo, C. Seaman, A Portfolio Approach to Technical Debt
Management, , Proceeding of the 2nd Workshop on Managing
Technical Debt, 31-34, 2011
[10] B. W. Boehm, "Software Risk Management: Principles and
Practices," IEEE Software, vol. 8, pp. 32, 1991.
[11] M.B. Miles, A.M. Huberman, "Qualitative Data Analysis: An
Expanded Sourcebook", 2nd ed., Sage, 1994.