QMIS

Uarterly
E xecutive

IT Project Estimation: Contemporary

Practices and Management Guidelines
Many IT projects continue to suffer from poor estimation. Indeed, the accuracy of
estimation has hardly changed from that reported in a seminal study carried out over
20 years ago. Based on findings from two recent survey-based studies, which replicated
and then extended the original study, we provide guidelines for improving IT project
estimation, taking account of the greater use today of Agile, rather than traditional
Waterfall, development methods.1,2

R. Ryan Nelson Michael G. Morris

University of Virginia (U. S.)

Many IT Projects Still Suffer From Poor Estimation ,

1 2

Despite a great deal of attention in the trade and academic press, IT projects continue to fail
at an alarmingly high rate. One of the most-cited reasons for these failures is poor estimation
practices.

Unrealistic expectations based on inaccurate estimates are the single largest cause of [IT
project] failure.3

Estimation is defined as an informed assessment of an uncertain event. For IT project

managers, accurate estimates are the foundation for effective project planning and execution
and, ultimately, project success. Unfortunately, most project managers do a very poor job
of estimating and, as a result, most IT projects are classified as failures61% in the latest
Standish Group report.4 The Standish Groups data shows some improvement in the overall
success rate since 2004 (which it partially attributes to the Agile development process and
improved project management expertise). However, its figures show a slight increase in both
time and cost overruns since 2010signaling that there is still much room for improvement.
According to Standish, 74% of challenged projects experience time overruns and 59%

1 Leslie Willcocks is the accepting senior editor for this article.

2 The authors would like to thank Steve McConnell and Arin Sime for their input throughout this research project. We would
also like to acknowledge the McIntire School of Commerce for providing financial support for this research project, thank research
assistant Meg Raymond and thank the Project Management Institute for posting a link to our survey.
3 Futrell, R. T., Shafer, D. F. and Shafer, L. I. Quality Software Project Management, Prentice Hall, 2002.
4 The Standish Group: Chaos Manifesto 2013, available at http://versionone.com/assets/img/files/ChaosManifesto2013.pdf.

March 2014 (13:1) | MIS Quarterly Executive 15

IT Project Estimation
experience cost overruns. Further evidence can
be found in a review of 180 IT projects completed
between 1999-2013,5 64% of which suffered from
poor estimation.
In this article, we examine the practice of IT
project estimation, report the findings from two
studies and provide recommendations to help
project managers improve project estimation.
An estimate is the most optimistic
prediction that has a non-zero probability
of coming true Accepting this definition
leads irrevocably toward a method
called whats-the-earliest-date-by-which-
you-cant-prove-you-wont-be-finished
estimating.6
Why Estimates Matter
Accurate estimation is very important to IT
project managers for a wide variety of reasons:
Avoiding the vicious cycle: poor estimation
results in more schedule pressure, which
in turn creates more stress, producing
more mistakes, and ultimately more
schedule slips, creating even more
schedule pressure, and so on.7
Avoiding the ripple effect: slippage in one
projects completion date can have a ripple
effect on other projects and stakeholders
throughout the organization. Effective
organizations require a portfolio of well-
coordinated projects to succeed, and better
estimation leads to better coordination
with other tasks and projects throughout
the portfolio.8
Avoiding late-in-the-project discovery that
the project has been underestimated,
5 See Nelson, R. R. Project Retrospectives: Evaluating Project
Success, Failure, and Everything in Between, MIS Quarterly
Executive (4:3), 2005, pp. 361-372; Nelson, R. R. IT Project
Management: Infamous Failures, Classic Mistakes, and Best
Practices, MIS Quarterly Executive (6:2), 2007, pp. 67-78; and
Nelson, R. R. and Jansen, K. J. Mapping and Managing Momentum
in IT Projects, MIS Quarterly Executive (8:3), 2009, pp. 141-148.
6 DeMarco, T. Controlling Software Projects: Management,
Measurement and Estimation, Yourdon Press, 1982.
7 McConnell, S. Software Estimation: Demystifying the Black Art,
Microsoft Press, 2006.
8 Boehm, B. Software Engineering Economics, Prentice-Hall, 1981.
This book contains a thorough discussion of software estimation and
scheduling, which is presented in terms of Boehms COCOMO cost-
estimation model.
16 MIS Quarterly Executive | March 2014 (13:1)
which is difficult, if not impossible to
correct.9
Facilitating better budgeting: accurate
budgets depend on accurate estimates of
size, effort and time.10
Evaluating project personnel:
compensation is often tied to project
performance.
Generating more credibility for the project
team: being part of a team that brings a
project in on budget and schedule can do
wonders for an individuals career, not to
mention job satisfaction.11
In practice, estimation is both an art and a
science. While the art of estimation is closely tied
to personal experience, heuristics and common
sense, the science of estimation can be based on
complex statistical analysis, algorithms, historical
data and computer-based tools.
To gain a better understanding of exactly how
these various methods are used in practice, we
conducted two research studies, focusing on
contemporary estimation practices and their
linkage to project success. The first study focused
on an organizational-level analysis of project
estimation. Results from 60 IT professionals
suggested that, consistent with responses
from a similar study carried out 20 years ago,
there is still much room for improvement in
how estimates are generated and used within
organizations. Moving down a level of analysis,
the second study focused on project-level data
from 115 IT professionals. It also compared
traditional Waterfall and Agile development
methods with respect to estimation practices and
project success. (The survey methodologies used
for each of these studies are described in more
detail in the Appendix.)
Research Study 1:
Organization-Level Analysis of
Project Estimation
In 1992, Lederer and Prasad published the
results of a survey of 115 IT professionals on their
9 Brooks, F. The Mythical Man-Month, Addison-Wesley, 1975.
10 Jones, C. Estimating Software Costs, McGraw-Hill, 1998.
11 DeMarco, T. and Lister, T. Peopleware: Productive Projects and
Teams (3rd Edition), Addison-Wesley, 2013.
misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines
Table 1: Importance and Accuracy of IT Project Estimation
The accurate estimation of IT projects is moderately/very important.
What percentage of all IT projects significantly overrun their estimates?
What percentage of all IT projects significantly underrun their estimates?
What percentage of all IT projects are completed at cost close to estimate.
organizations cost estimation practices,12 which
we have used as a baseline for our research.
That study provided an in-depth understanding
of the estimation process in the early 1990s
an era of organizational computing marked by
the rapid diffusion of the personal computer, the
rise of network computing (before the takeoff of
the Internet) and the beginning of the business
process reengineering movement. Lederer and
Prasads research exposed a stark contrast
between the perceived importance of accurate
project estimation and the state of practice, and
they prescribed a set of management guidelines
aimed at improving estimation practices.
The primary objective of our first study was
to discover how project estimation perceptions
and practices have changed over the more than
20 years since Lederer and Prasads original
study. We modified the survey used by Lederer
and Prasad and distributed it to a similar target
population.13 The results of our study are
compared and contrasted with the Lederer and
Prasad findings below.
There is Still a Significant Need for
Estimation Improvement
After 20 years, there is still a profound need
for improvement in IT project estimationas
highlighted in Table 1. Although the vast majority
of our respondents believe that estimation is
important (88% described it as very important
12 Lederer, A. L. and Prasad, J. Nine Management Guidelines for
Better Cost Estimating, Communications of the ACM (35:2), 1992,
pp. 51-59. This was one of Lederer and Prasads original studies on
cost estimation, with other informative articles being published over
a 10-year period.
13 While the general profile of respondents and companies in
our study is similar to the Lederer and Prasad sample, there was a
significant difference in industries representedwith their largest
percentage of respondents coming from manufacturing (32%),
compared with service industries (31%) in our sample.
Lederer &
Study 1
Prasad
84% 88%
63% 56%
14% 13%
23% 31%
or moderately important, the highest two
possible ratings on the five-point scale), the
data also suggests that only 31% of projects are
completed reasonably close to their estimated
time or cost. The good news, however, is that our
study shows that both estimation awareness and
performance have improved somewhat since the
early 1990s.
How Estimates Are Used
As stated above, estimates can be important
for a variety of reasons. Table 2 shows that
estimation seems to be used more for project
planning and control (rows 1 to 5) than for
evaluating estimators, developers and others
involved in a project (rows 6 to 8). These findings
are consistent with Lederer and Prasads findings,
with the top five remaining the same, albeit in a
different rank order.
Our respondents were then asked about
their organizations most common estimation
practices, and the top five responses are listed
in Table 3. Interestingly, although 75% of
projects use formal estimates, only 54% involved
developers in the estimation processsuggesting
that 21% of projects do not have any estimation
input from the developers even though they use
formal estimates. This finding is contrary to the
first of Lederer and Prasads Nine Management
Guidelines for Better Cost Estimating: Assign
the initial estimating task to the final developer. A
second Lederer and Prasad guideline is Monitor
the course of a project from the preparation of
the estimate through project completion. This
practice was reported for 71% of projects in our
study, which is virtually identical to the 70%
reported by Lederer and Prasad in 1992.
March 2014 (13:1) | MIS Quarterly Executive 17
IT Project Estimation
Table 2: How Estimates are Used
Use of Estimate
1. To select proposed projects for implementation
2. To staff projects
3. To schedule projects
4. To quote the charges to users for projects
5. To control or monitor project implementation
6. To evaluate project estimators
7. To audit project success
8. To evaluate project developers
Table 3: Project Estimation Practices
Top 5 Responses
1. Formal estimates are prepared
2. Formal monitoring of project progress
3. Estimate revised because requirements change
4. Cost-benefit analysis used to justify project
5. Developers participate in estimation
The Estimation Process and Causes of
Inaccuracies
Respondents were asked about the basis they
used for their estimations. As seen in Table 4,
they rely most heavily on comparisons to similar,
past projects when arriving at their estimates
whether these comparisons are based on
personal memory or on documented facts. It is
interesting to note that the rank ordering of the
bases by our respondents is virtually identical to
what Lederer and Prasad found in their research
20 years ago (only items 4 and 5 were reversed in
order).
Our survey included a question on peer or
team-reviewed estimates, which was not included
in the original Lederer and Prasad survey,
because this practice is promoted by the use of
Agile development. Our respondents reported
that this practice was the third-most extensively
used basis for estimation.
18 MIS Quarterly Executive | March 2014 (13:1)
Importance of Use
(1 = Very unimportant;
5 = Very important)
4.40
4.20
4.17
4.04
3.93
3.26
3.33
3.13
Percentage of IT
Projects
75%
71%
63%
61%
54%
The final part of our study replicating the
Lederer and Prasad study was to ask respondents
to identify the factors most responsible for
inaccurate estimatesgiving them a total of 26
causes to choose from (the top five are listed in
Table 5). Again, these findings are remarkably
consistent with Lederer & Prasads (the notable
exception being that Insufficient user-analyst
communication and understanding dropped
from fourth position in their study to ninth in
ours).
The clear message continues to be that initial
estimates will be inaccurate because of frequent
changes by users, lack of users understanding
of requirements, inadequate task/problem
identification and insufficient analysis at the
estimation stage. As a result, estimators are
encouraged to revise estimates during the
course of the project (reported by 63% of
our respondent organizations) and to delay
committing to a target for as long as possible.
misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines

Table 4: Bases of the Estimating Process

Extensiveness of Use
Basis (1 = Not used at all;
5 = Extremely extensive)
1. Comparison to similar, past projects based on personal memory 3.48
2. Comparison to similar, past projects based on documented facts 3.45
3. Peer or team-reviewed estimates 3.08
4. A simple arithmetic formula (such as summing task durations) 2.96
5. Intuition 2.94
6. Guessing 2.76
7. Established standards (such as averages, standard deviations,
2.71
etc.)
8. An estimation tool (e.g., software package) 1.63
9. A complex statistical formula (such as multiple regression,
1.57
differential equations, etc.)

Table 5: Causes of Inaccurate Estimates

Extent of Responsibility
Causes (Top 5 Out of 26) (1 = Not resp. at all;
5 = Extremely responsible)
1. Frequent user requests for changes 3.70
2. Users lack of understanding of their own requirements 3.54
3. Overlooked tasks 3.35
4. Poor or imprecise problem definition 3.26
5. Insufficient analysis when developing estimate 3.11

Research Study 2: Project-Level changes in the IT development landscape since

Analysis of Project Estimation
Study 1 enabled us to gain an overall view
of organizational project estimation practices.
We wanted to supplement these findings
with a more detailed second study of project
estimation practices that focused on project-level
data. In Study 2, we therefore surveyed 115 IT
professionals and asked them to respond with
data about their most recently completed IT
projects.
Another factor we wanted to explore in Study
2 (based on what we learned in Study 1 as well
as input from two software estimation experts
consulted throughout the research project) is the
Lederer and Prasads 1992 study. Specifically, the
growth of Agile development has fundamentally
altered the way in which projects are conceived
and managed. Collecting data at the project level
allowed us to learn more about the usefulness
and accuracy of project-estimation practices in
the context of specific development environments
(e.g., Agile vs. traditional Waterfall development),
which obviously carries important implications
for IT managers today.
As shown in Table 6, we found that traditional
Waterfall methodologies (or variants) were
still the most widely used. However, Agile
methodologies made up a significant proportion

March 2014 (13:1) | MIS Quarterly Executive 19

IT Project Estimation

Table 6: Development Methodologies

Methodology Count Percentage
Waterfall (or variant) 45 47.9%
14
AgileScrum 11 11.7%
Agileother 19 20.2%
Other (e.g., ad hoc and hybrid) 19 20.2%

Table 7: Cost Performance

Estimates vs. Actuals Total Waterfall Agile
Over budget 40% 47% 33%
Under budget 31% 33% 33%
On budget 29% 20% 33%

Table 8: Schedule Performance

Start Time Finish Time
Total WF Agile Total WF Agile
Late 36% 31% 30% 61% 67% 53%
Early 0% 0% 0% 17% 13% 13%
On Time 64% 69% 70% 22% 20% 33%

(nearly a third) of all projects in our Study 2
sample. 14
Consistent with the findings of Study 1, the
94 projects in Study 2 for which estimated vs.
actual costs were reported tended to come in over
budget, although there was also a healthy number
of projects on or even under budget in the sample
(see Table 7).
Clearly, the 40% of projects reported as
over budget indicates that estimation and/
or execution problems persist in IT projects.
However, the 60% of projects that are on or
under budget is a hopeful sign that the project
management discipline is progressing. Moreover,
Agile projects seem to be performing better than
Waterfall projects in terms of cost performance
against budget. Of the 38 (out of 94) projects
that were over budget, the average percentage
over the original estimate was nearly 73%, with a
range between 1% and 900%. For the 29 projects
under budget, the range was much smaller, from
2% to 93%, with an average of 17%.
14 Schwaber, K. and Beedle, M. Agile software development with
Scrum, Prentice Hall, 2002.
In terms of schedule compared to the original
estimate, on average, projects started 25 days
late (with none starting early), and finished an
average of 56 days late (with a maximum of 3.1
years). Once again, Agile projects tended to start
and finish closer to their estimated times. Table 8
summarizes the results for project timelines.
In terms of functionality compared to
the original estimate, regardless of start or
finish time, 90% of the originally specified
requirements were completed.
A Detailed Look at Project-Level
Estimation Methods and Practices
Based on the responses in Study 1 and input
from our two experts, we constructed a list of
commonly accepted estimation best practices
and asked respondents in Study 2 to say which
of them were followed during their most recently
completed IT projects (see Table 9). The good
news is that several of these practices are used
on projects in the organizations we sampled. Our
survey showed that Waterfall and Agile projects
employ similar estimation practices, with the
exception of preparing formal estimates, which,

20 MIS Quarterly Executive | March 2014 (13:1) misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines

Table 9: Estimation Best Practices In Use

Best Practice Total WF Agile
A formal estimate was prepared 86% 93% 73%
Progress against the estimate was formally monitored throughout the
project 80% 82% 80%
The same people who eventually executed the project plan (e.g.,
developers) also participated in the preparation of the initial estimate 67% 69% 63%
The estimate was revised to accompany changes in user requirements
during the project 61% 64% 57%
Project scope was reduced to meet a target completion date 38% 38% 40%

Table 10: Method for Estimating Project Cost

Most- to Least-used Method Total WF Agile
Comparison to similar, past projects based on documented facts 59% 67% 43%
Comparison to similar, past projects based on personal memory 58% 58% 57%
Estimates created by individuals and reviewed in a group setting 56% 58% 57%
Expert judgment without use of documented facts 54% 51% 67%
A simple arithmetic formula (such as summing task durations) 52% 47% 57%
Estimates created in a group setting 49% 38% 70%
Estimates created by individuals and reviewed by individuals 45% 51% 37%
Guessing 39% 31% 47%
Established organizational standards (such as averages, standard
deviations, etc.) 34% 51% 20%
An estimation tool (e.g., software package) 21% 27% 13%
A complex statistical formula (such as multiple regression) 9% 4% 20%

not surprisingly, is more prevalent in Waterfall
projects.
Table 10 lists the 11 most commonly used
methods for estimating the costs of respondents
most recent IT projects.
The rankings in Table 10 compare favorably
with the organizational-level data from Study
1 (see Table 4). (Note, however, that due to the
specific project focus of Study 2 the question was
asked differently.) The top three cost-estimation
methods were identical in both studies. Other
commonand surprisingfindings are the
apparently wide use of guessing as a common
basis of cost estimation (particularly in Agile
projects) and the relatively low use of estimation
tools to support the process (although the use
of tools has increased slightly from the 17%
reported by Lederer and Prasad in 1992).
Cost-estimation tools most commonly cited by
respondents were Excel, internally developed
proprietary tools and Construx Estimate (a
freeware tool), although none of these was widely
used.
Table 10 indicates some differences in the
cost-estimation methods used for Waterfall
and Agile projects. Whereas Waterfall projects
tend to compare with past projects, use
individually prepared and reviewed estimates,
and use established organizational standards,
Agile projects are more likely to rely on expert
judgment, formulas and group-based estimates.
To provide additional detail about the
estimation process and bases for estimation,
respondents also identified the most common

March 2014 (13:1) | MIS Quarterly Executive 21

IT Project Estimation

Table 11: Methods Used to Estimate Size and Complexity of IT Projects

Most- to Least-used Method Total WF Agile
Work breakdown structure 71% 73% 53%
Work drivers (number of interface changes, number of new modules,
46% 58% 33%
etc.)
Story points (relative estimate of the size and complexity of work in an
32% 16% 67%
Agile project)
Function points (formal count of number of user inputs, outputs,
inquiries, files and external interfaces based on IFPUG standards or 27% 36% 20%
other industry standard approach)
Lines of code 8% 7% 7%

Table 12: Responsibility for Estimate Creation

Person(s) Responsible (Multiple Responses Allowed) Total WF Agile
Technical lead 74% 73% 70%
Person(s) doing the work (not technical lead) 72% 64% 83%
Manager 53% 58% 43%
Independent estimator 14% 16% 13%
Someone else outside of the project team 20% 22% 17%

methods used to estimate project size and
complexity. Table 11 lists the five most common
methods used across the large sample of Study 2
projects.
Work breakdown structure is by far the
most commonly used method for estimating
project size and complexity, which suggests
a formality and structure that has developed
over time in project management. However, the
most surprising finding is the low level of use of
lines of code for estimating size and complexity,
despite prior studies and popular literature on
software estimation reporting a heavy reliance
on this method.15 There are two reasons why
this method is now less popular than in the past.
First, is the type of project typical in todays
IT environments. Increasingly, contemporary
projects include a large number of integration or
COTS (commercial off-the-shelf) projects, which
often make the use of lines of code challenging or
impossible. The other contextual change includes
the growth of Agile development methodologies,
which often promote the use of relative size
15 See for example McConnell, S., op. cit., 2009 p. 198.
estimates (e.g., story points in Table 11). Our
survey results suggest it may be necessary to
reconsider and refine techniques promoted in
the popular IT development literature concerned
with the use (and even relevance of) lines of code
to estimate many of todays IT projects.
Estimate Creation and Presentation
In addition to the methods and practices used
to create IT project estimates, we also wanted
to examine who was responsible for creating an
estimate, when and how it was created and the
form in which it was presented. Table 12 lists the
person(s) most responsible for estimate creation.
These findings were as expected and similar
for both Waterfall and Agile projects, with the
exception that Agile projects were more likely
to have the people doing the work (but not the
technical lead) responsible for creating the
estimates. Estimation experts recommend this as
a best practice.
The only surprising result in Table 12 is
the relatively large percentage of respondents
reporting that someone outside of the project
team (i.e., not directly involved with the

22 MIS Quarterly Executive | March 2014 (13:1) misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines

Table 13: Timing for Estimate Creation

When Estimate is Created (Multiple Responses Allowed) Total WF Agile
Before any work on the project was done 72% 67% 73%
Requirements complete 46% 42% 30%
Through product-concept-completion 32% 29% 40%
At a defined gate or milestone in a stage-gate process 29% 33% 23%

Table 14: Estimate Presentation

Most Common to Least Used (Multiple Responses Allowed) Total WF Agile
Cost or effort range 54% 56% 50%
Schedule range 49% 40% 53%
Feature list (e.g., definitely, probably or maybe) 49% 44% 60%
Single-point number 45% 47% 33%
Confidence factors (probability of delivering on-time and/or on-budget) 33% 36% 27%
Cases (e.g., best, most likely, worst) 29% 33% 17%
Plus-or-minus qualifiers (e.g., 6 months, +3 months, - 2 months) 26% 24% 23%

project), but not an independent estimator,
was responsible for creating the estimate. The
open-ended comments provided with these
responses suggested that people with little
formal (or informal) estimation expertise,
or even familiarity with the technology, are
frequently driving project estimates. Amazingly,
one respondent reported that the sales manager
was responsible for creating the estimate for her
most recent project.
To get a sense for the timing of estimate
creation, we asked respondents when the project
estimate was created. As expected, by far the
most common response was that estimates were
created before any work on the project was done
(see Table 13). Note that a relatively high number
of Agile projects (40%) were still creating
estimates through product-concept-completion.
Again, this is often viewed as a recommended
practice.
Additional qualitative data gathered from
respondents suggested that baseline estimates
are commonly prepared at the proposal stage
and updated as major milestones are achieved.
In fact, one respondent indicated that estimates
were refined at eight different points during his
last IT project. Revisiting estimates throughout a
project is often a recommended best practice, so
greater use of this practice could well improve
estimation accuracy.
Finally, respondents reported a variety of
techniques for presenting estimates to internal
management or external clients. Rather than
using a monolithic, single-point estimate, our
survey indicates wide variation in how estimates
are presented, including some differences
between Waterfall and Agile projects (see Table
14).
It is surprising (and somewhat encouraging)
that the use of a single-point estimate falls in the
middle of Table 14, with other more contextually
driven factors (ranges, feature lists or confidence
factors), exceeding or rivaling the popularity of
less flexible, single-point estimates.
To summarize, most of the 115 projects
reported on in Study 2 were either late and/or
over budget, with Agile projects faring somewhat
better in both areas. A possible explanation for
the better performance of Agile projects (in
terms of estimation) is that they tend to rely
more on expert judgment, formulas and group-
based estimates. In addition, estimates for the
Agile projects in the survey tended to be:

March 2014 (13:1) | MIS Quarterly Executive 23

IT Project Estimation
Prepared by people who would also be
doing the work
Baselined at the beginning of the project
and updated throughout the project
Presented in (cost/effort/schedule)
ranges rather than as single-point
numbers.
Common Estimation Problems
In light of the cost, schedule and functionality
estimation challenges reported above, we also
asked respondents which estimation-related
problems they encountered during their last
IT projects. They cited the following (listed in
descending order of influence as reported in
prior studies):16
1. Insufficient analysis when developing the
estimate
2. Lack of adequate methodology/guidelines
for estimation
3. Lack of historical data on past estimates
and actual results
4. Pressure from managers or other
stakeholders to increase or reduce
estimates, and over-reliance on a single
persons estimate
5. A lack of visibility or control over actual
data compared to estimates.
Although less directly linked to estimation per
se, other problems that are either antecedents
or consequences of poor estimation practice
identified by managers included:
Frequent user requests for changes
Users lack of understanding of their own
requirements
Overlooked tasks
Changing personnel
Poor or imprecise problem definition.
Qualitative data provided by project
managers also suggested that drastic budget
cuts by government agencies was a factor in
causing projects to miss the original estimates.
A representative comment was The program
was tracking in accordance with the estimate
16 The ranked order of 26 proposed causes of inaccurate estimates
from Study 1 and Lederer and Prasad (1992).
24 MIS Quarterly Executive | March 2014 (13:1)
both in terms of time and budget. The Federal
agency simply slashed the budget. Other
inputs from respondents implied difficulties in
managing offshore development teams (what
one person termed misinterpretation and rogue
creativity of team inputs), and undue pressure
from stakeholders, including variations of the
following comment: Every project we estimate
gets significant pushback from [stakeholders] to
cut the estimate.
Evaluation of Project Success
Finally, we asked Study 2 respondents to
evaluate seven project success criteria for their
most recent IT projects.17 As reported above,
most (61%) projects were late, and 40% were
over budget. While these numbers tell a similar
story of project failure to that reported by
the Standish Group, other criteria tell a quite
different story. As demonstrated in Figure 1
and Table 15, the vast majority of projects
in our survey met requirements, produced
products/services that were used by their
target constituencies, resulted in organizational
learning and provided value. Moreover, overall
stakeholder satisfaction was rated at 91%.
These findings suggest that roughly 9 out
of 10 projects can be classified as successful
failuresthat is, projects that fail on one or
more process criteria (schedule and/or budget
in particular), but succeed on all of the important
outcome criteria. Representative comments from
the respondents included the following:
We delivered 10% more scope with 10%
less budget three weeks late.
The project came in slightly over schedule
but exceeded expectations.
The board of directors had the project
audited this month by external auditors
and it is being showcased in our industry
groups, newspapers and to our shareholders
to demonstrate our capabilities.
It is also interesting to compare our survey
results with the benchmark results of 180
project retrospectives conducted in 130 different
organizations between 1999 and 2013. Once
17 For a more detailed description of project success, see Nelson,
R. R., op. cit., 2005, pp. 361-372.
misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines

Figure 1: Project Success Criteria

OVERALL
STAKEHOLDER
SATISFACTION
(91%)

TIME LEARNING
(39%) (87%)

COST OUTCOME VALUE

(60%) PROCESS (89%)

PRODUCT USE
(90%) (94%)

Table 15: Project Success Criteria

18
Yes Benchmark
The project came in on schedule 39% 40%
The project came in on budget 60% 45%
The project produced a product of acceptable quality and met other
product-related specifications, including requirements, usability, ease
of use, modifiability and maintainability 90% 68%
The projects resulting product/service is being used by its target
constituencies 94% 88%
The project increased stakeholder knowledge and helped prepare the
organization for future challenges 87% 82%
The project will directly result in improved efficiency and/or
effectiveness for the client organization(s) 89% 70%
Overall, stakeholders were satisfied with the project 91% NA

again, these collective findings suggest that IT
projects tend to come in over schedule and/or
budget, yet eventually meet requirements and
add value to the organization. So, the bad news
is that most IT projects continue to fail in terms
of meeting estimates, but the good news is that
stakeholders seem to be satisfied as long as the
project results in a product/service that adds
value within their organization. 18
As previously noted, the most common
determinant of project failure is poor estimation.
Data suggests that this classic mistake is the
most common of all, occurring in 64% of the 180
benchmark projects. In all of the studies reported
in this article, there are many possible causes
18 Updated data from Nelson, R. R., op. cit., 2005, on 180 project
retrospectives conducted in 130 different organizations between
1999 and 2013.
of poor estimation, including poor problem/
requirements definition, scope creep, lack of
requisite knowledge and skills, inadequate
estimation methodologies, lack of historical data,
lack of visibility or control, and pressure from
stakeholders. We now provide guidelines for
addressing each of these problem areas.
Guidelines for Improving IT
Project Estimation
Based on the findings of our research, we
provide 10 guidelines for project managers on
how to improve their estimation practices.
1. Admit You Have a Problem
One of the striking revelations from our two
studies is that some of the key findings are very

March 2014 (13:1) | MIS Quarterly Executive 25

IT Project Estimation
similar to those of Lederer and Prasad over 20
years ago. While there have been tremendous
advancements in the project management
discipline, technology, IT development processes
and the education level of IT professionals,
it is clear that developing accurate estimates
continues to be a huge problem. As one
respondent stated: Our estimation process has
been spotty at best. weve certainly not found
the holy grail of estimating (yet).
2. Revisit the Prescriptions Offered by
Lederer and Prasad
Although Lederer and Prasads work
was done over 20 years ago, many of their
prescriptions can be adapted for todays IT
development environment, even though that
environment has changed (e.g., the increased use
of Agile methods). Based on our studies, several
of their specific recommendations are still valid:
Assign the initial estimating task to the
final developers
Delay finalizing the estimate until the end
of a thorough study of requirements
Monitor project progress closely
including actuals versus estimates
Rely on documented facts, standards and
simple arithmetic formulas rather than
guessing, intuition, personal memory
or complex formulas to generate the
estimate.
3. Conduct Project Retrospectives.
Project teams should be encouraged to
perform status reviews throughout the life of a
project. Such reviews can happen, for example,
at the end of each day (daily standup), two-
week intervals, completion of a major milestone
and project completion. In addition to learning
lessons and making necessary adjustments,
actual data can be captured for use in estimating
the next sprint (the basic unit of development in
Scrum), phase or project.
4. Employ Estimation Tools
As shown in Tables 4 and 10 above, the use
of estimation tools (e.g., software packages)
was relatively low in both of our studies (only
21% of Study 2 respondents reported using an
estimation tool). Estimation tools can help to:
26 MIS Quarterly Executive | March 2014 (13:1)
Evaluate and sanity-check project plan
alternatives against industry data or an
organizations historical data (if captured
via project retrospectives)
Negotiate a reasonable schedule and
budget, using a tools reporting capability
Improve visibility and control over actual
data compared to estimates
Coordinate estimated and actual project
data within and across project teams.
In addition to Microsoft Project and Excel
(including templates and add-ons), dedicated
estimation tools include freeware19 and other
packages.20
5. Understand Agile Approaches
and Adapt Estimation Techniques
Accordingly
The findings from our studies underscore
the importance of Agile development methods
in modern software development. Therefore,
it is critically important that project managers
understand these methods and how traditional
estimation techniques can be adapted to fit the
context of Agile development.21 Key principles
behind the Agile Manifesto22 include:
Our highest priority is to satisfy the
customer through early and continuous
delivery of valuable software
Deliver working software frequently, from
a couple of weeks to a couple of months,
with a preference to the shorter timescale
Working software is the primary measure
of progress.
Clearly, frequent estimation and delivery
cycles are central to the Agile approachand an
effective way to improve estimation (one sprint
or deliverable at a time).
19 Freeware estimation tools include Construx Estimate (http://
www.construx.com/estimate) and COCOMO (http://sunset.usc.edu/
csse/research/COCOMOII).
20 Other estimation tools include QSM SLIM-Estimate (http://
www.qsm.com/tools/slim-estimate) and SEER (http://www.galorath.
com).
21 For a good primer that explains the philosophy, tools and
techniques associated with estimation for Agile development, see
Cohn, M. Agile Estimating and Planning, Prentice Hall, 2005. Agile
estimation methods include relative (a.k.a. story point) estimation
and planning poker.
22http://agilemanifesto.org/principles.html.
misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines
6. Avoid Cognitive Biases Toward
Estimation
People are subject to an almost limitless set
of biases when making subjective judgments or
decisions, and the process of estimation is no
exception. Perhaps the best way to avoid letting
biases creep into estimation judgments is to be
aware of them. Some specific biases that are
particularly dangerous for project managers in
estimating projects include the following.
Underestimation Bias. People have a strong
tendency to underestimate, particularly when
the focal object of estimation is unknown or
complex. This obviously includes software and/
or outsourced projects (and is worse when
requirements, technologies, etc. are new or
unknown). Indeed, in over 15 years of teaching
estimation to mid-career IT professionals
(average work experience more than 12 years),
we have found that perhaps the most common
tendency from in-class workshops is for them
to underestimate when asked to give a range
estimate for a series of questions.
The Planning Fallacy. Related to the
underestimation bias, the planning fallacy23 is
the tendency for people and organizations to
underestimate the time, costs and risks of future
actions and at the same time overestimate the
benefits of those actions. Thus, the planning
fallacy results not only in time overruns, but
also in cost overruns and benefit shortfalls.
Kahneman and Tverskys original explanation for
the fallacy was that planners focus on the most
optimistic scenario for the task, rather than using
their full experience of how much time similar
tasks require. It is interesting to note that this
bias only affects predictions about ones own
tasks; when uninvolved observers predict task
completion times, they show a pessimistic bias,
overestimating the time that will be taken.24 This
suggests the need to involve external auditors
and/or use peer- or team-reviewed estimates,
23 The term planning fallacy was first coined in 1979 in
Kahneman, D. and Tversky, A. Intuitive prediction: Biases
and corrective procedures, TIMS Studies in Management
Science, (12), 1979, pp. 313-327. Since then the effect has been
found for predictions of a wide variety of tasks, including tax
form completion, school work, furniture assembly, computer
programming and even origami.
24 Buehler, R., Griffin, D. and Ross, M. Inside the Planning
Fallacy: The causes and Consequences of Optimistic Time
Predictions, Gilovich, T., Griffin, D. and Kahneman, D. (Eds.)
Heuristics and Biases: The Psychology of Intuitive Judgment,
Cambridge University Press, 2002, pp. 250-270.
as our research shows that many firms are now
doing.
Wishful Thinking. As implied by the results
of our studies, project managers think tasks will
be finished quickly and easily because that is
what they want to happen. Unfortunately, there is
a bigger penalty for underestimating (non-linear
impact due to planning errors, shortchanged
requirements, high-risk practices, etc.) than for
overestimating (linear impact due to Parkinsons
Law, where work tends to expand to fill the
estimate).
Self-Serving Bias. By taking credit for tasks
that went well but blaming delays on outside
influences, project managers or those centrally
involved with creating project estimates often
discount past evidence of how long a task should
take. Experiments have demonstrated that when
people made their predictions anonymously,
they do not show an optimistic bias.25 This
suggests that people sometimes make optimistic
estimates to create a favorable impression with
others.
Focalism Bias. The focalism bias refers to
the mental discounting of factors believed to
be outside the specifics of the project.26 These
factors include overhead tasks (e.g., meetings,
sickness and vacations) and black swan events
(low-probability, high-impact risks).
7. Understand the Estimate-
Convergence Graph
The estimate-convergence graph (a.k.a.
the cone of uncertainty) derives from
research that found that project estimates fall
within predictable ranges at various stages
of a project.27 One of our survey respondents
reported on his companys practice of requiring
project managers to estimate in ranges. At
the beginning of the feasibility phase, project
managers are asked to come up with a 100%
cushion, at the beginning of the definition phase,
a 75% cushion, at the beginning of the design
phase, a 50% cushion and at the beginning of
the construction phase, a 25% cushion. Project
25 Pezzo, M., Litman, J. and Pezzo, S. On the distinction between
yuppies and hippies: Individual differences in prediction biases
for planning future tasks, Personality and Individual Differences
(41:7), 2006, pp. 1359-1371.
26 Wilson, T., Wheatley, T., Meyers, J., Gilbert, D. and Axsom,
D. Focalism: A source of durability bias in affective forecasting,
Journal of Personality and Social Psychology (78:5), 2000,
pp. 821-36.
27 Boehm, B., op. cit., 1981.
March 2014 (13:1) | MIS Quarterly Executive 27
IT Project Estimation
managers update their estimates at the end of
each phase, resulting in improved estimation
accuracy as a project progresses through its life
cycle.
8. Understand the Difference Between
a Target and an Estimate
Estimates are arrived at based on careful
analysis, whereas targets represent a desired
schedule or cost and can be set without any
analysis. Project managers should be encouraged
to carefully set targets within a reasonable
range of an estimate and delay making a firm
commitment for as long as possible.
Estimations role is to determine whether
you have a realistic chance of meeting your
targets. If the target is within roughly 20%
of the estimated outcome, you can control
the project to meet the targets (make
minor adjustments to features, schedule and
resources). If the gap between the target
and the estimate is greater than about
20%, it is not possible to control the project
to meet the targets.28
9. Manage Stakeholders
Given the role that stakeholders play in
defining project requirements, setting targets,
changing requirements (i.e., scope creep)
and exerting pressure on the project team
throughout a project, project teams need to find
more effective ways to involve stakeholders.
For example, another key principle of Agile
development is that business people and
developers must work together daily throughout
the project. This practice enhances transparency
and may help control stakeholder pressure.
10. Develop Estimation Competencies
of Project Managers and Team
Members
As emphasized by the first guideline, the
IT development community is simply not very
good at estimation. We urge project managers
to look for opportunities to develop and hone
their estimation competencies and those of
project team members. Such training might
involve everything from a multi-hour workshop,
to a multi-day short course, to broader courses
as part of an advanced degree. All of these
28 McConnell, S., op. cit., 2006.
28 MIS Quarterly Executive | March 2014 (13:1)
approaches are appropriate and can make an
immediate difference. Once trained, project
managers and their teams can help infuse
the training and ideas into internal training
programs or professional development
initiatives. Ample opportunities for such training
exist and project managers who continue to
ignore them do so at their own peril.
Concluding Comments
Insanity can be defined as doing the same
thing over and over again and expecting a
different result. In effect, this is what the IT
development community has been doing with
respect to estimation for more than 20 years. The
findings from Study 1 are remarkably consistent
with similar findings first reported in the early
1990s, with classic estimation mistakes still
being madeover-reliance on personal memory,
use of intuition and guessing. Moreover, the
common causes of poor estimation practices
today have changed little over the same time
spanchange requests from users, users lack
of understanding of requirements, poor problem
definition and insufficient analysis prior to an
estimate being made.
The results from our Study 2, which looked at
project-level data, are slightly more encouraging.
Our findings indicate that contemporary
practices, including Agile development methods,
seem to be making some headway toward
improving project success. Interestingly, most
projects met their stated requirements, produced
usable products and services, and improved
organizational value, leading to a high overall
stakeholder satisfaction rating. However,
somewhat paradoxically, most of the projects
were late and many (40%) were over budget,
reiterating the need for better time and cost
estimation. Thus, todays typical project can be
characterized as a successful failure.
Many of the deficiencies in current projects
can be traced back to poor estimation, so there
is clearly room for improvement in estimation
practices. We have provided specific guidelines
for IT project managers to help them reverse the
trend so that the estimation problems identified
by our studies do not continue for another 20
years.
misqe.org | 2014 University of Minnesota
 IT Project Estimation: Contemporary Practices and Management Guidelines

Appendix: Survey Research the Lederer and Prasad 1992 study. However,
we felt that a second study at a lower level of
Methodologies
analysisi.e., at the project level rather than
the organizational levelwould provide project
Research Study 1: Organization-Level managers with additional detail, richness and
insights. Furthermore, by collecting data at the
Analysis of Project Estimation
project level, we were more able to directly tie
The survey questionnaire included 35 multi-
project estimation practices to project outcomes
part questions requiring responses on five-point
(schedule, cost, functionality), which Study 1
Likert-type scales. Other questions required
suggested continue to be a challenge for project
respondents to report the proportion of projects
managers.
on which they carried out particular estimating
In reviewing the results of Study 1 and talking
practices. After piloting the survey with 40 IT
with project managers, we realized that it would
professionals and revising it to improve clarity
be much easier for them to talk about outcomes
and completeness, the questionnaire was
on a specific project (rather than projects in
distributed in 2010 via an Internet-based survey
general). To add additional validity and reliability
tool to several hundred IT professionals affiliated
to our insights from Study 1, we therefore
with a major U.S. graduate school program on the
decided we needed to collect information on a
Management of IT (alumni and current students;
wide variety and scale of individual projects.
all working IT professionals).
Armed with this project-level data, we felt we
The 60 respondents had a similar profile to
would be able to generalize our results for the
those in Lederer and Prasads 1992 survey: they
broader population of contemporary IT projects.
were responsible, educated and experienced IT
In Study 2, we surveyed 115 IT professionals
professionals familiar with their current firms.
during 2011 and asked each respondent to
While all respondents had a direct connection to
provide data about his or her most recently
project estimation, most (72%) were responsible
completed IT project. Virtually all of the
for project/department management, with the
respondents indicated they were project/
remainder involved with estimation, systems
program managers or directors, giving us
analysis/design, programming and/or database
confidence that they were in a position of
administration. All respondents had a four-year
responsibility over project timelines, costs and
college degree and at least some graduate-level
outcomes. Not surprisingly given the turnover
education. On average, respondents supervised
in IT-related fields, most (86%) had worked for
31 employees and had 15 years of experience in
their current employer for less than 10 years
IT with six years at their current companies.
and just over half (51%) had worked in the IT
The firms varied in size and industry. Their
field for more than 10 years. A wide variety of
annual sales ranged from $900,000 to $25 billion
industries were represented in the sample, with
with a mean of $4.5 billion. The average number
consulting/services having by far the highest
of employees was 9,423 with a range of six to
number of respondents (see the table).
77,000. On average, there were 1,539 employees
Additionally, the organizations in the Study 2
in their IT departments, ranging from one to
sample tended to be quite large. Annual revenue
18,000. IT department annual budgets ranged
averaged $9 billion, and the average number of
from $900,000 to $15 billion, with a mean of $1.1
employees was 27,000.
billion. The primary industries represented were
As in Study 1, the projects reported on in
consulting/services (31%), banking/finance
Study 2 covered a wide spectrum of size, scope
(24%), government (13%), education (7%) and
and functionality. This variance was expected
healthcare (7%).
because we asked each respondent to provide
Study 2: Project-Level Analysis of data on the last project that they worked on (not
Project Estimation any project or a typical project). Projects ranged
Study 1 provided a useful snapshot of overall from a small web application development
organizational estimation practices and enabled project costing several thousand dollars to a
us to directly benchmark the results against $350 million IT infrastructure project for a large
international airport. In between, there were

March 2014 (13:1) | MIS Quarterly Executive 29

IT Project Estimation

database and data center projects, business Decision Processes, Personnel Psychology, IEEE
intelligence/analytics, ERP, CRM, systems Transactions on Engineering and Management
and Decision Sciences, among others.
Survey 2 Primary Business Activity
Industry Percentage
Consulting/services 50.67%
Government 10.67%
Financial services 6.67%
Healthcare 5.33%
Communication 6.67%
Education 4.00%
Retail 5.33%
Utilities 4.00%
Insurance 2.67%
Legal 2.67%
Manufacturing 1.33%

integration and many web-development projects.

Project budgets averaged $11 million, with a
median budget of $440,000.

About the Authors

R. Ryan Nelson
R. Ryan Nelson (rnelson@virginia.edu) is
Professor, IT Area Coordinator and Director of
the Center for the Management of Information
Technology (CMIT) at the McIntire School of
Commerce of the University of Virginia. He
received his Ph.D. in business administration
from the University of Georgia in 1985 and spent
five years on the faculty of the University of
Houston before joining the University of Virginia.
He has published several articles on the topic of
project management in MIS Quarterly Executive
and currently serves on that publications
editorial board.

Michael G. Morris
Michael Morris received his Ph.D. in Management
Information Systems from Indiana University
in 1996. He has previously served as a senior
editor at MIS Quarterly and Information Systems
Research. His research has been published in MIS
Quarterly, Organizational Behavior and Human

30 MIS Quarterly Executive | March 2014 (13:1) misqe.org | 2014 University of Minnesota
Copyright of MIS Quarterly Executive is the property of MIS Quarterly Executive and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.