Forrester Research, Inc.

, 60 Acorn Park Drive, Cambridge, MA 02140 USA

Tel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.com
The Forrester Wave: Application Life-
Cycle Management, Q4 2012
by Tom Grant, Ph.D., October 23, 2012
FOR: Application
Development
& Delivery
Professionals
KEY TAKEAWAYS
Leaders Build Product Strategy Around ALM Use Cases
Forrester Wave evaluation Leaders IBM, Rally Sofware, PTC, CollabNet, Microsof,
and Serena Sofware ofer strong solutions for ALM problems that no single
point solution could address. Teir product strategies focus on such challenges as
delivery and the disruptions of Agile adoption, not just throwing new capabilities at
the customer.
Strong Performers And Contenders Find A Specic Audience
Atlassian and Rocket Aldon dont focus as much on specifc use cases that span
many ALM activities. Instead, they build tools to address smaller but widespread
problems, such as release management and developer collaboration. Meanwhile,
HP is expanding its ALM ofering beyond Quality Center.
ALM Is Going Through A Period Of Redenition And Innovation
Hot ALM issues such as DevOps, embedded sofware, and requirements defnition
show how the concept of ALM has expanded and changed. During this transition,
the seemingly crowded ALM market will beneft from the innovation that comes
from a wealth of competing and complementary product oferings.
2012, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available
resources. Opinions refect judgment at the time and are subject to change. Forrester

, Technographics

, Forrester Wave, RoleView, TechRadar,

and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To
purchase reprints of this document, please email clientsupport@forrester.com. For additional information, go to www.forrester.com.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
WHY READ THIS REPORT
In Forresters 116-criteria evaluation of application life-cycle management (ALM) vendors, we identifed
the nine most signifcant sofware providers in the category Atlassian, CollabNet, HP, IBM, Microsof,
PTC, Rally Sofware, Rocket Aldon, and Serena Sofware and researched, analyzed, and scored them.
Tis report details our fndings about how well each vendor fulflls our criteria and where they stand in
relation to each other to help application development and delivery professionals select the right partner
for their sofware-fueled business innovation eforts.
Table Of Contents
Innovation Drives Interest In Application Life-
Cycle Management
Application Life-Cycle Management
Evaluation Overview
The ALM Forrester Wave Finds Many
Leaders And Different Directions
Vendor Proles
Supplemental Material
Notes & Resources
Forrester conducted lab-based evaluations
in September and October 2011 and
interviewed nine vendor and user companies:
Atlassian, CollabNet, HP, IBM, Microsoft,
PTC, Rally Software, Rocket Aldon, and
Serena Software.
Related Research Documents
The Time Is Right For ALM 2.0+
October 19, 2010
The Forrester Wave: Agile Development
Management Tools, Q2 2010
May 5, 2010
The Forrester Wave: Application Life-Cycle
Management, Q4 2012
IBM, Rally Software, PTC, CollabNet, Microsoft, And Serena Software
Lead The Pack, With HP, Atlassian, And Rocket Aldon Following
by Tom Grant, Ph.D.
with Kyle McNabb and Alissa Anderson
2
6
10
14
17
OCTOBER 23, 2012
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 2
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
INNOVATION DRIVES INTEREST IN APPLICATION LIFE-CYCLE MANAGEMENT
Business innovation now drives the application life-cycle management (ALM) market. Te
contribution that application development and delivery makes to a companys business goals making
workers more productive, creating engaging customer experiences, and bringing new sofware
products to market more successfully must be more direct and successful. Sofware increasingly
plays a central role in a frms ability to deliver new products and services or exploit new channels.
Firms can no longer accept historical gulfs between business and application development and delivery
teams as, increasingly, frms now expect to manage application development and delivery as a business
and treat it as a competency.
1
Unfortunately, everyone has their long list of reasons why they struggle with better managing
application development and delivery. Te work is creative, and teams must fgure out how to work
with one another. Tools or techniques that impose an assembly-line model will fail. Application
development and delivery still need management, but the tools of management must ft the nature
of the creative and collaborative work.
Sofware and application development work itself is changing. Tis Forrester Wave evaluation
examines the most signifcant developments that afect both ALM strategy and ALM tools. For
example, the defnition of ALM has stretched to include not just development but also delivery (see
Figure 1). Some of the vendors we assess have expanded their tools portfolio to add or enhance their
support for release management and other delivery-related activities.
Te need to support the business management of application development and delivery underlies
this Forrester Waves evaluation criteria:
Delivering business value, not just driving process efciency. ALMs original aim was
process improvement. Firms turned to ALM to do a better job of managing project timelines,
dependencies, and deliverables; ensure that every requirement could be traced to a real
sofware capability; and test to ensure that the sofware worked. While process improvement is
still important, increasing efciency in building and delivering sofware is as important a goal
for ALM as proving the value of the sofware, as it contributes to business outcomes.
Supporting individual, team, and institutional management. ALMs frst wave of tools
focused on in truth application development management (ADM), designed to increase
developer and tester productivity. Now, frms seek ALM tools to improve team success while
linking these teams to the larger application development and delivery activity within the
organization or, in many cases (such as ofshore partners), outside of it.
Extending life cycles to include delivery. Organizations now recognize that the life cycle
of ALM does not stop with check-in or build phases. Te real life cycle and process goes on
much longer, arguably all the way to the moment the application goes live and even beyond to
support continuous feedback.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 3
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Managing sofware that gets deployed everywhere. Sofware is truly everywhere, from data
center servers to laptops, from mobile devices to cars and refrigerators. Regardless of where
the sofware winds up, sofware development and delivery management is still necessary.
Integrating across tools. Vendors have reluctantly admitted that customers are not interested
in their suite of tools, no matter how impressive or cost efective they might be. Switching
costs can be high, and many capabilities, such as issue tracking and source control, are so
commoditized that switching makes little sense. Some tools, such as HP Quality Center, are so
widely adopted that other vendors must integrate with them in some fashion. Te speed of tool
innovation in some areas, such as continuous delivery and requirements, outpaces what any
single vendor can keep up with, providing another reason for integrating across tools.
Using reporting and dashboards as instruments of change. Many application development
and delivery organizations aspire to be learning organizations and see their investment
in ALM tools in that light. Te data in an ALM system provides a yardstick for a teams
continuous improvement eforts. Some teams use this information to communicate
with outside parties about the shape and value of new Agile and Lean practices. Larger
organizations may use the same information to track sofware value streams across projects
and teams. In these and other scenarios, reporting and dashboards provide the ultimate payof
for ALM investments.
Figure 1 ALM Boundaries Expand To Encompass Te Sofware Value Chain
Source: Forrester Research, Inc. 60080
Production planning
closed loop
Change management Service management
Portfolio management
Change-aware
continuous integration
Just-in-time
demand
management
Release
management
Testing and
quality assurance
Build and
software
confguration
management
Deployment
Project
management
Production control
closed loop
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 4
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
ALM Vendors Have Taken Many Steps Forward, But A Long Road Remains
ALM tools have made signifcant progress since out last evaluation:
ALM vendors have discovered macro diferentiators. In past years, ALM vendors did far less
in their product strategy and product marketing to clearly diferentiate their products. With
notable exceptions such as Rally Sofware, a company that built its ALM strategy around Agile
support, ALM vendors had noticeable diferences only if you carefully examined the details of
their product features. Today, ALM vendors more clearly identify the types of customers they
serve and the problems they can help these customers address. Serena Sofware, for example,
tells a very clear story about the challenges orchestrating development and delivery in large IT
organizations and how its products help organizations address those challenges.
Vendors help improve the efciency of the larger ALM process. As noted earlier, the
boundaries of ALM have expanded to include more of the delivery phase. In response,
ALM vendors have expanded their capabilities in this area to remove the inefciencies that
arbitrarily lengthen and complicate the delivery phase.
Requirements, a part of ALM, need innovation attention. Recent innovations in
requirements tools, such as visualization and social media integration, have increased the
depth and accuracy of insights and the speed of collecting them. Tese innovations have
occurred in requirements-focused companies such as Balsamiq Studios, Blueprint Sofware
Systems, iRise, and Visure Solutions. Even though ALM vendors provide requirements tools,
they have not kept pace with these innovations.
Mobile and embedded development scenarios get insufcient attention. Embedded
sofware developers still get scant attention, with the exception of MKS (PTC) and IBM.
Surprisingly, given how hot the mobile development market is, ALM vendors by and large do
nothing special for mobile developers, including in areas such as testing where there is a large
need for better tools.
Collaboration is the lightning that ALM vendors cant capture in a bottle. Many of the
biggest ALM challenges, such as coordinating the work across geographically distributed teams,
boil down to collaboration problems. Collaboration presents some extremely vexing puzzles for
tools vendors, particularly given how idiosyncratically a specifc set of people may collaborate
with each other. Still, some basic collaboration features desktop sharing for team members in
diferent locations and wikis for capturing and updating information, for example are far less
common than expected.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 5
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
The Market Is Still Open For Innovation And New Contenders
Te vendors we assessed in this evaluation provide an impressive array of tools with a broad range
of capabilities. Nevertheless, they do not represent every possible innovation in the expanding and
evolving realm of ALM. Te ALM market is still wide open for anyone to devise a better way to
address a problem, identify new and unmet challenges, or give a particular class of customers more
attention than the bigger, general-purpose vendors can.
Vendors listed in this section were not evaluated but represent various innovative approaches to
ALM, demonstrating how much room remains within this market for innovation and specialization:
Blueprint focuses on requirements. Some aspects of ALM pose tough challenges that dont
have an easy solution. Smaller vendors focused exclusively on some of these challenges can
innovate in ways that larger suite vendors may not. Blueprint, a requirements tools vendor,
provides a good example of the important role that these smaller companies with a much
narrower focus play in todays ALM market.
Hansof caters to video game developers. Since the video game market is nearly as large as the
movie industry, it should be no surprise that at least one vendor, Hansof, has identifed this
market as its core constituency.
Polarion tackles difcult compliance challenges. Few of the vendors we evaluated have taken
steps to address complex compliance requirements, leaving no small amount of opportunity for
a smaller vendor such as Polarion Sofware.
SmartBear Sofware makes quality the starting point. While many ALM vendors are shifing
their attention to later parts of the development and delivery process, SmartBear Sofware
makes those stages the starting point for its product strategy. Tis strategy includes both code
quality improvement through review and testing and quality of service maintained through web
monitoring.
Tasktop is the Switzerland of integration. Each ALM vendor has its own set of integrations
with other tools, both open source and commercial. Each ALM vendor also has its own
approach to integration. Unsnarling this tangle sometimes requires going to a neutral third
party, such as Tasktop, for cross-ALM integration.
ToughtWorks provides thought leadership in continuous delivery. A real thought leader
in continuous delivery, ToughtWorks provides a modest range of tools but a very established
consulting business (primarily aimed at helping with Agile adoption).
VersionOne supports Agile. Rally has built Agile and Lean support into its tools, and
other vendors such as Microsof and IBM provide Agile-specifc confgurations. With Agile
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 6
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
continuing to spread, there is still room for another Agile-focused ALM vendor such as
VersionOne to meet the needs of Agile teams in a diferent way than its competitors.
APPLICATION LIFE-CYCLE MANAGEMENT EVALUATION OVERVIEW
To assess the state of the application life-cycle management (ALM) market and see how the vendors
stack up against each other, Forrester evaluated the strengths and weaknesses of nine top vendors.
Tese companies fell into two broad categories:
Large sofware companies that, among other products, also ofer ALM tools. In this category,
HP, IBM, and Microsof maintain a strong position in the ALM market.
Smaller sofware companies that specialize in ALM. Other companies included in this
evaluation, such as Atlassian, CollabNet, Rally Sofware, Rocket Aldon, and Serena Sofware,
have made ALM their exclusive business. Until its recent acquisition by PTC, MKS also fell into
this group.
Evaluation Criteria Measured Breadth Of Portfolio And Support For Use Cases
Afer examining past research, user need assessments, and vendor and expert interviews, we
developed a comprehensive set of evaluation criteria. We evaluated vendors against 116 criteria,
which we grouped into three high-level buckets:
Current ofering. ALM covers a vast expanse of functionality for developers, business analysts,
testers, project managers, and people in a wide array of other roles. Terefore, we developed
more than 86 distinct criteria to measure the strength of each vendors tools.
Strategy. Te value that a vendor brings to the table goes beyond just its current tools, so we
included 18 criteria that assessed each vendors ability to support customer implementations, its
commitment to the ALM market, its ALM product strategy, and its corporate strategy.
Market presence. We also took into account the size of the footprint each vendor has in the
sofware market. Forresters clients want to know how safe a bet it will be that a given ALM
vendor will be in business several years from now in order to gauge how safely they can invest in
that vendors ALM tools.
Tis years ALM evaluation represents a departure from the 2010 Forrester Wave evaluating Agile
development management solutions in three ways:
2
Opening the aperture to encompass all ALM scenarios. Te 2010 evaluation focused
exclusively on Agile support among ALM vendors. Tis year, we include both Agile and non-
Agile use cases in the criteria.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 7
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Treating Agile and Lean as a critical test of ALM oferings. At the same time, this years
evaluation by no means leaves Agile behind. Instead, we treat Agile and Lean as critical tests of
an ALM vendors ofering.
Incorporating systems engineering and embedded sofware. Given the increasing quantity
of hybrid products those incorporating sofware into physical products ranging from cars
to medical devices we have also included criteria covering sofware engineering outside of
corporate IT and independent sofware vendors (ISVs).
Evaluated Vendors Have Strong Tools And Substantial Market Presence
Forrester included nine vendors in the assessment: Atlassian, CollabNet, HP, IBM, Microsof, PTC,
Rally Sofware, Rocket Aldon, and Serena Sofware. Each of these vendors has (see Figure 2):
Depth and breadth of ALM ofering. While there are many commercial and open source point
solutions for particular activities (test management, build management, issue tracking, etc.),
the title ALM vendor goes to companies that do more than provide a single capability (source
control, defect tracking, etc.), however good it might be. We expect these tools to cover the
majority of ALM use cases, as Forrester defnes ALM.
A broad ALM customer base. We also looked at the number of customers for a vendors ALM
products in 2011 (at least 750) as well as the growth in the vendors customer base from 2010 to
2011.
Defnitive revenue from ALM. Finally, the criteria included the revenue from ALM. Once
again, we included both the numbers for 2011 (at least $19 million) and the growth since 2010.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 8
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Figure 2 Evaluated Vendors: Product Information And Selection Criteria
Source: Forrester Research, Inc.
Vendor
Atlassian
CollabNet
HP
IBM
Microsoft
PTC
Product evaluated
Atlassian OnDemand:
JIRA
GreenHopper
Bonfre
Confuence
SharePoint Connector for Confuence
Team Calendars for Confuence
Bamboo
FishEye
Crucible
Clover
Crowd
Bitbucket
Atlassian IDE Connector for Eclipse
Atlassian IDE Connector for IntelliJ IDEA
Atlassian IDE Connector for Visual Studio
2008 and 2010
HipChat
Stash
Sourcetree
JIRA Mobile Connect
JIRA OnDemand
GreenHopper OnDemand
Bonfre OnDemand
Confuence OnDemand
Team Calendars for Confuence
OnDemand
Source OnDemand Bundle (FishEye,
Crucible, and Subversion)
TeamForge
ScrumWorks Pro
Subversion Edge
Lab Management
HP Application Lifecycle Management (HP
ALM) 11.5
Rational Collaborative Lifecycle
Management Solution (Rational CLM):
Rational Requirements Composer 4.0
Rational Team Concert 4.0
Rational Quality Manager 4.0
Microsoft Visual Studio
Integrity
Product version
evaluated
5.0
5.10
2.2
4.2
1.5.3
2.3
4.1
2.7
2.7
3.1.6
2.4.2
Weekly builds
3.5 or later
7 and later
N/A
Weekly builds
1.1
Weekly builds
N/A
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
6.2
6.0
3.0
2.5
11.5
4.0
2010
10.0
Version
release date
June 2012
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
Weekly builds
June 2012
May 2012
April 2012
May 2012
June 2012
June 2012
May 2010
December 2011
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 9
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Figure 2 Evaluated Vendors: Product Information And Selection Criteria (Cont.)
Source: Forrester Research, Inc.
Vendor
Rally Software
Rocket Aldon
Serena
Software
Product evaluated
Rally Community Edition
Rally Enterprise Edition
Rally Unlimited Edition
Rally Quality Manager
Rally Support Manager
Rally Product Manager
Rally Community Manager
Rally Time Tracker
Rally Idea Manager
Rally Portfolio Manager
Aldon Lifecycle Manager
Aldon Deployment Manager
Aldon Community Manager
Aldon Report Manager
Aldon Agile Manager 1.0
Development Manager
Release Manager
Requirements Manager
Service Manager
Demand Manager
Request Center
Dashboard
ChangeMan ZMF
Agile Planner
Product version
evaluated
Weekly builds
(e) 6.1/(i) 7.6
N/A
9.5.4
1.5
1.0
1.2
1.0
2.1
3.1
3.1
3.1
2.1
7.1
2.3
Version
release date
Weekly builds
May 2011
N/A
November 2010
June 2011
January 2011
May 2012
Vendor selection criteria
The vendor provides designated ALM product(s) and a stated ALM road map.
The vendor has more than 750 customers.
The vendor provides tools that cover the majority of ALM use cases, as Forrester defnes ALM.
The vendor generates more than $19 million in revenue.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 10
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
THE ALM FORRESTER WAVE FINDS MANY LEADERS AND DIFFERENT DIRECTIONS
Te vendors we evaluated for the ALM Forrester Wave provide a wide range of capabilities.
Superfcially, they look very much alike. Project management? Check. Issue tracking? Check. With
such feature breadth, you might conclude that ALM is a mature market.
But where are the vendors heading? Te answer is, in very diferent directions. Sofware may be
everywhere, but the form of sofware development and delivery varies widely across organizations.
While this market now has many genuine leaders, they have reached this point only by identifying
their core market. Terefore, the market is less mature than it might seem on the surface. When
evaluating ALM vendors, its important to consider not just the capabilities they provide but also the
use cases for which they are optimized.
Nowhere is this diferentiation more clear than in the overarching principles that guide the
evaluated vendors product strategies (see Figure 3):
Leaders IBM and Microsof provide broad ALM capabilities. Both IBM and Microsof have
maintained a strong portfolio of ALM tools designed to support common use cases for sofware
development and delivery. Te two companies both provide broad swaths of capabilities for
common ALM activities, such as project defnition and management, reporting, issue tracking,
and task assignment.
IBM and Microsof come from a more general-purpose perspective on ALM, and both
treat Agile practices as a special instance of ALM workfows, metadata, and reporting. Both
companies support Agile as a template, a special confguration of their ALM tools.
Leaders PTC and Rally Sofware lead by focusing on specifc use cases. In contrast to IBM
and Microsof, both Rally and PTC (originally MKS, before PTCs acquisition of the company in
2011) have succeeded in the ALM market by fnding a niche in which to specialize for Rally,
Agile, and for PTC, compliance. While this approach may limit their ability to support a broader
swath of scenarios, their focus gives them the opportunity to provide thought leadership in their
respective areas. Rallys focus on Agile and Lean, for example, has led the company to build
top-notch planning capabilities into its tools. PTCs focus on compliance, including the highly
regulated world of embedded sofware, has made it the only ALM vendor to provide out of the
box many of the workfows and reports needed to satisfy regulatory authorities.
Leaders CollabNet and Serena lead through delivering diferentiated and broad capabilities.
Both CollabNet and Serena have tailored their ALM oferings to more-general value
propositions and needs. CollabNet sees its opportunity in the cloud, where ALM tools support
has been lagging behind ITs need to manage cloud development, testing, and deployment more
efectively. Serena has recognized how important it is for organizations to efectively orchestrate
the larger meta-process of sofware development and delivery, which spans the typical
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 11
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
activities (coding, testing, building, etc.) that guide the development eforts of other commercial
and open source tools.
Strong Performers Atlassian and HP provide focused capabilities. Atlassian and HP show
how far an ALM vendor can go with strong support for a piece of the overall life cycle. However
capable individual tools such as JIRA and Quality Center may be, they are no substitute for
a broader ALM ofering. Te two companies view this gap diferently. Rather than defocus
its product strategy, Atlassian prefers to provide a smaller range of strong capabilities than
potentially do a mediocre job tackling a wider range of ALM challenges. Tis strategy makes it
easier for Atlassian to deal with one of its biggest challenges, perception as an enterprise-ready
vendor, without the added complexity of too many products and features.
In contrast, HP has pushed hard to fll in the gaps in its ALM ofering, moving far past Quality
Center. Te current version, ALM 11.5, provides a very robust set of tools that can compete with
other ALM suites on a wide range of capabilities. Some of these new capabilities, such as the
executive dashboard features, demonstrate that HP understands many tough ALM challenges
and will invest in providing credible, diferentiating features to address them. However, these
tools are relatively new, and Forrester has not encountered many implementations of them yet.
Te next year will be critical for HPs ALM ofering in gaining market traction.
Contender Rocket Aldon is the operational dark horse of ALM. While most other ALM
vendors strive to move down the value chain to the point of release and deployment, Rocket
Aldons product strategy starts with these later stages of the application life cycle. Automation
plays a prominent role in Rocket Aldons oferings, as do IT-governance-related features. Te
Rocket Aldon road map is, to some extent, extending upstream to cover scenarios (for example,
product owner review of the backlog) that it currently does not address as thoroughly as
production-related activities.
Tis evaluation of the ALM market is intended to provide a starting point for your own evaluation.
We encourage readers to view the detailed product evaluations and adapt the criteria weightings to
ft their individual needs through the Forrester Wave Excel-based vendor comparison tool.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 12
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Figure 3 Forrester Wave: Application Life Cycle Management Tools, Q4 2012
Source: Forrester Research, Inc.
Go online to download
the Forrester Wave tool
for more detailed product
evaluations, feature
comparisons, and
customizable rankings.
Risky
Bets Contenders Leaders
Strong
Performers
Strategy Weak Strong
Current
ofering
Weak
Strong
Market presence
Atlassian
CollabNet
HP
IBM
PTC
Rally
Rocket Aldon
Serena
Microsoft
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 13
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Figure 3 Forrester Wave: Application Life Cycle Management Tools, Q4 2012 (Cont.)
Source: Forrester Research, Inc.
CURRENT OFFERING
Platform support
Installing the product and
getting going
Administration
Security
Integration and customization
Management
Change elements
Life-cycle traceability
Reporting
Analytics
Running a project
General collaboration capabilities
Process confguration
STRATEGY
Support for implementations
Product strategy
Corporate strategy
Price
Commitment
MARKET PRESENCE
Installed base
Financial strength
Employees
Support services
Channel partnerships
Global presence
A
t
l
a
s
s
i
a
n
3.47
3.40
4.80
4.00
2.20
4.80
3.95
2.47
3.63
2.75
1.38
4.75
4.70
2.24
2.92
2.50
2.90
4.50
0.00
5.00
3.94
3.30
4.00
2.00
4.10
5.00
3.00
F
o
r
r
e
s
t
e
r

s
W
e
i
g
h
t
i
n
g
50%
5%
5%
2%
3%
10%
5%
10%
15%
5%
10%
10%
10%
10%
50%
15%
80%
5%
0%
0%
0%
35%
5%
0%
35%
20%
5%
C
o
l
l
a
b
N
e
t
3.51
4.10
4.40
4.75
2.60
4.90
3.89
3.74
3.26
3.70
2.28
4.30
2.90
2.36
3.95
4.90
3.70
5.00
0.00
5.00
4.42
4.20
3.00
1.00
5.00
4.00
5.00
H
P
3.53
4.30
4.20
5.00
3.80
4.00
3.96
5.00
3.85
3.85
3.36
2.90
2.80
1.18
3.22
4.50
2.90
4.50
0.00
5.00
4.95
5.00
4.00
4.00
5.00
5.00
5.00
I
B
M
4.38
3.80
4.00
5.00
4.60
5.00
3.79
4.48
4.38
4.60
5.00
4.30
4.80
3.20
4.76
5.00
4.70
5.00
0.00
5.00
4.51
4.50
5.00
3.00
4.10
5.00
5.00
M
i
c
r
o
s
o
f
t
3.91
3.90
3.60
5.00
4.20
4.75
3.96
4.74
3.47
4.30
3.20
4.30
4.20
2.56
3.25
5.00
2.90
3.50
0.00
5.00
4.32
3.20
4.00
3.00
5.00
5.00
5.00
P
T
C
4.01
1.50
3.60
4.00
5.00
4.75
4.42
4.21
3.64
4.65
5.00
3.70
4.25
3.31
3.72
5.00
3.40
5.00
0.00
5.00
3.23
2.20
3.50
3.00
4.10
3.00
5.00
R
a
l
l
y

S
o
f
t
w
a
r
e
3.95
3.20
4.60
5.00
3.40
5.00
3.97
4.75
4.08
4.55
4.28
4.15
3.80
1.26
4.39
4.10
4.40
5.00
0.00
5.00
4.26
3.10
3.50
1.00
5.00
5.00
5.00
R
o
c
k
e
t

A
l
d
o
n
1.97
0.10
2.82
1.50
5.00
1.05
2.06
1.44
0.66
1.80
2.44
3.10
1.05
4.45
2.04
2.10
1.90
4.00
0.00
5.00
2.22
2.20
1.50
1.00
3.20
1.00
1.00
S
e
r
e
n
a

S
o
f
t
w
a
r
e
3.62
3.10
2.80
3.75
2.20
5.00
4.14
3.38
3.38
4.15
4.26
3.25
3.15
3.61
3.44
3.10
3.40
5.00
0.00
5.00
3.73
2.90
2.50
2.00
4.10
5.00
3.00
All scores are based on a scale of 0 (weak) to 5 (strong).
ALM Vendors Have Yet To Fully Address Many Common Challenges
While many aspects of the ALM tools market have improved substantially in the past two years
since the Agile development management Forrester Wave evaluation, other areas remain ripe for
innovation and improvement:
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 14
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Testing. With one or two notable exceptions, the vendors we included in this study did not excel
at any aspect of testing support. To some extent, the weakness of testing features is characteristic
of the ALM tools market, not just the vendors we assessed. However, its still worth noting that
some specialty vendors have done a better job of connecting testing to other ALM activity.
Some requirements tools vendors, for example, provide rapid test generation from requirements,
something that many purveyors of bigger ALM suites have yet to implement.
Collaboration. While we have seen some progress in this area, thats damning by faint praise
considering how ofen collaboration is at the core of an organizations ALM challenges. Take,
for example, the very common challenge of collaborating with an ofshore partner. While some
features, such as feeds or wikis, make it easier for onshore and ofshore teams to eavesdrop on
each others activities, these are not vehicles for communicating product vision, mentoring
people on the other team, or suggesting ideas for process improvement.
Mobile support. At the risk of stating the obvious, many aspects of mobile development user
experience, testing, deployment, etc. are signifcantly diferent from the elements involved
in building middleware. Yet the typical ALM vendor today has done very little to address these
needs, despite the continued expansion of the mobile market.
Integration. ALM vendors have acknowledged the inevitability of integration with other
commercial and open source tools. However, were a long way from arriving at a common
integration strategy that makes it easier for customers to select ALM tools based on their
individual merits outside of integration. Some vendors provide an application programming
interface (API) with few or no prebuilt connectors. Others provide plenty of connectors, with
perhaps a weaker API for custom-built integrations. Te fact that small companies, such as
Kovair and Tasktop, have established themselves as third-party providers of ALM connectors is
a testament to the weaknesses of the overall ALM markets integration eforts.
VENDOR PROFILES
Leaders
IBMs ofering is becoming more than the sum of its parts. IBMs position in this years
assessment of the ALM market is a testament to an important principle: Its not enough just
to have good products the products, in combination, must support common activities
in sofware development and delivery. In previous years, IBM has certainly provided strong
solutions, such as Rational RequisitePro, Rational Team Concert, and Rational DOORS in
specifc functional areas such as requirements, planning, and project management. However,
IBM provided multiple solutions in some of these areas, leading to confusion about what exactly
each of these tools was supporting and how they all worked together.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 15
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Tat confusion has largely dissipated with IBMs current ALM ofering. Not only has IBM
continued development of its strong suite of products, but it has also stitched them together in
a more coherent way. Te company has also made clearer the use cases it supports with its tools,
such as Agile teams and embedded sofware development. Because of the companys strong
product and portfolio strategy, it is possible to write about IBMs ALM oferings taken as a
whole with far less irony.
Rally Sofware continues its leadership in Agile/Lean. Rally targets a very healthy and
growing opportunity: the expanding number of organizations that have adopted Agile and
Lean. Rallys tools are optimized for Agile planning, project management, status reporting, and
other actions that happen within and outside sprints. Te companys acquisition of AgileZen,
a Lean project management tool, was a natural ft for both Rally and its customers. So too was
the addition of Rally Portfolio Manager, a tool for planning, decision-making, and management
above the level of an individual project or product.
Rally also continues to provide thought leadership in the Agile and Lean community. However,
this focus on Agile has a downside: non-Agile teams will fnd Rallys products and services far
less attractive than other general-purpose ALM tools. Rallys leadership rests with its breadth and
depth of capabilities for Agile teams, combined with a strong and focused corporate strategy.
PTC continues its leadership addressing regulated and digital products needs. PTC
provides strength where product-based development and compliance intersect. No other
company, with the possible exception of IBM, has done as much to support teams of embedded
sofware developers and systems engineers. PTC helps frms address digital product
development needs for products such as appliances, vehicles, and medical devices that have
sofware components while still maintaining support for teams building and delivering
sofware-only products and projects.
PTCs strength rests on its long-standing support for addressing governance and compliance
challenges. Burrowing into this niche has its concomitant cost: a more complex product
that is harder to justify adopting for teams that dont have to satisfy the needs of auditors or
government regulators.
CollabNet provides a fexible ALM framework optimized for the cloud. As already noted,
one of the strengths of CollabNets ALM suite is the fexibility with which users can defne
ALM content. We also see this fexibility pay of when trying to efectively address the needs of
complex teams or complex projects, where there is less chance for a one-size-fts-all design.
CollabNet has also invested heavily in supporting cloud development and deployment. Not
only are CollabNets tools available in the cloud (CloudForge), but they also include features
optimized for developing, testing, and deploying cloud systems.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 16
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
Microsof shortens cycle times. Starting with Visual Studio at the front end and through to
Team Foundation Server (TFS) at the back end, Microsofs ALM portfolio provides broad
life-cycle support. Microsofs overarching goal is to shorten cycle times, which requires a
combination of individual measures (task management, test automation, etc.).
Microsof deserves recognition for understanding how central collaboration is to ALM. While
the M word, management, is part of the ALM acronym, ALM initiatives frequently fail because
they emphasize control over collaboration. Features such as visualization via PowerPoint and
translating recorded manual testing into automated tests emphasize collaboration in these
two examples, with business users.
Serena Sofware orchestrates IT projects from idea to deployment. Since retooling its ALM
strategy, Serena has made impressive advances at both the individual tool and the suite level.
Central to its product strategy is orchestration, a vision of ALM as an ongoing, rhythmic
activity. Not surprisingly, given this vision of a regular fow of activity (conceive, build, test,
deploy), Serena has one of the best workfow designers available. Serenas recent enhancements
to its ALM portfolio, such as the Serena Release Manager, provide more support in this end-to-
end orchestration model.
Strong Performers
For HP, quality is truly central, but other ALM capabilities are growing. HP Quality Center is
the fulcrum on which HPs ALM strategy rests. HP Quality Center is a ubiquitous part of many
organizations ALM infrastructure. Consequently, other ALM vendors integrate with HP Quality
Center as a matter of course, almost as a price of entry into any potential customers shortlist.
HP has improved upon and expanded its ALM capabilities beyond test management and test
automation in the two years since the previous Forrester Wave evaluation of this space. HP
now provides improved requirements management and traceability. Another example, HPs
Application Lifecycle Intelligence (ALI), ofers a broad set of ALM data aggregation and
analysis features, with strong reporting and dashboarding components. Given the importance
of this information for everything from team-level continuous improvement to executive-
level portfolio management, it represents one of many areas where HP has moved quickly to
distinguish itself as a serious ALM player.
Atlassian focuses on issue tracking and collaboration within teams. Atlassians JIRA, a widely
popular issue tracking tool, ofen serves double duty as a requirements management system.
Te companys Confuence Wiki is equally popular with development teams. Te two products,
in tandem, support the various kinds of day-to-day work done within sofware development
teams, which are always looking for ways to better collaborate (for example, through a wiki).
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 17
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
While Atlassian has decided to provide a narrower set of ALM tools than other vendors, it
continues to hone the capabilities in the areas where the company believes it can excel, such as
providing a marketplace for its developer community to provide plug-ins. Te company has
made moves into more enterprise-level scenarios, with new capabilities such as Stash, a tool for
the management of Git repositories that enterprise customers demand. Atlassian has also made
enhancements to its support, sales force, and other aspects of its business needed to gain ground
in the enterprise market.
Contenders
Rocket Aldon provides strong operational support. Unlike other ALM vendors, Rocket
Aldons roots lie in sofware delivery, not sofware development. Rocket Aldon has focused from
the beginning on the operational side, including both automation and reporting. Unlike other
vendors, Rocket Aldon is expanding backward, not forward, in the sofware timeline, adding
support for more upstream ALM activities, such as demand management and requirements.
SUPPLEMENTAL MATERIAL
Online Resource
Te online version of Figure 3 is an Excel-based vendor comparison tool that provides detailed
product evaluations and customizable rankings.
Data Sources Used In This Forrester Wave
Forrester used a combination of three data sources to assess the strengths and weaknesses of each
solution:
Hands-on lab evaluations. Vendors spent one day with a team of analysts who performed a
hands-on evaluation of the product using a scenario-based testing methodology. We evaluated
each product using the same scenario(s), creating a level playing feld by evaluating every
product on the same criteria.
Vendor surveys. Forrester surveyed vendors on their capabilities as they relate to the evaluation
criteria. Once we analyzed the completed vendor surveys, we conducted vendor calls to clarify
and confrm their products capabilities.
Customer reference calls. To validate product and vendor qualifcations, Forrester also
conducted reference calls with two of each vendors current customers.
FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
The Forrester Wave: Application Life-Cycle Management, Q4 2012 18
2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012
The Forrester Wave Methodology
We conduct primary research to develop a list of vendors that meet our criteria to be evaluated
in this market. From that initial pool of vendors, we then narrow our fnal list. We choose these
vendors based on: 1) product ft; 2) customer success; and 3) Forrester client demand. We eliminate
vendors that have limited customer references and products that dont ft the scope of our evaluation.
Afer examining past research, user need assessments, and vendor and expert interviews, we develop
the initial evaluation criteria. To evaluate the vendors and their products against our set of criteria, we
gather details of product qualifcations through a combination of lab evaluations, questionnaires,
demos, and/or discussions with client references. We send evaluations to the vendors for their review,
and we adjust the evaluations to provide the most accurate view of vendor oferings and strategies.
We set default weightings to refect our analysis of the needs of large user companies and/or
other scenarios as outlined in the Forrester Wave document and then score the vendors based
on a clearly defned scale. Tese default weightings are intended only as a starting point, and we
encourage readers to adapt the weightings to ft their individual needs through the Excel-based
tool. Te fnal scores generate the graphical depiction of the market based on current ofering,
strategy, and market presence. Forrester intends to update vendor evaluations regularly as product
capabilities and vendor strategies evolve.
ENDNOTES
1
Forresters research indicates that the dependence on sofware to achieve business outcomes will only
continue through the next decade. Terefore, while packaged applications will continue to be important,
sofware development and delivery for building new applications, customizing packaged systems, or
performing custom integrations will be critical for bridging the gap between sofware capabilities
and business needs. See the January 30, 2012, BT 2020: To Thrive In The Empowered Era, Youll Need
Software, Software Everywhere report.
2
Te previous Forrester Wave contained many similar questions but focused more narrowly on Agile team
support. See the May 5, 2010, The Forrester Wave: Agile Development Management Tools, Q2 2010
report.
Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to
global leaders in business and technology. Forrester works with professionals in 17 key roles at major companies providing proprietary
research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 29 years, Forrester has been making
IT, marketing, and technology industry leaders successful every day. For more information, visit www.forrester.com. 60080

Forrester Focuses On
Application Development & Delivery Professionals
Responsible for leading development and delivery of applications
that support your companys business strategies, you also choose
technology and architecture while managing people, skills, practices,
and organization to maximize value. Forresters subject-matter expertise
and deep understanding of your role will help you create forward-thinking
strategies; weigh opportunity against risk; justify decisions; and optimize
your individual, team, and corporate performance.
ANDREA DAVIES, client persona representing Market Insights Professionals
About Forrester
A global research and advisory frm, Forrester inspires leaders,
informs better decisions, and helps the worlds top companies turn
the complexity of change into business advantage. Our research-
based insight and objective advice enable IT professionals to
lead more successfully within IT and extend their impact beyond
the traditional IT organization. Tailored to your individual role, our
resources allow you to focus on important business issues
margin, speed, growth frst, technology second.
FOR MORE INFORMATION
To fnd out how Forrester Research can help you be successful every day, please
contact the ofce nearest you, or visit us at www.forrester.com. For a complete list
of worldwide locations, visit www.forrester.com/about.
CLIENT SUPPORT
For information on hard-copy or electronic reprints, please contact Client Support
at +1 866.367.7378, +1 617.613.5730, or clientsupport@forrester.com. We ofer
quantity discounts and special pricing for academic and nonproft institutions.