Continuous SCRUM: Agile Management of SAAS Products

Puneet Agarwal
TCS Innovation Labs Delhi
154B, Block A, Sector 63,
Noida, Uttar Pradesh 201301 India
+91 120 6657366
puneet.a@tcs.com

ABSTRACT

1. INTRODUCTION

Hosted software-as-a-service products provide an opportunity to

provide consumers with continuous deployment of new features,
as opposed to scheduled version upgrades as is the norm for
products installed on-premise. In order to exploit this opportunity,
a SaaS provider needs to adopt an agile process that is capable of
releasing new features rapidly. The SCRUM [5,6] process is
ideally suited for this purpose: However, when SCRUM has been
used for agile development of an installed product, parallel,
overlapping sprints are executed by separate teams, each dealing
with short, medium, and longer-term enhancements to the
product[3]; with the result that version upgrades are therefore
easier to manage. In contrast, in the case of a SAAS product,
version upgrades are no longer a constraint, so we can do better.
In this paper we describe Continuous SCRUM, a variant of
Type-C SCRUM, augmented with engineering best practices, in a
manner ideally suited for managing SAAS products. In our
approach, bug-fixes, minor enhancements, as well as major
features are released continuously, on a weekly basis by a single
team, in contrast to Meta-SCRUM [3]. We also present field
data from our experience with using Continuous SCRUM for a
hosted platform-as-a-service product for more than two years. Our
experience reinforces other recent evidence [11] that rapid,
smaller releases are often preferable to infrequent, larger ones.
Continuous SCRUM provides a mechanism to achieve and sustain
a rapid release cycle, for SAAS products as well as, we believe,
for custom applications developed in-house.

A recent Forrester report [17] states that Some IT execs put off
upgrades as long as possible to avoid costs and minimize business
disruption. More-over there is an old saying: if it's not broke
dont fix it. As a result, enterprise software products installed onpremise are usually upgraded only when necessary, either due to
obsolescence (often forced by the vendor) or (more rarely) a real
need to incorporate new features. In contrast, applications
developed in-house are enhanced more frequently.
The reluctance to accept new versions of software products is
usually because of non-backward-compatible versions or due to
issues related to version-compatibility of other inter-dependent
software products in use. In both cases the risk of destabilizing
business services warrants care before accepting a new release. In
essence the issue is related to poor-quality releases rather than the
fact that new versions are deployed.
Additionally, it is also recognized that new technology as well as
competition between product vendors, rather than genuine userneeds, are often the genesis of new versions of products. As
consequence, users are even more skeptical of accepting new
versions; as a result, not more than one major and 3-4 minor
releases annually are accepted for deployment in practice.
In contrast, hosted software-as-a-service (SaaS) products offer
consumers new features on a far more regular basis. Often the
user is not even aware of a new feature until it suddenly becomes
available, resulting in a positive experience of delight.
Thus, from the perspective of the consumer, using a SaaS product
eliminates the cost of an on-premise upgrade. The task of
upgrading the solution is the responsibility of the service provider
and it is bundled with the service, and, as mentioned above, users
are in fact often unaware of the upgrade.

Categories and Subject Descriptors

K.6.3 [Software Management]: Software process
D.2.9 [Management]: Software configuration management,
Software process models (e.g., CMM, ISO, PSP), Productivity.

So, what is the difference between SaaS products and the

processes needed for their management, and the development of
in-house applications or products deployed on-premise? How do
agile methods apply to the development, testing, release and
deployment of SaaS products? Are there any lessons from the
SaaS world that could also be of use for the in-house custom
development? This paper is our attempt to shed light on some of
the above issues based on our experience of managing a hosted
platform-as-a-service product for a period of over two years, using
an agile process, which we call Continuous SCRUM.

General Terms
Management, Design

Keywords
SCRUM, Agile Process, Continuous Deployment, Release
Management, PAAS, SAAS, Configuration Management
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, to republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
ISEC'11, February 23-27, 2011 Thiruvananthapuram, Kerala, India
Copyright 2011 ACM 978-1-4503-0559-4/11/02 ... $10.00

In this paper we focus on the agile process required to manage a

hosted product and deliver new features on a weekly basis in a
sustained manner over a long period.
In SCRUM [5,6], software development proceeds as a series of
regular `sprints; we describe SCRUM in more detail in a later

51

section. We use a variant of Type C SCRUM [3], which we call

Continuous SCRUM, to achieve fast-paced continuous product
evolution and deployment on a weekly basis, without
compromising product quality.

Rational Unified Process, assuming that development, at least for

the particular product version, is over. The Rational process does
not say much about software maintenance, or managing the
release of multiple versions of a deployed software product.

In this paper we argue that SCRUM, or more precisely Type C

SCRUM [3], is by itself not sufficient to achieve a sustained
weekly release cycle. Our approach, which we call, Continuous
SCRUM uses a triple-sprint-overlap pattern, and additionally
incorporates the following extensions and best practices to Type C
SCRUM, each of which we shall describe in detail in Section 6:
1.

2.
3.
4.

5.
6.

7.

Extended sprint planning including the clarification of

product requirements and detailed item-level planning by
defining post-fix scenarios.
Introducing product-owner reviews during the sprint to
enable and approve changes to product requirements.
Triage meetings that enable the product-owner to manage the
back-log of product requirements.
A release-aware design process, together with an R&D
process for exploration, development, and de-construction of
significant new features that cannot immediately be broken
down into small enough work segments.
Automation and integration of configuration management,
build, release and testing processes.
Time-boxed sprints (as opposed to the more commonly used
feature-boxed sprints); i.e., some features may be dropped
from the sprint in order to meet the deadline.
Managing critical situations systematically using a decisiontree-based approach to avoid or manage both feature and
schedule slippage when required.

Figure 1: Dimensions of Agility in Software Engineering

It is widely recognized, as well as documented in studies [16, 22],
that one of the major causes of software project failures are poor
quality requirements; thus, while the developed software functions
perfectly, i.e. runs without errors, it fails to fulfill the actual
needs of users. Agile methodologies to address the requirements
phase in an iterative manner, often interleaved with development
activities, include Extreme programming (XP) and Feature
Driven Development (FDD):

The remainder of the paper is organized as follows: Section 2 sets

the context with an overview of agile methods with respect to
different dimensions of agility. Section 3 describes the challenges
of achieving weekly releases and summarizes how these
challenges are addressed by our Continuous-SCRUM approach. In
Section 4 we place our work in the context of related work,
followed by an overview of SCRUM in Section 5. In section 6 we
describe our Continuous-SCRUM process in detail. In section 7,
we present our experience and field data from the use of
Continuous SCRUM over a period of two years, followed by our
conclusions and suggestions for future work.

In Extreme programming (XP) [9], the software life-cycle begins

with writing of requirements in the form of User-Stories. This is
followed by by Release Planning Meeting; wherein developers
estimate the time required to build all these use-stories, customerrepresentative decides which of the user-stories should be built
first, resulting in a release-plan. Test cases are also prepared as
soon as the user-stories are ready. Next, in the the iterative
development process, user-stories are then broken into smaller
tasks (each task requiring lesser than 3 days of effort) and
assigned to developers. All the code to be included in the product
delivery is written through pair programming and follows testdriven-development approach. The customer-representative is also
part of the team and is always available for discussions. The
strength of XP is its integrated engineering practices. One of the
most important highlights of XP is that it allows changes in the
functional scope as well as change in user-stories during
corresponding iteration. However, it is not necessary for each of
the iterations to result into production release.

2. Dimensions of Agility
Fred Brooks has argued in [15], and more categorically stated in
his recent book [14], that the waterfall model does not work.
Agile methodologies attempt to rectify the shortcomings of the
waterfall model through variations of the iterative development
model, wherein certain or all phases of the software development
life cycle (SDLC) are executed iteratively. We can classify agile
methods based on which particular phase of the SDLC is executed
iteratively. Further, depending on which phases are involved,
different agile methods, such as RUP [7], XP [8,9] and SCRUM
[5,6], address different dimensions of agility, viz. development,
requirements, and continuous deployment respectively, as
depicted in Figure 1.

Similar to XP, FDD [10] first iterates over the requirements(two

phases) and then the three of the phases are iterated over and over
again until objectives are met i.e. Plan by Feature, Design by
Feature and Build by Feature. FDD is still evolving and this
approach has shown considerable results for many a situations.
Continuous deployment in an iterative manner is addressed by the
SCRUM family of agile methods. SCRUM is an iterative,
incremental process of software planning, development, testing,
release and deployment. Thus in SCRUM, all phases the SDLC
are iterative. It is important to note that both XP and FDD focus
on the development of a software artifact, i.e. a product version,
and, unlike SCRUM, do not mandate a working increment to the
software product that is available and deployed in production at

The Rational Unified Process (RUP) and its variations [7], such as
the Essential Unified Process, Open Unified Process and Agile
Unified Process mainly prescribe multiple iterations of the
development phase of the SDLC. Each of the iterations produces a
working prototype of the software product being developed, not a
released version deployed in production. Software release and
deployment is handled during the transition phase of the

52

the end of each iteration. (In practice, combinations of XP/FDD

and SCRUM are often used with the aim of improving the quality
of requirements as well as enabling continuous deployment.)

approach. In the Java world, tools such as JUnit provide the

means to achieve test-driven development. However, such
techniques and tools require significant effort and training. Testdriven development, while desirable, is therefore also costly,
especially in the context of a weekly release cycle. Further, the
test-driven approach can at the best improve the quality of unit
level testing. For these reasons, we do not use the test-driven
approach, and submit that it may not be warranted for shorter
release-cycles.

3. Challenges in Achieving Weekly Releases

There are many popular accounts of nightly-builds as an
example of rapid and agile development. However, a build is
different from a release. The root of this difference lies in the
often experienced anecdote: My work is 90% complete is
actually found to be 50% complete mostly because that remaining
10% takes more effort to complete than anticipated. Bob Galen
recently described these and similar experiences in [4]. For the
context of this paper we shall use the following definitions of a
build and, in contrast, a release:

Developers and testers use a variety of technologies for

component level testing treating the components as a black-box
that take some input and produce the output (e.g. JUnit). Similar
technologies are used for link testing of multiple components
working together (e.g. StrutsTestCase, XMLUnit and Cactus).
Managing a multitude of test scripts using different technologies
and refactoring the test automation programs on weekly basis to
match the pace of feature evolution is a major challenge; an
effective QA process using a single family of test automation
tools in an integrated manner is essential aspect of any process
supporting weekly releases.

Build: an integrated version of a software system encompassing

all components that compiles successfully, is deployable and all
basic test-cases succeed.
Release: an integrated version of a software system encompassing
all components, which has also undergone sufficient quality
assurance (QA) testing, and can therefore be deployed in
production and released to the users of the system.

Finally, what if a critical bug is discovered few hours before the

release? Complex situations often pose a challenge for the
development teams, and lead to a state of fix it fast; a tricky or
unusual situation arises often leading to delays, loss of man hours
and also bad-quality. Further, if the cycle breaks once or it starts
breaking frequently it becomes very difficult for the team to go
back to normal cycle. Therefore, a balanced and predictable
mechanism needs to be prescribed for each such exception
condition so that it is handled in a controlled manner without
affecting either the product quality or upsetting the release
momentum.

In the traditional world of installed software products, it usually

takes anywhere from 4 to 30 weeks for a new version of a
software product to be released. Builds, on the other hand, can be
a daily (or nightly) affair.
Weekly releases can be achieved by making the functional spread
of each upgrade smaller. However, reducing the functional scope
alone does not automatically reduce the duration of software
development life cycle from many weeks to one week, or result in
a stable release, for reasons described in more detail below:
Upgrading infrastructure software often does not add direct value
in terms of new functionality for users, and is usually only an
enabler for the development teams to deliver features better in
future. At the same time, it is often the case that the effort
required for the upgrade is high along with the risk that it might
bring about functional instability in the system. Thus, the cost of
such upgrades leads to an argument that we dont need so many
frequent upgrades. It is therefore very important to make these
weekly releases acceptable by the user community. For this to
happen, not only should the cost of an upgrade be minimized, it is
also highly desirable to bundle infrastructure upgrades along with
some new functional features so that the upgrades value is
directly perceived.

We propose a new approach in this paper that we call Continuous

SCRUM, which is a variant of Type C SCRUM wherein parallel
overlapping cycles of work are executed by a single set of teams.
We also augment our approach with engineering best practices to
combat the challenges described above so as to ensure regular
weekly releases. Collectively, these practices help in achieving
better developer productivity, regularity in the weekly releases,
hourly builds, backward-compatible releases, better quality of
releases and most importantly in meeting end-user expectations.

4. Related Work
As we have already discussed, iterative development processes,
such as RUP [7] or XP [9], focus on either the development or
requirements phases and do not directly apply to the problem of
rapid and continuous deployment of new versions of a software
product in production.

A faster release cycle should not compromise on quality: Users of

a SaaS service may often attribute any problem encountered to the
version upgrade; it is therefore very important for the engineering
and QA teams to ensure backward compatibility of enhancements,
so that users perceive only improvements rather than experience
any loss of functionality, even if it is replaced by newer and better
features.

One of the first descriptions of an agile production process was

by Takeuchi and Nonaka in 1986 [1], in the context of lean
manufacturing. Sutherland and Schwaber [2] re-applied the core
concepts of lean manufacturing to software production in order to
enable continuous development and deployment, with their
definition of the SCRUM agile software development
methodology as described in Section 5 below.

Open source communities along with many other agile software

development teams have been following a nightly build process as
a matter of practice. Even using nightly builds these teams are
able to deliver a new version of software at best on a monthly
basis, and most often not even that. A weekly release cycle must
do better, and hourly builds need to be supported along with the
required automation infrastructure.

An application of SCRUM methods in a manner so as to enable a

large number of rapid releases, termed as a hyper-productive
process, was described by Jeff Sutherland in [3]. The approach
used was a Scrum-of-Scrums, or a Meta-SCRUM, as described in
detail in Section 5.2 below. As documented in [3], this approach

Open source communities and other practitioners of agile software

development usually follow test-driven development [9]

53

was capable of producing up to 65 releases a year, and in practice

about 45 could be achieved.

between only concerned team-members. Daily scrum meetings

have a maximum time limit of 15 minutes [6].

Our Continuous SCRUM process is also able to achieve a similar

release volume, with 49 releases being deployed in a year, as
detailed in Section 7 below. However, our approach improves on
that of Meta-SCRUM [3] in that the frequency of release for all
types of changes, (including major and minor enhancements or
problem fixes), is based on a weekly release cycle, and not only
that for problem-fixes as in [3]. Thus, as we shall recount in
Section 7.5, there have been cases when two major features have
been deployed within the same month, or many minor
enhancements in successive weeks. Using Meta-SCRUM, on the
other hand, major features are deployed quarterly, minor features
monthly, and only problem-fixes on a weekly basis.

Each sprint must result in a complete and working and adequately

tested build of the software product. Further, some sprints may
also result in a production release that is made available to
users. After the completion of a sprint, a sprint review meeting
is called: In this meeting team-members demonstrate the newly
developed features to the product-owner and other stakeholders,
such as user representatives. Each sprint review meeting can last
for at most 4 hrs [6].
During a sprint, an updated burn-down chart indicating the
pending items from the sprint-backlog, is available to all the teammembers to assess and determine the quantum of pending work
remaining in that sprint. Developers keep these burn-down
charts up-to-date by marking the work-items completed as and
when they finish their work.

Additionally, earlier descriptions of SCRUM-based development

have been for installed software products rather than for a hosted
SaaS service as we describe. The ability to release enhancements
of all types as rapidly as possible becomes especially important in
the case of hosted services. We readily concede that many hosted
services may be using variations of SCRUM which may be very
similar to the process we describe. However, we have not come
across a publication of the details of how weekly releases should
be organized in such a context. Thus, our description of
Continuous SCRUM and its application to hosted product
management is, to our knowledge, an original contribution to the
body of knowledge, which should be instructive for developers of
hosted SaaS services.

Figure 2: Types of SCRUM

Lastly, our Continuous SCRUM process includes a number of

detailed process improvements (best practices) as outlined in
Section 1 and described in detail in Section 6 below. We believe
that taken together, our process and set of practices constitute a of
novel improvement to the SCRUM methodology.

Based on the time gap between end and start of two consecutive
sprints on, or the degree of overlap between multiple sprints, there
are different types of SCRUM, identified as Type A, Type B and
Type C, as described in [1, 3]. These different types of SCRUM
are illustrated in Figure 2. (It has been observed that Type B and
C SCRUM result into hyper-productive teams that produce new
features at a very a rapid rate [3].) We describe each type of
SCRUM in detail below.

5. ABCs of SCRUM
At the core of SCRUM is an iterative, incremental process of
software planning, development, testing, release and deployment.
Thus, all phases of the SDLC are iterative. The set of
requirements for the software product, at a given point in time, is
divided into smaller isolated cycles of work called sprints.
Regular team meetings are another important feature of SCRUM.
The meetings before the start and end of each sprint are especially
important Before a sprint, a sprint planning meeting is held to
define the list of features to be included in the sprint, their
estimated time for completion and feasibility; this list is also
referred to as the sprint backlog. (We shall also refer to features
as work-items.) Both team members and managers attend this
meeting which usually lasts for eight hours (often in two spells of
4 hours each) [6]. Once a sprint-backlog is approved and
committed to by the team members; it cannot be modified until
the end of sprint. Usually no schedule slippage is allowed and
some groups practice SCRUM with 10% as maximum permissible
schedule slippage. If the estimated slippage is greater, some
features are dropped from the current sprint-backlog.

5.1 Type B SCRUM

In Type A SCRUM, there is a small gap between two consecutive
sprints. This is the time when the product-owner and developers
prepare the product-backlog: The product backlog [5] is A
prioritized list of project requirements with estimated times to
turn them into completed product functionality. In order to
reduce time-to-market, the product-backlog is kept ready, much
before end of every sprint so as to reduce the time lag between
two consecutive sprints. This is the main objective of Type B
SCRUM.
Type B SCRUM was first practiced in Fuji-Xerox Japan [1]. In
Type B SCRUM, the first and last phase of a sprint overlap with
the preceding and succeeding sprints respectively. Thus, a single
sprint has at least three phases, with the first phase of a sprint
overlapping with the last phase of preceding sprint (and the last
phase of the sprint overlapping with the first phase of next sprint).
Thus also, every sprint has a phase which does not overlap at all
with any other sprint (as shown in Figure 3).

In the original description of SCRUM [2, 6], a sprint would last

for 30 calendar days. On all working days in this period a daily
scrum meeting is held where only team-members speak and
managers listen. Team-members synchronize their work with each
other; discuss the work they have accomplished in the previous
day, and what they are planning to do the next day. If any issues
surface out, a separate meeting is called after the daily meeting,

Figure 3: Type B SCRUM Non-Overlapping Phases

54

Type B SCRUM results in a hyper-productive development

team [3]. If practiced with disciple, it can result in a more
predictable and regular release cycle than Type A SCRUM [3]. In
most descriptions of the use of Type B SCRUM [3], the typical
sprint duration is a few months, and functional features are not
dropped even if doing so results in minor schedule overruns, as
long as they do not affect the next sprint.

since multiple sprints overlap; however each sprint is executed by

a different team and therefore Meta-SCRUM is also known as
Scrum of Scrums (as shown in Figure 4). Note however that the
lower level SCRUMS are not Type C SCRUMS themselves, since
there is no overlap among consecutive sprints therein.

A situation where Type B SCRUM is found useful is when other

team members who become idle in the non-overlapping phase of a
sprint, (typically those involved in planning or testing functions),
can be involved in some other products in the same development
organization. However, synchronization between the sprint
schedules of different products can be a challenge. Therefore, in
practice Type B SCRUM is adopted as a stepping stone for a team
to ultimately move to Type C SCRUM.

Figure 4: Type C - Scrum of Scrums

For the above reasons, and as also conceded by Sutherland in [3],
Meta-SCRUM is not a true implementation of the intent with
which Type C SCRUM was defined. Sutherland also describes
what could constitute True Type C SCRUM. Brent and Evan
[13] have discussed what it takes for an organization to implement
Type C SCRUM. The conclusion drawn is that Type C SCRUM is
a very hyper-productive process. Thus, it is usually difficult for
teams to follow, and equally difficult for stake-holders (sales staff
or end users) to absorb and assimilate the fast pace of feature
evolution, albeit in the context of traditional installed software
products. However, as we now show, Continuous SCRUM is, in
sprit, a true Type C SCRUM that can and has been achieved in
practice, as well as consumed with ease by users, in the context of
a hosted SaaS product.

5.2 Type C SCRUM and Meta-SCRUM

Nonaka and Takeuchi pioneered the usage of Type C SCRUM in
Honda and Cannon [1]. They describe an approach wherein
different phases of the same project overlap to create a product
faster and better. A typical cycle was that of R&D-DesignProduction-QA-Marketing-Sales. Instead of following a
traditional relay-race model they used overlapping phases: First to
engage was the R&D team; before R&D finishes, the production
and QA teams also began working on the product release. The
production team could therefore inform the R&D team of any
adjustments to the design from the perspective of ease and
efficiency of production; similarly the QA team would understand
the new product ahead of time to be able to execute QA more
efficiently when their turn came. Further, the R&D team remained
involved even towards the very end of the project so that lessons
could be learnt for the next production version, i.e., the next
sprint.

6. Continuous SCRUM
The process we describe now abstracts from our experience of
practicing weekly releases in the engineering team of a hosted
platform-as-a-service (PaaS) product called InstantApps [19, 20
and 21], over a period of more than two years. We have also
compiled a number of best practices from our experience, which
together are enablers of a sustainable weekly release cycle. These
best practices are technology agnostic and can be applied to any
agile methodology with suitable tailoring. At the same time, the
process we have used, which we call Continuous SCRUM, is, we
claim, an example of True Type C SCRUM. Continuous
SCRUM achieves fully overlapping sprints using a single team,
with different team-members purposefully engaged in different
phases of consecutive sprints.

It is instructive to examine applying the Type C SCRUM of

Nonaka and Takeuchi [1] to a software development life-cycle,
consisting of Requirements, Design, Construction, Testing, User
Documentation and Production Deployment: Nonaka and
Takeuchis process merely translates into beginning later phases
(such as Construction, Testing and User Documentation) before
Requirements gathering finishes. User documentation and QA
teams can get engaged right in the beginning of the project and
assist the design team to follow, for example, test-driven
specification. Beginning construction early is merely a form of
rapid prototyping. Thus, there are unlikely to be significant
improvements in agility (i.e., release frequency), although
software quality may certainly improve due to overlaps between
some aspects of later phases (QA planning) with earlier ones
(Requirements definition).

6.1 Triple-Sprint Overlap Pattern

We submit that each sprint can be considered to have three
phases: planning, development and QA. (Our model can be
extrapolated easily to cases where there are more than three
distinct phases in every sprint; however, we confine ourselves to
three phases per sprint.)

Sutherland [3] first formally defined Type C SCRUM as a

software development methodology. In Type C SCRUM, by
definition, all phases of a sprint overlapped with other sprints for
the same product (Figure 2). It is not necessary for these different
sprints to be carried out by the same team, nor does is it mandated
to have different teams for each of these overlapped sprints. In his
main paper [3], Sutherland described a Meta-SCRUM as an
example of a practical application of Type C SCRUM, but with
different teams working on overlapped sprints. In Meta-SCRUM,
there are three types of sprints that go-on in parallel: Weeklysprints provide bug-fixes, monthly-sprints provide customerfocused features and quarterly-sprints provide major product
enhancements. Meta-SCRUM is an example of Type C SCRUM

The team is segregated into team-members who are responsible

and capable of executing each of these phases; so there are three
sub-teams, each having a different function: planning,
development and QA. Each sub-team works on the same phase,
but for a different sprint each week.
As shown in Figure 5, each sprint is time-boxed into a three week
period. Continuous SCRUM is very similar to a production line
where material moves on a belt and each work-station add value
as it passes. In the first week of a sprint, the product-owner, and
scrum-master , together with inputs from the development team
(who are involved part-time as needed) formulate the plan for
next remainder of the sprint. In the following calendar-week, the

55

above sub-team (i.e., the P1-Team) starts planning the next

sprints. Meanwhile the development team (the P2-Team) starts
development on the sprint that was planned by the P1-Team in the
prior week. Similarly the QA team (i.e., the P3-Team) performs
QA of what was developed by P2 team in the prior week. Thus,
consecutive sprints overlap with a phase-lag of one week.

single sub-team assigned to this task. Note once more how our
approach differs from the multi-team Meta-SCRUM [3], where
different teams develop different types of features. Our approach
also avoids the need for intra-development-team synchronization,
which is needed in Meta-SCRUM, with the team working on
larger sprints (i.e. focused on customer required features or the
one focused on product roadmap) needing to co-ordinate with the
weekly team on regular basis, which in some situations may
warrant considerable additional effort.
At the end of phase P3 (i.e. QA) a sprint normally results in a
well-tested set of features constituting a new release, which is also
deployed into production. (We describe details of how this is
actually achieved in the sections below.) Thus, a new release is
deployed every week, even though each spring itself takes three
weeks.
However, if a release-candidate still has critical bugs with which
the software cannot be released, the release needs to be cancelled
and either the current (almost complete) sprint needs to be merged
with its successor. As a consequence the functional scope of the
successive release will increase to that of two sprints. In this
manner we ensure that all sub-teams always have work, nobody is
idle, and most importantly the cycle of releases is not broken.

6.2 Best Practices for a Weekly Release Cycle

Is vanilla SCRUM, even with the triple-phase overlap described
above, sufficient to ensure a sustainable weekly release cycle? In
our experience, we have had to introduce a number of practices
over and above the basic SCRUM methodology, which we
described below.

6.2.1 Sprint Planning and Project Management

Daniela and James [16] and Sutherland [3] argue that clarity of
requirements directly impact overall productivity. Further, there is
a stronger relationship between the clarity of requirements and
productivity than between the quality of design specifications and
productivity. It therefore becomes important to ensure better
clarity of requirements in order to achieve higher overall
productivity.
Figure 5: Continuous SCRUM
As highlighted in [3], in Type C SCRUM all sprints are timeboxed, as in our process. Therefore, similar to a production line, a
regular output can be expected unless there is some blockage
along the line. The triple-sprint overlap with multiple functionally
segregated sub-teams, as described above, has many important
benefits that contribute to the efficacy of a weekly release cycle,
especially in the context of continuous deployment of a hosted
solution:
As described in [14], to maintain conceptual integrity of a design
it is always better to have single person responsible for ensuring
that specific concerns or needs are addressed. Representing the
users point of view is the responsibility of the product-owner;
similarly compliance with architecture guidelines and quality of
service needs must be ensured by the engineering team through
the scrum-master. Further, the QA team should report to productowner directly, and not to the scrum-master (engineering
manager); thereby reinforcing the users perspective during
testing.

In order to plan a sprint in SCRUM, items from the productbacklog are chosen (also termed as sprint-backlog) to be
fixed/developed in that sprint. The scrum-master then assigns
these work-items to individual developers.

For the same reason, i.e., maintaining conceptual integrity, it is

desirable for all actual software programming to be done by the

Developers should replicate every bug assigned to them. If it is a

new feature requirement, they use the product in an as-is manner

Figure 6: Sprint Planning

However, our sprint planning phase (P1 in Figure 5) lasts an entire

week. In this period, in addition to deciding on the sprint-backlog,
developers also clarify requirements in detail (see Figure 6):

56

and then understand the new feature requirement. Each developer

publishes a post-fix-scenario against every work-item and sends
it to the product-owner for review (This may sometimes involve
liaison with originator of the bug/feature, both by the developer
and product-owner). In this phase developers also perform
backward compatibility analysis of every work-item and make
suitable adjustments to the post-fix-scenario. After these postfix-scenarios are approved by the product-owner, the team is
ready to start executing P2 of the sprint. Detailed requirements
clarification in the above manner during P1 ultimately contributes
to higher overall productivity in the long-run.

teams leads to better understanding and reduce inter-team conflict.

(iii) Triage meetings help the development team in understanding
the rationale for work-items, thereby leading to better quality of
fixes/code.

6.2.3 Release Aware Detailed Design Process

As highlighted earlier in Section 3, ensuring full backward
compatibility and exemplary quality of releases is very important
for the acceptability of new versions of the product on a regular
and frequent basis. Agile methodologies [9, 11] propose that this
goal be achieved by dividing the large version upgrade into
multiple smaller releases i.e. a new feature is divided into smaller
tasks; these tasks get released into production over multiple
releases in sequence. Consequently a new feature may get
partially released into production and not be visible to end-users
and then finally when the last task for a new feature is released it
becomes visible to them: Therefore, in the planning phase of our
sprints, the high level feature requirements are be broken down
into smaller tasks. In this manner we achieve parity among workitems thereby also making it easier to measure overall progress on
a weekly basis. (The above approach is very similar to the one
followed in XP [9], of dividing the user-stories into smaller tasks.)

Finally, developers are also allowed to make changes to the postfix-scenario during P2; subject to review and approval by
product-owner.

6.2.2 Backfill Product-Backlog Triage

The work-items being released through the overlapping series of
sprints need to fall in line with end-user expectations as well as
pre-planned overall product-roadmap. By maintaining a productbacklog, higher priority work-items can be identified during the
sprint planning process. However, note that SCRUM does not
impose any specific approach for prioritization of work-items.

The product-backlog typically contains three types of items:

bugs, enhancement request and new features. Effort required for
any bug to fix is usually less than 4 working days. If the effort
required is more than 4 days, then the work-item is divided into
smaller work-items.

Many of approaches, such as [18, 10] have been used for

assigning priorities to requirements. We introduce a light-weight
approach, sufficient for the purpose of assigning priorities to finegrained work-items on weekly basis. More detailed approaches
are best used at the time of defining the product-roadmap. We
convene a weekly Triage Meeting to assign priorities to the
work-items in concurrence with end-user expectations and
product-roadmap.

Enhancement requests often require more than 4 days of

construction effort. All such items are carefully divided into
smaller work-items and their dependency to the main work-item
marked and tracked. The correct sequencing of these smaller
work-items is also noted and tracked.

As illustrated in Figure 7, the product-owner defines a triage

committee. The triage committee has three fixed members:
product-owner, scrum-master and QA-lead (shown with crossing
hash-lines in Figure 7). Apart from these three fixed members,
three members are nominated every week with one from each of
the users community, engineering team and QA team (shown
with horizontal hash-lines in Figure 7). As far as possible the
nominated members are changed frequently (e.g. every month).

Some new feature requests can also be managed in the similar

manner as described for bugs or enhancement requests as above.
However, there are certain new features that require significant
research oriented effort, resulting in a completely new component
of the product. We ensure that such exploratory activities are not
done by the engineering team; instead a separate team called the
R&D team performs such explorations. It may appear that we are
reverting to the Meta-SCRUM approach by this separation of
teams: However, it is important to note that the R&D team does
not have check-in privileges in the source-code repository. They
develop such a new feature as stand-alone component, as far as
possible in a manner that it becomes pluggable to the main
product without enhancements to the core. Sometimes, the R&D
team may require that product enhancements are required even in
order for making the new feature pluggable with to the main
product; such enhancements are requested for and scheduled into
the main sprint series, and executed by the engineering team.

Figure 7: Triage Meeting

Triage meetings are chaired by the product-owner and are timeboxed to two hrs. In a triage meeting, the QA Team or productowner puts forward bugs, enhancement requests, and new feature
requests for discussion, and the severity and priority of each item
is decided collectively. (Occasionally, the scrum-master also
suggests new items that might improve the stability or
maintainability of the product). The triage committee also submits
regular reports that are in turn used to maintain the overall product
roadmap.

Once such a new component is created and tested by the R&D

team (note that the QA team is not involved at this stage), it is
reviewed by the product-owner and broken up into smaller workitems to be included in the sprint planning process to be taken up
by the product development team. The developers often modify
the component as prototyped by R&D so as to meet architectural
guidelines and quality of service needs. Thus, in-spite of another
team developing the new feature, the conceptual integrity of the
product design [14] is maintained.

The advantages of including triage meetings are as follows: (i) All

teams, and over time, all team-members, get to understand the
product vision and principles. (ii) Increased interaction among the

57

6.2.4 Efficient Build and Release Infrastructure

6.2.6 Readiness for Adverse Situations

An efficient build and release infrastructure is key to maintaining

a rapid release cycle. This infrastructure is dependent on the
technology being used for development of the software, IDE in
use by developers, configuration management system and
technical architecture of the software. The key technologyindependent aspects of an efficient build and release
infrastructure, such as the one we have built, are outlined below.

Often situations arise such as: A critical bug is found in last few
hours of testing just before a release is planned; further, it is
determined that fixing it will not only require more time but also
extensive testing. A well-defined decision making process as
shown in Figure 8 is followed.

1.

2.

3.

4.

5.
6.

As soon as any developer commits some code into the

system, this event triggers certain build scripts on a
central machine (dev-server), resulting in event driven
builds (as opposed to traditional nightly builds).
The dev-server is used by peers for testing and therefore
the team is immediately aware if the build breaks;
further some simple tests are also executed
automatically immediately after the build.
Simple controls for developers to move changes
corresponding to a work-item from one environment to
another (i.e. Dev to QA).
Simple controls for a release manager to pick and
choose items and create a draft release or release
candidate.
Simple controls for release manager to release a release
candidate version.
Simple controls to upgrade an instance of the software
to the newer version.

Figure 8: Sample Decision Tree for Release Management

Additionally, the product-owner uses tools to ascertain the
criticality of certain bug such as a crawler that can excavate
information about usage of certain feature in the PaaS
environment. (This may be feasible in PaaS environments only.)
In essence, a high degree of preparedness on the part of productowner and scrum-master renders the entire release process more
regular and predictable.

7. Case Study: InstantApps

InstantApps is a platform for building web-based transaction
processing applications [19, 20 and 21]. InstantApps is centrally
hosted and available to all TCS employees in a PaaS (platform as
a service) mode, and has been managed using the Continuous
SCRUM process described above for over two years. InstantApps
successfully released 49 versions of the product in 2009; in 2010,
by the time this paper was written, 29 versions have been released
in 30 weeks.

The build activities are performed by all the team members in

different stages of their work, rather than a single build-manager.
Instead, we do have a single release manager whose job it is to
prepare the final release once it has gone through full QA testing.
The degree of automation we use is quite detailed; for example:
Sometimes, a developer inadvertently fixes and checks-in a workitem that is not planned in current sprint. To safeguard against
such errors, developers are asked to mention the work-itemidentifier in the check-in comments and the configuration
management repository has check-triggers that ensure that a workitem is included in the current sprint plan.

The InstantApps has 13 people in its engineering team (including

the scrum-master), 2 people in the QA team, and a product-owner.
InstantApps is hosted in four different locations world-wide, for
faster access to global users. Product releases are deployed in all
four environments as they are released every week. Only once in
the past two years did we have had to revert an upgraded
production instance.

6.2.5 Test Automation

Automatic testing is an important aspect of agile development [8].
There are primarily two approaches for test automation: Codedriven testing and GUI testing. In code-driven testing developers
write testing programs (following a framework such as JUnit,
Cactus, etc.) to test the application units or components. Codedriven testing is best suited when following test-drivendevelopment. Extreme Programming recommends the usage of
code-driven and test-driven-development. However, code-driven
testing requires extensive efforts to maintain test-programs and
automates only the unit or component level testing. SCRUM does
not propose any specific approach for automation of testing.

7.1 Tailored SCRUM Practices

The guidelines of SCRUM mandate a sprint planning meeting
with all the members of development team participating that is
time-boxed to eight hrs (divided in two spells of 4 hrs each).
Instead, we followed a process wherein the scrum-master planned
the sprint alone, and later this plan was shared with the
developers; they were allowed to raise objection if any and were
given a day to do so. Thereafter one or two lead developers, the
scrum-master and product-owner would have a meeting timeboxed for one hour, where they would modify and approve the
plan for next sprint. We introduced this modification primarily
because the development team was relatively junior, with an
average industry experience of one or two years; team-members
were therefore not comfortable speaking their mind in larger
meetings. Therefore also, the InstantApps team did not have daily
SCRUM meetings as mandated by SCRUM, as it was realized
that junior team-members rarely spoke in such group meetings.
Instead, at random intervals during a day, short meetings were
called to discuss issues as and when they surfaced during
development; importantly, all required stake-holders, such as the

We recommend and use automated GUI testing in Continuous

SCRUM. In GUI testing [12], a test-automation software is used
to record mouse movements and key-presses, and replay these
when needed. The QA team has to test the application in
recording mode and create test-scripts for future regression
testing. At the same time, in our experience while using many
GUI testing tools we still face challenges in identifying the
occurrence of errors. The QA team-members have to watch the
system manually as the automatic testing takes place. (We believe
this is possibly an area for possible future research contributions,
utilizing advanced techniques in video analytics.)

58

product-owner or QA team, were called in to resolve such issues

in an impromptu fashion.

6 RCT

The InstantApps team built an application named Tracker

(incidentally, using the InstantApps tool itself) to track productbacklog, product-dashboard, sprint planning, Dev-to-QA
promotion, QA-to-ReleaseCandidate promotion, and release
management. The Tracker application also tracked the progress of
work and indicated any items that were left out in a sprint as
compared to its initial plan. Sutherland [3] also points out that
there should be single application for performing such functions;
the Tracker application, in a sense, embodies the Continuous
SCRUM process as applied to InstantApps development.

Release Candidate Testing: Detailed feature

testing performed by the QA team in the prerelease environment on the release candidate.
This is first round of testing in third phase of all
sprints. While this testing happens no upgrades
are allowed to the release candidate.

7 BCT

Backward Compatibility Testing: Pre-built

applications are tested on the newer version to
ensure that the existing applications continue
working fine after the new release. This is most
critical round of testing for acceptance of the
weekly releases.

7.2 Development Environments

8 PRT

After the new version is released and applied in

the User Trial environment, single thread
integrated test is carried out to ensure that the
production upgrade is successful.

Figure 9 summarizes all the environments that are used by the

engineering team of InstantApps as the sprint progresses through
different phase towards a release. Most of these environments are
actually a family of environment in themselves because of the
various flavors of infrastructural software supported by
InstantApps (e.g. databases, application servers, browsers etc.). (It
is important to note that, as mentioned above also, that the
production environment is additionally replicated in four global
locations, on each of multiple software platform combinations.)

7.4 An InstantApps Sprint

The first phase of each sprint starts with the Triage meeting as
described in Section 6.2.2. Once the sprint-backlog is ready, the
scrum-master prepares the plan, shares it with the team and the
product-owner. Once finalized, work-items are assigned (in
Tracker) to the respective developers by the scrum-master.
Developers replicate the problems or ask questions about clarity
of business need and product-owner clarifies those aspects.
Next the sprint moves into development; as and when each
developer completes an individual work-item they mark its status
as completed (in the Tracker application) after performing their
round of UT (unit-testing; see Table 1). The scrum-master, or
delegated team-members, perform peer testing of all such workitems, which is also updated in Tracker. Developers then promote
such work-items to the QA server and test (QA1). After
successful testing they mark the status of the work-item as
verified. The QA team performs QA2. Towards end of the week
in P2, the scrum-master announces a code-freeze; the QA team
then locks the QA environment and performs QA3. After QA3 is
successful, all these work-items are moved to the releasecandidate environment and the QA environment us unlocked, so
that developers can start promoting work-items for next sprint.

Figure 9: Developer Ecosystem

7.3 Testing Cycles

To ensure quality and stability of each release, many stages of
testing are carried out for each sprint in various phases as
described in Table 1.
Table 1: Rounds of Testing for every sprint
S Testing
N Code

Description and Purpose

1 UT

Basic unit testing is performed by developers

on their respective desktop environments.

2 PT

Peer testing performed on the Dev Server, to

verify following:

In the third week of the sprint, the QA team locks the releasecandidate environment and performs RCT and BCT. In case any
work-items are rejected by QA, they are removed from the release
(with the assistance of the respective developer). Once releasecandidate is declared as ready by the QA team, the release
manager generates upgrade scripts that are applied in all the
production environments. (Occasionally, the QA team may
require an entire release to be withheld and postponed by a few
days, or rarely, even cancelled and merged with the next sprint.)

1. Whether some important aspect has been

missed related to the reviewers module.
2. Whether the main claim / objective of the
work-item has been achieved.
3 QA1

After promoting a work-item to the QA server,

developers test it themselves, in that server,
mainly to check whether it has been promoted
successfully and ensure that the release scripts
for the work-item have been created and tested.

4 QA2

Main claims and objective of the work-item are

tested by the QA team against the requirements.

5 QA3

After code-freeze is announced in P2, a last

round of testing is collectively performed by
QA Team for all the newly promoted workitems. The primary objective is to cover all
critical use-cases of the product to qualify the
final build of the week for a release candidate.

7.5 Productivity Statistics

As a result of our use of the Continuous SCRUM process, apart
from regular bug fixes, many enhancement requests as well as
radically new features were often released together in the same
month. This helped the InstantApps team to being responsive to
critical needs of its users. Table 2 illustrates the nature of workitems released in the month of Jun 2010 (four weekly releases).
As is clear from the table, along with 42 defect fixes, 18 minor
enhancements and 5 new features were released in the same
month. (Note that this would have been impossible using say, the
Meta-SCRUM approach.)

59

Table 2: Types of Work-Items in the 4 Releases in June10

Solicited By

Type

Count

Planned

Defect

41

Planned

Enhancement

13

Planned

New Feature

Customer Requested

Defect

Customer Requested

Enhancement

Customer Requested

New Feature

Total

Defect

42

Total

Enhancement

18

Total

New Feature

[4] Bob Galen Setting Agile-Centric Release Criteria aka

Driving Done-Ness Agile 2009 - Nashville USA.
[5] Ken Schwaber, Agile Project Management with Scrum
Microsoft Press 2004.
[6] Ken Schwaber, Mike Beedle Agile Software Development
with Scrum Prentice Hall, 2001.
[7] Ivar Jacobson, Grady Booch, and James Rumbaugh The
Unified Software Development Process Addison Wesley,
1998
[8] Kent Beck, Martin Fowler, Planning Extreme
Programming Addison Wesley 2001.
[9] Kent Beck, Extreme Programming Explained: Embrace
Change, Second Edition Addison-Wesley Professional
2004.

8. Conclusions & Future Work

[10] Palmer, S.R., & Felsing, J.M. (2002). A Practical Guide to

Feature-Driven Development. Prentice Hall.

We have presented Continuous SCRUM, a variant of Type C

SCRUM, augmented with a number of engineering best practices
specifically tailored towards achieving a weekly release cycle for
a hosted software product. We have also described in detail how
our process has been applied to achieve such a weekly release
cycle successfully for over two years while managing a hosted
software development platform called InstantApps. We have also
emphasized that Continuous SCRUM should be deployed only if
there is sufficient appetite for frequent enhancements and that it
requires significant involvement from the business-users and
product-owner.

[11] Poole, D. (2008). Breaking the major release habit. ACM

Queue, 4(8), 46-51.
[12] Peter Farrell-Vinay Manage Software Testing; Chapter 7,
Auerbach Publications 2008, ISBN: 9780849393839
[13] Brent Barton, Evan Campbell Implementing a Professional
Services Organization Using Type C Scrum 40th Hawaii
International Conference on System Science (HICSS) 2007.
[14] Brooks, Jr., Frederick P. The Design of Design: Essays
from a Computer Scientist Addison-Wesley 2010.

We believe that the methodology and best practices we have

employed are potentially widely applicable: Enterprise IT
departments are gradually recognizing that they themselves need
to become `software-as-a-service providers to their businesses, in
order to bring down costs, improve productivity and deliver value
faster to their users. Thus, the problem of managing rapid releases
for a hosted product is likely to become even more important in
the future, cutting across vast tracts of enterprise IT in addition to
publicly hosted SaaS products. Thus, the Continuous SCRUM
process is potentially of interest for enterprise IT in general.

[15] Brooks, Jr., Frederick P. The Mythical Man-Month

Addison-Wesley 2010.
[16] Daniela Damian and James Chisan An Empirical Study of
the Complex Relationships between Requirements
Engineering Processes and Other Processes that Lead to
Payoffs in Productivity, Quality, and Risk Management
IEEE Transaction of Software Engineering, Vol32, No.7,
July 2006.
[17] Paul D. Hamerman, Connie Moore and Ralph Vitti
Application Upgrades: How To Make Upgrade Decisions
When Business Value proves Elusive Forrester June 25th,
2010

However, it remains to be seen how best to adapt our Continuous

SCRUM process for situations where large numbers of IT systems
need to be managed collectively, as also for legacy systems. Last
but not least, our experience with Continuous SCRUM has been
with a co-located team: The process will also need to be
significantly enhanced for it to be applicable in the case of
distributed teams as are commonplace in the light of global
distributed software development. In both the above cases, a
concrete next step which we are contemplating is a generalization
of the Tracker application as used in InstantApps, which embodies
the Continuous SCRUM process along with our best practices, to
settings such as general bespoke enterprise applications as well as
distributed teams.

[18] P. Berander, Evolving Prioritization for Software Product

Management, ser. Blekinge Institute of Technology
Doctoral Dissertation Series. Blekinge Institute of
Technology, 2007.
[19] Gautam Shroff, Dev 2.0: Model Driven Development in the
Cloud, Keynote Address - ACM SIGSOFT conference on
Foundations of Software Engineering, Nov 2008, Atlanta.
[20] Gautam Shroff, Puneet Agarwal, Premkumar Devanbu,
Instant Multi-tier Applications Without Tears, 2nd India
Software Engineering Conference, Pune, India, Feb 2009.

9. REFERENCES

[21] Gautam Shroff, Puneet Agarwal, Premkumar Devanbu,

InstantApps: A WYSIWYG Model Driven Interpreter for
Web Applications, 31st International Conference on
Software Engineering (ICSE) 2009, Vancuvar, Canada.

[1] H. Takeuchi and I. Nonaka, "The New New Product

Development Game" Harvard Business Review, 1986.
[2] K. Schwaber, "Scrum Development Process" in OOPSLA
Business Object Design and Implementation Workshop, J.
Sutherland, D. Patel, and J. Miller, Eds. Austin, TX, 1995.

[22] Charette, R. Why software fails? IEEE Spectrum vol. 42,

no. 9, pp. 4249, Sep. 2005.

[3] Jeff Sutherland, Future of Scrum: Parallel Pipelining of

Sprints in Complex Projects Agile 2005 Conference Dever.

60