Executing Agile in a Structured Organization: Government

All copyright and trademarks on the Material remain the property of The City of Calgary.

This document and information (the "Material") is provided solely for general illustration purposes and
does not create a business or professional services relationship. Laws and regulations vary by jurisdiction
and change from time to time; compliance with such standards depends on the particular circumstances.
Any reliance on this Material provided is solely at the user's own risk. The City is not responsible for any
liability for any direct, indirect, incidental, consequential or other damages resulting from the use, misuse
or misinterpretation of the Material. Before making business decisions, please consult a professional
advisor.
 Executing Agile in a Structured Organization: Government
Janette Scott, Robyn Johnson, & Michael McCullough
The City of Calgary
Janette.Scott@calgary.ca Robyn.Johnson@calgary.ca
Abstract
The City of Calgary recently completed a business
process reengineering of their application development
process. As a result, an Agile Pilot was born. This
experience report shows our Agile journey from
conception, through adaptation and execution.
1. Agile Here?? No WAY!
Would Agile practices fit in The City of Calgary’s
application development environment? Would our
team be willing, able and authorized to make
decisions? Would we be able to adapt processes? We
have followed structured processes for so long, would
the corporate culture be able to adjust to this much
change?
Calgary, the fourth largest city in Canada, has over
one million citizens. This dynamic municipal
government has 28 Business Units, and employs
approximately 14,000 civil servants. As would be
expected, our organization’s culture is similar to other
government institutions and large, well established
corporations. We are characterized by:
 Constraining policies, procedures and
processes
 Rigorously defined roles and responsibilities
 Hierarchical authorization and reporting
structures
The four hundred IT professionals provide
technical services ranging from maintaining
infrastructure to application development and support.
The reality of the environment we work in is:
 Client engagement varies. Their participation
may be challenging to get and to keep past
requirements sign off
MichaelM@quadrus.com
 Communication and collaboration are
hampered when conforming to the hierarchical
structure
 Decision making is slow due to the top down
single direction flow of information
 Delays in approvals often result in
unauthorized work
 Authority to make work related decisions is
limited due to the hierarchical structure
 Team dynamics are focused on managing the
team as apposed to leading it
 Planning for the entire project is done up front
with a focus on task management
 Development teams (IT staff) are expected to
follow a traditional waterfall application
development process
 The complexity of our process can obscure the
end goals and diminish motivation of the
team’s effort
 Many Project Managers have 3 or more
projects at varying stages of development, so
they are continually switching context from
one project to another
 The development team is made up of union
and non-union employees whose roles and
responsibilities are carefully defined and
adhered to
The City of Calgary IT Business Unit is in the
midst of change, focusing on process and quality. A
business process reengineering of the application
development process resulted in the recommendation to
assess the viability of Agile within our environment.
2. Why me?
We were so excited to be selected for the Agile
Pilot, and then panic hit! Would this work in our
culture? Could we find a client who would work
differently with us? Could we really get a “single”
focused team? Did we really understand what
management was asking for? Did management truly
understand what they were asking for? Could there be
flexibility in the processes? What did we know about
Agile?
Investigation into Agile led us to believe that our
clients and IT could benefit in several ways, but the
confidence from initial research was wavering. Based
on our culture it might be too much. The impact to our
careers could be positive or negative! We felt there was
no middle ground. What had we gotten ourselves into
and where would we start?
3. This will never work
We settled down and focused on getting
everything to launch the Agile Pilot in place; we
needed a mentor, a workspace, a client, a project, and a
team. We were now in our comfort zone of project
planning.
In order to try Agile and make any sort of
recommendation on viability at The City of Calgary we
needed to try it with a team of certain size and makeup.
There were several meetings with Managers to present
our needs and goals for resources.
Client selection was based on several factors;
support of the general concept and objective of the
Agile Pilot, with a clear understanding and agreement
to a 100% daily availability. While the client as a team
member would be knowledgeable of business
requirements, they would also be empowered and
authorized to make decisions. The project had to have
the potential to demonstrate a solid set of Agile
components within the constraint of six months.
The team, now considered to include client and IT
staff, was brought together for the first time in a
training class to meet and familiarize everyone with the
approach, tools, and techniques of Agile development.
More importantly, it was an opportunity to allow for
team building. There was a mixture of excitement and
confusion; excitement to be part of the team and
confusion over roles and responsibilities. Developers
came from different sections within Application
Development and Application Support. Many didn’t
even know each other.
One class exercise that particularly resonated with
the team was the development of a list of team values.
Rather than simply listing values, the team was
challenged to describe how they would demonstrate
these values to one another.
Physical space was an enormous problem, but we
were able to get a temporary location for the team to be
co-located, including a dedicated meeting room.
IT staff was curious. Some had past experience
and wished to share, others had questions and concerns.
There was primarily skepticism and doubt about Agile.
Our culture is apprehensive and resistant to change and
this was viewed as big change. We heard the full
spectrum of comments ranging from:
 A large corporation in Calgary had tried and
backed out of Agile, so surely it would fail
here too
 Some Agile components such as Test Driven
Development (TDD) are really hard and
would be impossible to incorporate here
 We didn’t know enough about Agile and TDD
to make it work
 Agile is only a development practice and
applies to design, develop, test phases, but it
is not relevant to project management
 Agile will never fit with our current processes
 I hope this doesn’t work
 TDD promotes high quality (the only word of
encouragement we received)
We realized that we needed to communicate!
Communicate! Communicate!
A great deal of effort was spent with IT staff that
would be directly or indirectly impacted by the Agile
Pilot. Communication included the goals of the Agile
Pilot, an overview of Agile, and the candidate Agile
process. We knew at this point we were not going to
change anyone’s mind, so our goal was to dispel fear
and rumors.
Service groups, including Enterprise Application
Integration, Database and Server Administration,
required additional working sessions. This was
necessary since these groups were used to scheduling
services for projects that were developing systems
using a traditional approach. It was important to
establish a potential working relationship with these
groups since development and deployment of each
release depended upon it. Additionally, the Agile Pilot
needed to help them understand how they could be
structured to help future projects adopting Agile.
4. This is really going to happen
Somehow everything fell into place; we had a
mentor, a willing client, a project, a team, and a really
great workspace. The team had been sent on training
and was mostly familiar with the terminology and
approach for what we were going to do. The gap
between theory and practice had to be filled with the
confidence that only experience can bring.
 The greatest uncertainty for the development team
was over TDD. Could they make it work? Would
everyone adopt it?
The project kickoff was the first opportunity for
the team to discuss the actual product they were going
to build together. The meeting itself was not
remarkable except that it was the first opportunity to
discuss the working relationship we would have.
Our culture had been one of command and control,
so the development team would typically expect that
they would be largely spectators to the kick off. In fact,
most of the requirements, analysis and planning would
have been completed before they were involved.
Additionally, given our hierarchical nature, most of this
information would have been disseminated in a top
down fashion.
The development team sensed they were in for
something different since they were being involved so
early. They were not used to this sort of early and open
engagement with their client and so they were quiet and
reserved in the initial stages of the project. As time
went on the team became vocal and active participants
in project discussions. They then became vocal within
the organization and people started to recognize that
something very unique was going on with this team.
We set a goal at the kickoff to complete our
envisioning activities and to start development within a
week and a half. This meant we needed a product
backlog, development tools and environments set to go
in 8 working days. This goal was a bit of a shock to
some at first but we accomplished it with little trouble.
We started the following day with a user story
workshop. The team approached it with hesitant
optimism, believing they could do it, just not knowing
how. This was a clumsy and awkward affair at first.
The team was still used to reading requirements; they
had not been actively engaged in an activity like this
before. The team had just been formed, so the lines of
communication and trust were just being established.
The mentor had each person write at least one user
story in that first meeting to ensure that everyone
understood they need not be spectators.
To add to the awkwardness of this first meeting,
several external service group representatives attended
as they were expecting the typical project requirements
kickoff. We then had a large and confused audience for
the team as they tried to write their first set of user
stories.
For the client, there was uncertainty over how to
communicate their requirements. How do we tell them
what we need without writing down all the details?
This uncertainty was tempered with the persistence and
influence of the mentor who assured all that we were
on the right track.
By the second meeting, the team had gotten past
the initial hesitation and was easily and naturally
creating user stories. They had developed sufficient
user stories to talk about iteration and release planning.
Estimation, prioritization and planning followed.
The mentor was with us full time through the
envisioning period helping to run the meetings and get
things moving on the project. He promised to step out
of the way as soon as we could walk.
The introduction of Agile relied on the mentorship
of the team and team leadership. This intense
mentoring continued through the first three iterations.
5. The world didn’t end
The first iteration started and things got off to a
reasonable start. This was the first time for daily stand
up meetings, managing tasks and applying TDD.
Nothing really broke, but then again we were not
blazing through developing our user stories either. In
fact, as the first few iterations came to a close, our pace
was looking glacial.
Iteration planning meetings and task
decomposition was slow and laborious at first.
Historically the team had not been involved in these
activities. Typically the tasks and assignments would
be determined by someone else. This was a real
cultural adjustment for the developers.
It was through this period that the team and the
client had to hold on to their belief that this could
work. Our mentor assured us that this was perfectly
normal but it was hard to see that at this point. As we
looked at the list of user stories that still needed to be
built it seemed like we were facing a monster.
The major accomplishment through this period
was the application of TDD. The team worked in pairs
making a conscious effort to work in this new way. By
the end of the third iteration everyone saw the value
and believed they were as fast at developing with TDD
as without.
In the end, this was a period characterized by a
small degree of confidence gained. The team had
proven they could work this way. Whether it was the
best way to work had not yet been clearly shown.
6. This might work
On the fourth iteration things started to turn
around. Based on prior progress, the team had chosen a
small number of user stories. They had these completed
in just half the iteration. Now they were going back to
the client looking for more.
Not only was the team’s productivity improving
but the process itself was evolving. The team was
actively reviewing the way they worked at the iteration
review and implementing changes that made a real
impact. Changes included improvements to the wall
where tasks were being tracked, creating testable tasks
and managing project risks on the wall with the release
and iteration.
This was a striking change to the culture where the
way people worked and their roles had been rigorously
defined. The team was stepping out of this mold and
challenging their expectations of themselves and their
work processes.
The mentor started to step back at this point to
allow the team the opportunity to gain confidence. As
our mentor explained, if he did not step back and allow
the team to start solving their own problems, they
would develop an unhealthy reliance on him. The goal
was to have the team realize that they could make the
decisions themselves.
The team at this point was running on its own. Not
without feedback though, as the mentor was still
available and in touch but at arm’s length.
The mood had changed. The process was not only
okay but actually worked. Work was being
accomplished and the quality of the product was very
high.
7. It really works
We continually checked in with the team. We
wanted to know their thoughts on working with Agile.
Everyone was eager and thrilled about the new work
environment. A noticeable cultural difference was
observed.
Historically there was a sense of distance from the
project from the client’s perspective. Our highly
structured culture had created barriers to visibility,
transparency and communication with our clients. The
client’s observations of the Agile Pilot were:
 They felt they owned the product and were
part of the team whereas before they were an
observer
 They saw working software early and often
whereas before they did not see the product
until much later in development
 They felt they had the ability to direct the
project plan whereas before the Project
Manager made all the planning decisions
 Progress reporting was openly and visually
displayed whereas before reporting was
summarized and sometimes unclear
In the end our clients reflected on the experience and
stated this showed that: “IT really wants to listen!”
The development team also observed the dramatic
cultural change that Agile presented. Before the Agile
Pilot, projects were run with a top down command and
control approach. Developers were assigned their work
and were generally not in regular communication with
the client. The team’s observations of the Agile Pilot
were:
 They had control over what they did and how
they did it whereas before tasks were created
and assigned by someone else
 They knew their client through face to face
conversations whereas before they didn’t feel
as comfortable speaking with them
 Given control of their work they were better
able to focus on their tasks whereas before
there was less of a sense of control when work
was assigned
 There was a stronger focus on quality and
testing with the application of Test Driven
Development and a Test Analyst on the team
whereas before testing and quality was
inconsistent
We continually heard from the developers: “I want
to keep working this way”. At time of writing there is
sense of disbelief among them that they will be able to.
They recognize the serious challenge that Agile
presents to the predominant culture here. The Senior
Management at The City of Calgary has continually
demonstrated their support, openness and willingness
to change throughout the Agile Pilot, so we are
confident that the team will be able to continue
working with Agile.
8. Everyone is excited!
There was keen interest on how the project was
going. The buzz that Agile could really work at The
City of Calgary was out there. Individuals wanted to
know what was making this such a fun project; the
team’s enthusiasm was spreading!
The excitement and enthusiasm of the Agile Pilot
team was noticed throughout all of IT!
9. Where do we go from here?
 Today the culture at The City of Calgary is still The Agile Pilot was successful in demonstrating
command and control, structured and hierarchical. how Agile can work at The City of Calgary. It has
However, the culture and dynamics on this team has proven that a structured organization can incorporate
been noticed. Many individuals and groups are now Agile, and in addition, IT has a higher degree of
starting to consider the impact of Agile. There will be confidence in Agile.
many challenges to overcome. We have only scratched Executive sponsorship and endorsement of the
the surface of change. Agile Pilot was critical to the project’s success. The
How do we facilitate change within the client Management team demonstrated support,
community? Due to the Agile Pilot, we now have a encouragement, and openness to change. Service
respected champion! We need to draw on this to groups were very willing to collaborate and adapt.
promote Agile. We now need to implement Agile as an accepted
Self-organizing teams will be a challenge for some and valued practice at The City of Calgary.
Project Managers and team members, each having to
let go of some apprehension and uncertainty. Roles and
responsibilities will change.
The roles and responsibilities for team members,
particularly for the Project Manager and Technical
Lead, need clarification. The Technical Lead was a
new role within the organization and some of the
responsibilities were traditionally the Project
Manager’s. This led to some confusion on who should
take ownership of certain activities. For instance, there
was an issue with the quality of testing and visibility
into progress on testing. The Technical Lead and
Project Manager were both unsure of who should be
addressing this.
Functional testing was executed within the
iteration and the Test Analyst was located with the
team. The test effort, however, was largely a black box
to the development team and leadership. There was
inadequate visibility into what was being tested and
how. This led to uncertainty on the progress of testing
and quality of the product. Test Analyst responsibilities
will be defined by another initiative underway: the
creation of a focused test centre.
The reporting and tracking of progress for the
Agile Pilot was highly effective for the development
team and client. The visibility into status and progress
for Senior Managers, however, was not adequately
addressed. Management reporting will need to be
defined within the broader rollout for Agile at The City
of Calgary.
The Program Management Office (PMO) was
developing project management processes and best
practices while the Agile Pilot was running. Initially the
PMO did not think there was project management in
Agile. Over time, they recognized the need to engage
with the team to create convergent processes and
approaches to managing Agile projects.

10. Conclusion