Overview on Extreme Programming

Jeremy Choi
Nestea@cs.ust.hk
March 20th , 2002

Introduction

The focus of Software Engineering is mostly on resolving technical issues on software

designs. With tools such as UML deve lopers can produce detail descriptions on all parts
of the end production in a technical point of view. However, little attention has been
paid on formalizing the interaction between developers and end users and on assuring
that design produced by developers are full understood by customers and satisfactory to
the customers. Most importantly, UML is not dynamic. A change in the requirement of
end-users usually results in huge effort in redesigning the structure of the end product.

Along came Extreme programming (XP), a lightweight development process created

by Kent Back. Unlike UML, which is used to pinpoint the details of implementation,
Extreme programming is a methodology that copes with the ever-changing demands of
end-users.

Extreme Programming is a discipline of software development based on 4 principle

values: simplicity, communication, feedback, and courage. It works by bringing the
whole team together in the presence of simple practices, with enough feedback to
enable the team to see where they are and to tune the practices to their unique situation.
Based on these 4 principles, 12 practices are implemented:

l Planning Game
l Testing
l Refactoring
l Simple Design
l Collective Ownership
l Metaphor
l Continuous Integration
l On-Site Customer
l Pair Programming
l Small Releases
 l Coding Standards
l 40-hour work week

This paper attempts to first give an overview of the 4 principles and then explain the
reasons behind the 12 practices

1. The 4 principles

1.1 Communication

XP encourages extreme communication. If the customer needs to see the program

in action to fully formulate requirements, put the customer on the development
team and release working versions every 2 to 4 weeks, developing requirements
along the way. If the programmers need clarification on requirements, the
customer is a member of the team -- lean back in your chair and ask the customer.

Programmers work in pairs: two people to every one machine. Pair members
rotate regularly. All code is owned by the team, not by individuals. This promotes
communication of technical knowledge throughout the team. When technical
challenges arise, the team is more able to address the problem.

Pair programming also is excellent in matching up programmers of differing

abilities. Less experienced members are constantly mentored, and the risk of less
experienced code being added to the application is minimized.

1.2 Feedback

The faster change is identified, the faster it can be dealt with. XP strives for the
fastest feedback possible in every aspect of the project.

Unit Tests are written for most every piece of production code. Unit tests must run
at 100% before any code is checked in. When production code needs to be changed,
side-effect problems are identified. After new code is checked in, it's immediately
integrated with the latest changes from the rest of the team and again, all unit tests
 must be made to run at 100%.

Acceptance Tests are written with the customer to verify the application is doing
what the customer needs it to do.

Pair Programming provides constant code reviews. No more dreary code review
meetings -- put two sets of eyes on the code as it's written. Collective Ownership of
the code by all members of the team helps ensure even more eyes will see the code,
increasing the amount of code review performed.

1.3 Simplicity

The customer is the ultimate driving force in XP. Do what the customer needs, as
simply as possible, and nothing else. Take on a mindset to reduce complexity from
the beginning.

All code should be refactored as often as possible. Refactoring is a process of

improving a code's structure without changing its functionality. Refactoring
produces highly decoupled objects which makes them easy to test, easy to use,
more flexible, and therefore, more changeable.

1.4 Courage

No more fragile code. With smooth communication, quick feedback and simple
code, programmers have the support they need to dive into changes when they
come ... and they will come.

2. The 12 practices

2.1. Planning Game

Each iteration begins with the Planning Game, an informal process that sets the
agenda for the iteration. The game starts with the customer defining requirements,
or 'user stories'. Technical members work with the customer to normalize these
stories into manageable chunks and break them down into specific tasks, as well
as introduc ing technical tasks needed to support the customer's requests (e.g.
upgrading development software, automating builds, etc). Then the developers
place ideal time estimates on the stories/tasks. Based on the time estimates given
combined with the team's velocity (average number of stories/tasks the team has
completed in past iterations), the user stories are prioritized by the customer.
Programmers then sign up for tasks and the development section of the iteration
begins.

Case Study: Chrysler Gemstone Project

Ken Back was invited to Chrysler Gemstone Project for performance tuning.
The first step in planning, done by the business people or the end- users, was a
complete low resolution pass through the system as use cases, which resulted
in 135 4x6 cards of description. At the same time, developers tried to build
prototypes of all the parts of the system they thought would be hard.

Next step was to convert these use cases from the end-users and the
experience gained from building prototypes into a schedule.

First the user sorted the stories by priority, into three piles:

1. These have to be done if the system is to meet its business goals

2. These have a measurable impact on the system's business goals
3. These would make the user happy, but without explicit jus tification

The priority was written in one corner of the card.

Then the developers sorted them by risk into three piles:

1. We know exactly how to do this

2. We think we know how to do this
3. We have no idea what this means or how to do this

The risk of each task was written in another corner of the card.

These sorting activities were a key to the whole rest of development. It was in
seeing all the cards laid out on a great big table that everyone understood the
scope of the system. It was in discussing what the cards meant that lots of
alignment began, developers to customer and developer to developer.

Finally determining how many weeks of ideal developer effort each task
 would take produces estimation of schedule. Without prior user-cases and
prototypes, these numbers would have been hard to come by. With the
exploration, estimates can be produced with high confidence.

The schedule turned out to be too long, so the user fitted the tasks to the
available time by setting aside low priority cards.

2.2. Testing

Testing is a crucial practice on an XP project. XP succeeds by making a project

resilient. Resilience means accurate and frequent feedback. Testing provides this.

In XP, there are two categories of tests: unit tests and acceptance tests.
A unit test is a piece of code that exercises one aspect of a piece of production
code. Unit tests should be small and fast. For example, given an object that does
monetary conversions, a unit test would go through the following steps:

• 1) Create an instance of the conversion object

• 2) Pass in an amount of a known currency
• 3) Request a conversion to another currency
• 4) Compare the output from the object with a hardcoded expected value

This test is now a sentry assigned to monitor this piece of functionality. At the
time it's written, it may seem too small a thing to worry about. However, the
amount of code in even a small project can grow very quickly. Without an
automatic way to check every important detail in the code, soon the code will not
be exercised regularly. Growing a collection of unit tests from day one becomes
an extremely powerful tool later on in the project.

All unit tests must pass 100% before programmers can check-in any new code.
After check- in, integration is done on a separate machine with all of the latest
changes and again all unit tests must pass. Any problems should be fixed
immediately.

To help ensure that a suitable suite of unit tests are built throughout development,
a coding practice known as "test- first" programming is implemented. Before
adding a new piece of functionality, a unit test exercising the non-existent code is
written. Then, enough production code is written to allow the test to compile, but
not pass. The unit test is executed to ensure that it really does fail. If it passes at
this point, there's probably a bug in the unit test. Then production code is written
until the test passes. This style helps ensure a consistent test suite is developed
alongside the production code.
Having a consistent test suite eliminates fragile code. In the later stages of a
project, programmers are usually hesitant to revisit previously written code
because it's either buggy and/or messy or because it's solid -- which pretty much
runs the gamut. The reason is the same regardless of the quality of the code -- fear
of making a mistake and introducing new problems or breaking things that have
been working.

The unit tests remove this fear. Programmers can make bold changes and see the
cost of the change immediately by re-running the test suite.
Unit testing also promotes good design. It's easy to create highly coupled code
without unit tests because for every one production object often times there's only
one other production object using it, at least when the code is first written. Unit
tests ensure that everything tested will have 2 clients from inception -- the
production code that uses it and the unit test that tests it. Requiring code to be
responsive to multiple clients forces less coupling of objects which promotes long
term flexibility. This flexibility is a crucial contributor to resilience.
As an added bonus, unit tests make for great usage documentation. Separate
example code does not need to be written. If unit tests are written thoroughly,
example code exists for free. This can greatly reduce if not eliminate the amount
of internal documentation required.

Acceptance tests are distinguished from unit tests in a couple of ways. First, they
should test the system end-to-end. Second, the customer is involved in creating
the acceptance tests. Ideally, a framework should be built that allows the customer
the full ability to add new tests for the system without any technical intervention.
For example, an XML file could store input/output data for each test, and a
framework coded to load each case from the XML file and execute it. Unit tests
ensure each technical detail is working properly. Acceptance tests ensure each
customer requirement is working properly.

The testing practices are a main support for eliminating up-front design. It doesn't
have to be done right the first time because it's cheaper to change it on the fly, it's
resilient. Refactoring also plays an important role in creating resilience.
2.3. Refactoring

Refactoring is the process of improving the design of code without changing the
functionality. The code should be clean and readable. Any duplication should be
consolidated.

Refactorings should be done on an ongoing basis throughout development of the

code. As soon as structure improvements make themselves known, they should be
done. In many environments this would be risky, as the tendency for developers to
introduce side-effect bugs while making changes is high. But the unit tests
provide the quick feedback required to steer a programmer back on track during a
refactoring. Refactorings also tend to get postponed because no new functionality
is being created. However, allowing poorly structured code to exist in a project is
a risk that accumulates over weeks of development. Refactoring is best practiced
on a regular basis.

2.4. Simple Design

To help ensure frequent feedback, it's important that the application's design be
kept simple and kept to providing business value. While there will always be tasks
that are primarily technical and necessary to support providing business value, it's
important these tasks be kept as simple as possible.

The main reason for this is to eliminate unnecessary development time. It's easy
for the technical members of a team to overdo aspects of the project because we
might need it. Here we come to a couple of XP's acronyms: YAGNI and
DTSTTCPW.

YAGNI stands for "You Aren't Going To Need It." Too often, the team attempts to
build in functionality they might need in the future. This is a reaction from the
anticipation style of doing things. If future changes are expensive, make sure you
do it right the first time and think of everything the first time. YAGNI takes
advantage of the resilience created by testing and refactoring. Since the provided
resilience will allow for inexpensive future changes, we only need to build for
today.
 DTSTTCPW stands for "Do The Simplest Thing That Could Possibly Work."
One note here, Simple should not imply Poor Quality or even the Fastest, though
speed is the main issue with simple design. Simple should include as much
elegance as possible without getting burdensome. But the tendency is still
towards simplicity over elegance. Here's why.

Thorough refactoring practices will cover any initial problems. The fact is, it's
hard to get it right the first time. Simple design says, don't try to get it right the
first time. Write the unit tests and get those passing in a simple way. If the code
just written turns out not to be too important to the overall project, then it's okay if
it's less than ideal, as long as it works (passes testing). If the code is crucial, it's
guaranteed to be revisited many more times. The process of refactoring allows the
less than ideal but simple design to have its kinks worked out, and no one wasted
a lot of time trying to get it designed right the first time.

2.5. Collective Ownership

Collective ownership refers to the code. Many development efforts assign

specific portions of the application to individuals. Sometimes programmers and
managers like this because it can be easier to measure individual contributions to
the project. However, the potential for bottlenecks with individual ownership is
higher than with collective ownership.

When only specific individuals are familiar with certain portions of the codebase,
others on the team can be held up waiting for changes to be made. In addition, the
amount of work needed for different sections of a project can change during the
course of development. If individual ownership of each section is promoted too
much, work loads can become lopsided.

Collective ownership allows anyone on the team at any time to work with any
piece of code. If a pair working with object A needs object B to change, that pair
can go immediately make the change in object B to accommodate the needs of
object A.

The practice of quickly dipping into a related piece of code to make a quick
change can be dangerous, because quick changes can often times create
side-effect bugs. In an XP project, however, every piece of code is developed
test- first, ensuring each piece of functionality is unit tested. If the quick edit fixed
one thing but broke several others, the test suite will be run shortly after the edit
was made and report the errors immediately to the programmer who just made the
edit. If a bug slips through at this level, acceptance testing provides a second net
for catching the problem.

Pairing will be more difficult if the code any pair is working on is 'owned' by only
one member. The non-owner of the pair may not feel comfortable suggesting or
making changes to code that they don't have rights to.

Pair programming and collective ownership does not snuff out individual
contributions (good or bad). Bonus plans and other types of performance rewards
may have to be changed.

XP aims to avoid hang- ups caused by individual ownership by focusing

ownership of the team on the whole. Take pride in the whole, not in the individual
contributions to the whole. The project itself must succeed as a whole to be
effective, and that should be the goal of any development effort.

One concern about collective ownership is ensuring each programmer is a

positive contributor to the team and not riding on others' efforts. An XP team is
very people centric, relying on a lot of interaction (especially with pairing) and
verbal communication. It would be difficult for a no n-contributing programmer to
hide out for long in this environment. In fact, it would be easier to hide out in an
environment where individuals hole up in their cubes for weeks at a time while
they code their own thing.

Another concern with collective ownership is building expertise with the

application. There are limits to the familiarity an individual can have with the
codebase. Individual ownership helps solve this problem by limiting the amount
of code a programmer creates to promote expertise in certain areas.
Documentation of the code helps to cover the gaps as well. However,
documentation must be maintained and while individual ownership can help with
code expertise, the fact is it's still easy to out-code the limits of a programmer's
memory.

Collective ownership doesn't help in this regard because it's concerned about
overcoming individual ownership bottlenecks by spreading knowledge of the
 whole codebase to every member.

XP answers this problem of code familiarity by moving the responsibility from

the developers to the testing structure (unit tests and acceptance tests). Instead of
relying on the programmers or documentation to remember every aspect of the
code, XP makes the code responsible for maintaining its own accuracy with unit
tests.

2.6. Metaphor

The project metaphor is, more or less, an informal architecture of the system. The
metaphor describes the system in simple concepts. The concepts can be literal or
figurative, depending on the clarity of the actual system.

Perhaps the system resembles a post office, an assembly line, or an anthill. The
point is to pick something common enough that each member of the team can
understand. Since an XP project has little to no formal architecture documentation,
the metaphor can be a useful tool to aid in communication among team members,
especially between programmers and customer. A good metaphor can sometimes
inspire improvements to the application itself. But it's important to remember the
metaphor is simply a tool of communication and should be changed as the needs
of the project change.

2.7. Continuous Integration

Mixing the latest code from each programmer together can be a difficult process,
especially if this task is not done often. To stay resilient, after writing new code
that passes all tests locally, programmers must then integrate their changes with
the latest code base and ensure all the tests still pass. If not, fixes must be made
right away until all tests again pass.

It's recommended this task be done many times a day, usually on a dedicated
integration machine.
2.8. On-Site Customer

Every XP project has one or more individuals to fulfill the customer role on the
team. The customer's job is to write and prioritize stories (tasks from a user's
perspective that the software must perform), assist with acceptance testing and be
on hand to answer questions from the development team as they arise. It's typical
for the customer to continue on with regular duties of their job, but it's important
they physically office with the development team.

The project starts and ends with the customer. The customer determines what
must go into the product and declares the work successful through approved
acceptance tests which were authored by the customer.

Having the customer be an active member of the team provides for frequent and
cheap communication. There's a much smaller need for formal documentation
with direct communication. If the customer is physically separate from the team,
intermediate forms of communication (conference calls, email, documents)
become more valuable because it's harder for the developers to get the
information they need from the customer directly. Maintenance of this
information can become a project in and of itself and distract the team from
working on the actual product.

An involved customer can see the product as it evolves and can redirect
development efforts as requirements and the fulfillment of those requirements
come into focus. Changes are identified quickly and in smaller, more manageable
portions with fewer side-effects.

2.9. Pair Programming

All programming on an XP team is done in pairs, two people at one machine.

Each task from the Planning Game is owned by an individual. When the day starts,
pairs form up, each person either pairing to help someone else, or requesting help
on their own tasks. Pairs stay together until a logical break comes up. One takes a
turn 'driving' while the other actively participates verbally. As ideas flow between
the two, the keyboard can be swapped off as often as necessary to get the best
code on the screen. Pair assignments are fluid and change throughout the course
of a day.
The benefits of pairing are numerous. One thing pairing provides is constant code
review -- no line of code is written without two sets of eyes. This can reduce or
eliminate the need for code review meetings which can often be boring and
wasteful.

Pairing helps distribute knowledge of the code more evenly throughout the team.
This can eliminate personnel bottlenecks due to illness, vacations or team changes.
If only one team member understands how a crucial layer of the system functions,
progress can be severely derailed if that individual leaves for another job.

Pairing provides balancing of talent within the team. A project that assigns
portions to specific individuals can suffer from varying quality as the functions
written by the senior members work well, but are held back by the bugs and poor
design of the functions written by junior members. With the pairing process, no
junior level member has to write code alone. They can be constantly mentored
through the pairing process which will educate them on the job and build up the
quality of the code throughout.

Individual programming moves a project towards the average abilities of the team.
Pair programming moves a project towards the maximum abilities of the team.

Pairing can also be faster than individual programming. When writing code, one
rarely works in one mental level. Switching levels takes a bit of time. Having two
brains working on a problem allows thought to occur at more than one level
simultaneously. Two people can leapfrog each other. While one writes code, the
other can think at a higher level, organization of code, where to go next, tests that
still need to be written.

A pair of programmers also has reduced distractions. There's built- in

accountability to ensure the mind and mouse don't stray too far from the task at
hand. Writer's block (as well as Debugger's block) is less apt to steal time away
from two people.

Pairing is probably one of the most controversia l XP practices because of a

number of challenges facing it. Management may be a hard sell because they see
pairing as a waste of resources. Many of the benefits of pairing are long term and
hard to see up front. Other benefits, like increased development speed, may have
 to be experienced to be believed because it's easy to assume development is a
simple labor task when in fact there's a high degree of creativity and thought that
must go into it. The math that says, "Two can work twice as fast as one" simply
doesn't hold up for many development tasks.

While management may struggle with pairing, many programmers take issue
with the practice as well. Successful programmers depend on good tools and
sound structure. Introducing pairing, which requires sharing of tools and practices,
can create friction as differences rub up against one another.

Everything from tool sets to work environment can be up for debate. In addition,
many programmers believe that they do their best work alone, or don't enjoy the
mentoring aspect of pairing with a less experienced person.

2.10. Small Releases

Small releases are a key part of generating feedback and making a project
resilient.

An XP project is a series of iterations, each lasting 2 to 4 weeks. Each iteration

starts with the Planning Game, an activity that determines the tasks for the current
iteration, and ends with a 'finished' product: all tests pass and the product is as
functional as possible.

Each release may or may not be a production release, especially in the early days
of a project. Most projects simply aren't useful until a minimum feature set is
complete. But it's important to approach each release aiming to give the customer
the most business value possible. When technical items are carried over from
iteration to iteration, they can accumulate additional functionality that may not be
needed.

One project I worked on spent a good bit of time constructing 26 elaborate reports
for the system. Once the application got to a beta stage, it was put into a
production environment in 3 locations, and all 3 quickly made good use of the
product. 4 months into the beta, a problem with the reporting module was
detected during development. It was a major bug -- none of the reports could be
opened. It wasn't long before the development team wondered why none of the 3
 beta sites had come across this problem. It turned out that none of the beta sites
had even tried to use the reports. If they didn't need the reports, was it really
necessary to build them?

In another situation, a new data entry form was being added to an existing
application. The GUI for the form had been designed to be as easy to use as
possible. Implementing the design required use of some third party GUI tool that
was complex and time consuming to work with. Many additional weeks were
added to the timeline tweaking out the GUI. After it was complete, the new form
was still useless, however, because the data entered into the system through this
form had not been included into other processes required in the application.
Following the practice of small releases that are as close to functional as possible
would have delayed the more elaborate GUI features to focus on including the
new data elsewhere in the app. If an early iteration worked well enough for the
customer, the more elaborate GUI might have been easily scaled down to a less
efficient, but still quite usable form, allowing the customer to move on to more
important items.

As noted earlier, many times customers need to see a working product to help
crystalize requirements. Sometimes verbal agreements on design decisions don't
stand up in the face of actual implementation. It's important to keep a working
product in front of the team to generate precise feedback.
Small releases also help the technical members of the team give accurate
estimates. Any time estimate beyond 2 to 4 weeks tend to be very imprecise.

2.11. Coding Standards

Having a coding standard for a project is a commonly accepted practice in most

projects regardless of methodology. This practice is equally important within an
XP team, especially in light of Collective Ownership and Refactoring practices.

2.12. 40-Hour week

XP promotes a well rested team. Its founders do not believe in the extended
overtimes, which creates fatique and defeats concentration. Tired workers make
mistakes and their desire to work decreases.
3. Bibliography

• Beck, Kent. Extreme Programming Explained: Embrace Change, Addison-Wesley

• Fowler, Martin. “The New Methodology”. 12 Sept 2000

http://www.martinfowler.com/articles/newMethodology.html

• Beck, Kent; Fowler, Martin. Planning extreme programming, Addison- Wesley

• Extreme programming: a Gentle Introduction

http://www.extremeprogramming.org

• Xprogramming.com : http://www.xprogramming.com/