White Paper

FitNesse as Acceptance test tool in the Agile

environment

Author:

Mukund Kagatikar
Designation: Senior Software Quality Engineer
Email: mukund.kagatikar@sabre-holdings.com

Sabre Travel Technologies Pvt Ltd

Navigator Building, Level 2
Whitefield Road
ITP, Bangalore - 66
Abstract:

This paper is to provide insight to FitNesse as testing tool for agile development teams.
FitNesse being one of the most powerful open-source tool but yet less explored by
software community. It’s web server so, easily accessible and since it is also Wiki can
be editable by everyone with ease. It will first cite the explanation for “FitNesse” as
testing tool which will comprehend the need of acceptance test, help in adopting the
TDD. Next it will suggest that FitNesse as a viable testing tool for use in agile
environment with the necessity of rapid release schedule. Then it will provide a
number of benefits to demonstrate the simplicity and power of FitNesse.

Audience:

Quality Engineer interested in exploring the new tools

Business Analyst for creating the effective acceptance test

Project Manager looking for open source tool

The Development team where they can develop product with TDD approach

Area of Application:

For DB & Web services testing

Best suited for command based application

Benefits:

For writing better acceptance tests with less effort by BA

Customizable as per project need as it’s open source

Developer can write the unit tests

Continuous Integration possible through CruiseControl

We can integrate it with Bugzilla, PMT and QC

Issues and Challenges:

Establishing traceability will be challenge [needs to depend on other tool]

Integrate with DB for test case back-up.

GUI Testing is not as simple as record & playback.
 Table of Contents
ABSTRACT: ................................................................................................... 2
1.0 INTRODUCTION: .................................................................................... 4
2.0 CONTENT ............................................................................................... 4
3.0 CONCLUSION ......................................................................................... 8
4.0 DEFINITIONS, ABBREVIATION AND ACRONYMS .................................... 9
5.0 REFERENCES .......................................................................................... 9
6.0 ACKNOWLEDGEMENTS ........................................................................... 9
1.0 Introduction:

There has been many tools used for acceptance testing in the software world and each of these
has their own features and drawbacks. One must diligently choose the tools with their need,
business model adopted and financial goals. FitNesse is one of the acceptance test tool which
suites to .Net and Java environment, easy to use by anyone involved in the project and most
suitable to agile environment. FitNesse has been in use for long time now but its strength and
capabilities yet to be understood by the wide software community. We always face the
challenge of translating the acceptance test provided by the customer to be understood and
verified by testing team. The best way for such challenges will be that a user be able to write
the tests in the automated environment. This will satisfy both the customer on the
functionality implemented and the testing team on achieving the regression. The attempt is
made here to provide a brief about the tool in terms of its usage area and how applicable to
agile rather than its technical specification and customization of the tool.

2.0 Content

What FitNesse can provide as Acceptance test tool?

“Developers write unit tests to determine if their code is doing things right. Customers write
acceptance tests to determine if the system is doing the right things.”

The term acceptance testing itself is strongly related to the agile software development
method, the customer requirements are gathered in the form of user stories. Acceptance tests,
also called customer tests, are tests owned and defined by the customer to verify that a story's
implementation is complete and correct.

Acceptance tests represent the customer’s interests. The acceptance tests give the customer
confidence that the application has the required features and that they behave correctly. In
theory when all the acceptance tests pass the project is done.

Advantage of FIT and FitNesse for acceptance testing is that they use an approach that enables
users and user representatives to write acceptance tests via a spreadsheet-like approach. This
feature decouples the creation of acceptance test suites from dependence on developers. It
also enables a mixture of documentation and testing that can result in testing documents that
also provide considerable insight into how systems should work. Such test suites can really
help when the time comes to produce formal documentation and training materials.
How FitNesse fits into Agile?

User stories are basically “3Cs – written on a card, serve as promise of conversation and serve a
confirmation” for developers and customers to work on a problem together. The outputs of
that conversation are detailed understand of the story, estimates of the amount of effort each
task will take, intermediate candidate solutions, and ultimately acceptance tests.

General Acceptance testing proceeds is:

1) The customer writes stories.
2) The development team and the customer have conversations about the story to flesh
out the details and make sure there is mutual understanding.
3) If it is not clear how an acceptance test could be written because there is not enough
to test it against yet, the developer does some exploration to understand the story
better.
4) When the exploration is done, the developer writes a “first cut” at one or more
acceptance tests for the story and validates it with the customer.
5) Once the customer and the developer have agreed on the "first cut" acceptance test(s),
he hands them over to business people (QA people on our project) to write more tests
to explore all boundary conditions, etc.

The above is most general situation prevails in the agile environment. The involvement of
customer at each stage is good part of application development but how much real time he/she
can be able to spend with Dev team after the stories are handed over? Will it be easy for him to
write his acceptance test rather than reviewing of the tests written by development team? To
answer these questions FitNesse comes as handy where user can write the acceptance tests in
excel based template and save time on effective communication on business related queries
rather than test itself.

Some of the reasons that agile web development teams would want to adopt a rapid release
schedule
• Super-short releases allow the marketing department to obtain rapid customer
feedback and to respond to that feedback quickly. They can test modifications
to existing products and release new products sooner and more often. The agile
concept of rapid feedback works well for the marketing team, too!
 • Frequently updated web sites perform better in organic search engine results.
This enables the business to attract customers to its web site without paying
for sponsored links.
• Businesses that support diverse customer niches demand more agility from
their products.

Framework for Integrated Test (FIT) is an open source framework for user acceptance testing,
and a tool for enhancing the communication and collaboration between analysts and
developers. FIT lets analysts write acceptance tests using simple HTML tables or in
spreadsheet. Developers write fixtures to link the test cases with the actual system itself. FIT
compares these test cases; written using HTML tables or in spreadsheet, with actual values,
returned by the system using fixtures, and highlights the results with colors and annotations.
These tests can be stored and run as regression suite to achieve the rapid release schedule
keeping all the acceptance and functional tests in one tool.

All about FitNesse

The translation of idea to concept in software development is an area plagued with tension,
miscommunication and failure. Every software development process has its own take on how to
solve the issue and streamline the translation of requirements into tested code. In the real
world, applications keep growing in size and complexity, and change frequently; thus, the
necessity for continuous testing constantly increases. But although the idea seems simple, in
practicality it requires discipline and application.

FitNesse is a tool/approach to this age old problem that looks at it from a new and surprisingly
effective point of view. By treating easy-to-create tests as the actual requirements, FitNesse
allows non-technical Business Analysts to specify their needs in the form of executable
requirements. These requirements then become a living part of the software ecosystem instead
of a set of stale documents that grow further out of sync with the code base as time goes by.

FitNesse is a tool created by Robert Martin, Micah Martin, and Michael Feathers. It is a wiki-
based test execution engine that allows users to write, edit, and run FIT tests. Together FIT &
FitNesse enable developers and users to express acceptance tests in a tabular format and
execute them from a simple web interface.
Agile programming prescribes automated acceptance testing so that tests can be run often,
while facilitating regression testing at a low cost.
Just two steps are required to automate user acceptance tests using FIT:
1) Express a test case in the form of a FIT table
2) Write the code in Java, called a fixture, that bridges the test case and system under
test
Tables are the crux of FitNesse. Acceptance tests in FitNesse are defined in the tabular format.
This deterministic tabular format is understood by the FIT library. These tests or tables are
defined in terms of input and expected output data. In order to read those tables and test your
application under test with the data used in these tables, you need to write Fixture Code. You
can imagine that Fixture Code is a bridge between the table and application under test. It
understands the language of table and uses this information to exercise the functionality of
application.

Figure1: Architecture
How FitNesse helped in our project

Our application was basically to write the query on mainframe system (command prompt) and
check the output we received to be in some particular format and then transact this
information to the end-user over the web. To achieve this task we had around 20 story cards
covering four major functionality. Each story card would have average 35 test cases. Since the
query on each time for every test cases yield the different result, but with some fixed format.
So, in this scenario BA wrote the acceptance tests to make the team understand on what the
end result should be and the working of the functionality. QA team developed the test cases in
FitNesse using the Regular expression to match the end result received with the expected one.
To parse the test cases to mainframe system and executed at its end and collect the
information displayed on the screen and send back to the FitNesse all this was achieved by
writing the Fixtures in Java. Here in this activity FitNesse proved to be very handy in terms of
customization of the tool w.r.t our need. This clearly helped us in achieving the functionality
testing completed along with satisfying the acceptance tests and also we integrated this with
CruiseControl to achieve the nightly regression.
In agile practice environment we were able to deliver the functionality every iteration and
execute the same test in the upcoming iteration as part of the regression. The results
displayed at the FitNesse are as Green for pass, Red for Fail and Yellow for Exceptions this,
clearly helped in concentrating and doing the troubleshoot for those tests which are either
failed or gave the exceptions.

3.0 Conclusion

As discussed, FitNesse can prove very valuable and efficient tool in the rapid development
environment where BA, PM, Dev team & Testing resource all work very closely on day to-day
basis. Although we can adopt it in all type of Dev lifecycle model but we have seen and read
about it using widely in XP kind of models. The advantage it provides over other tools is it is
lightweight, open-source framework for enhancing collaboration in software development,
enables customers, testers, and programmers to learn what their software should do and
compares customers' expectations to actual results.
Hence, it is ideal for the XP development environment making all the stake-holders to be
understood, implemented and tested for the same business needs. Since it is very easily
customizable it is difficult to ascertain on its limitation and gives every team to explore as per
their need by developing the fixtures which can help them in achieving their goals.
4.0 Definitions, Abbreviation and Acronyms

Acronym Description
BA Business Analyst
PMT Project Management Tool
QC Quality Control/Quality Centre
DB Database
Dev Development
PM Project Manager
XP Extreme Programming
FIT Framework for Integrated Testing
GUI Graphical User Interface
HTML HyperText Markup Language
QA Quality Analyst/Assurance

5.0 References

Item Description
Miller, R. W. and Collins, C. T., Acceptance testing. XP Universe, 2001;
1
http://www.xpuniverse.com/2001/pdfs/Testing05.pdf
Beck, K. and Andres, C., Extreme Programming Explained: Embrace Change (2nd
2
Edition). Addison-Wesley Professional, 2004
3 Fit documentation URL http://fit.c2.com/wiki.cgi?IntroductionToFit
4 FitNesse user's guide URL http://fitnesse.org/FitNesse.UserGuide
5 Acceptance testing URL http://www.cs.helsinki.fi/u/jaarnial/jaarnial-testing.pdf
6 http://en.wikipedia.org/wiki/FitNesse

6.0 Acknowledgements

Name Description
Sreekanth Tadipatri Agile Mentor, Sabre India
Sridhar Natarajan PM, Travelocity
Sebastian Batog Lead, Sabre (eHotels Group)
Rafal Markowicz QA, Sabre