Sie sind auf Seite 1von 116

Acceptance Test Driven

Development
Naresh Jain
naresh@agilefaqs.com

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

1
Tutorial Schedule

• Part discussion, part exercise


• Starts light and fluffy
• You’ll get your hands dirty by the end
• Questions welcomed all the time

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

2
Welcome

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

3
Welcome

• Continuum

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

3
Welcome

• Continuum
• Hopes and Concerns

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

3
Getting Started Exercise

Warmup Scenarios

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

4
Getting Started Exercise

Warmup Scenarios

• Going out for Movie (THX sound and Digital


projection)

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

4
Getting Started Exercise

Warmup Scenarios

• Going out for Movie (THX sound and Digital


projection)
• Going out for meal (one veg.)

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

4
Getting Started Exercise

Warmup Scenarios

• Going out for Movie (THX sound and Digital


projection)
• Going out for meal (one veg.)
• Going shopping ($50)

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

4
Getting Started Exercise

Warmup Scenarios

• Going out for Movie (THX sound and Digital


projection)
• Going out for meal (one veg.)
• Going shopping ($50)
• You are a party of 5.
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

4
What are
Acceptance Tests?

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

5
What are Acceptance Tests?

Criteria for Completion

• Written by the “Customer”


• Given to developers

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

6
What are Acceptance Tests?

Communication
• Forces customers, testers, and developers to
work together
• creation
• implementation
• execution
• maintenance
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

7
What are Acceptance Tests?

Feedback

• A source of data
• A tool for project management

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

8
Data From Acceptance Tests
Total ATs Failing ATs Passing ATs

90

72

54

36

18

0
1 2 3 4 5 6 7 8 9 10
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

9
Acceptance Tests:
A Critical Piece of Agile

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

10
Acceptance Tests: A Critical Piece of Agile

Traditional Approach
1 May 1 Jul 1 Sep 1 Nov

Analysis

Design

Implementation
ERD
DFD
DD
ST

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

11
Acceptance Tests: A Critical Piece of Agile

The Agile Aproach


May 1 Jul 1 Sep 1 Nov 1

Analysis
Test

Implementation

Design

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

12
Acceptance Tests: A Critical Piece of Agile

The Agile Approach

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Acceptance
Criteria

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Acceptance
Criteria

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration

Acceptance
Criteria

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration
Automated
Acceptance Acceptance
Criteria Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration
Automated
Acceptance Acceptance
Criteria Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration
Automated
Acceptance Acceptance
Criteria Tests

Automated
Acceptance
Tests
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration
Automated
Acceptance Acceptance
Criteria Tests

Automated
Acceptance
Tests
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

The Agile Approach


Iteration
Automated
Acceptance Acceptance
Criteria Tests

Automated Acceptance
Acceptance Criteria
Tests
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

13
Acceptance Tests: A Critical Piece of Agile

Criteria for DONE

• Every story must have at least one


Acceptance Test
• A story is not DONE until it passes it’s
Acceptance Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

14
Acceptance Tests: A Critical Piece of Agile

Manual Acceptance Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

15
Acceptance Tests: A Critical Piece of Agile

Manual Acceptance Tests

Manual
Acceptance Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

15
Acceptance Tests: A Critical Piece of Agile

Manual Acceptance Tests

Manual
Acceptance Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

15
Acceptance Tests Are
Automated

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

16
Acceptance Tests: A Critical Piece of Agile

The Button
• How often would
you press it?
• When would you
press it?
• Who would press
it?
• Testers, Developers,
Managers, Customers,
Spectators, etc.
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

17
Acceptance Tests: A Critical Piece of Agile

Automated Tests Require:

• High level, yet, precise language


• Unambiguous detail
• “Examples”
• Interaction with the system

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

18
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

19
Criteria for DONE

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

19
Criteria for DONE
+

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

19
Criteria for DONE
+ Automated

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

19
Criteria for DONE
+ Automated

Executable Specification

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

19
Acceptance Tests: A Critical Piece of Agile

Executable Specification

• A new paradigm for testing


• Puts quality first
• Removes ambiguity from requirements

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

20
Who Writes Acceptance
Tests?

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

21
Who Writes Acceptance Tests?

The Customer
• Yeah right! Then who?
• The Customer Role
• Stake holder
• Business Analyst
• Quality Assurance
• Product Owner
• Developercopyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

22
Who Writes Acceptance Tests?

Tests Get Technical

• The “Customer” may need technical help to


write tests
• Developers and QAs are technical
• Pair test authoring

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

23
Who Writes Acceptance Tests?

Business Rules Get Fuzzy

• Sometimes developers need help


understanding tests
• Customers know business rules
• Pair test implementation

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

24
Who Writes Acceptance Tests?

Business Decisions are


Hard To Make

• Sometimes, the “customer” will be


challenged to make decisions
• The Man ($$$) can help
• Do you see a pattern of communication?

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

25
Exercise #1

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

26
Exercise #1

The Login Test


• Write a test plan, in plain text, for
the business rules of logging in.
• Web application
• User credentials are stored in
relational database
• Successful login redirects to
“Welcome” page
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

27
Writing Good
Acceptance Tests

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

28
Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

29
Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

29
Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page


1. Enter the username ‘wallace’

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

29
Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page


1. Enter the username ‘wallace’

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

29
Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page


1. Enter the username ‘wallace’

Build a Testable Environment First

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

29
Writing Good Acceptance Tests

BOC

Build
Operate
Check
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

30
Writing Good Acceptance Tests

Login Test Possibilities

1. Add some users to the system

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

31
Writing Good Acceptance Tests

Login Test Possibilities

1. Add some users to the system

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

31
Writing Good Acceptance Tests

Login Test Possibilities

1. Add some users to the system


3. Enter a value into the username field

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

31
Writing Good Acceptance Tests

Login Test Possibilities

1. Add some users to the system


3. Enter a value into the username field

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

31
Writing Good Acceptance Tests

Login Test Possibilities

1. Add some users to the system


3. Enter a value into the username field

Be Specific
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

31
Writing Good Acceptance Tests

Tests are Examples

• Use concrete examples


• Specify concrete behavior
• No ambiguity allowed

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

32
Writing Good Acceptance Tests

Login Test Possibilities


1. Insert into User table values (’wallace’,
‘ilikecheeze’)

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

33
Writing Good Acceptance Tests

Login Test Possibilities


1. Insert into User table values (’wallace’,
‘ilikecheeze’)

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

33
Writing Good Acceptance Tests

Login Test Possibilities


1. Insert into User table values (’wallace’,
‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

33
Writing Good Acceptance Tests

Login Test Possibilities


1. Insert into User table values (’wallace’,
‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

33
Writing Good Acceptance Tests

Login Test Possibilities


1. Insert into User table values (’wallace’,
‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp

Avoid Implementation Details

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

33
Writing Good Acceptance Tests

Avoid Implementation Details

Tests UI

System
Database
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

34
Writing Good Acceptance Tests

Login Test: Possible Solution


• Add user to system: (’wallace’, ‘ilikecheeze’)
• Process login with username ‘wallace’ and
password ‘blah’
• Check login failed
• Process login with username ‘wallace’ and
password ‘ilikecheeze’
• Check login succeeded
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

35
Tools

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

36
Tools

Commercial Tools
• WinRunner TestPartner EggPlant
• Silk QTP TestComplete
• RFT Squish WindowTester

Are not suitable for Acceptance Testing in


an Agile environment

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

37
Tools

Open Source Options


• FIT Sahi Frankenstein
• FitNesse Watir SharpRobo
• Selenium Abbot WET

Among the few tools that support Test


Driven Development
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

38
Tools

FIT

• Framework for Integrated Tests


• Created by Ward Cunningham
• Open Source
• The most accepted solution for agile
acceptance testing

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

39
Tools

FitNesse

• Environment build around FIT


• Makes everything easier
• Created by Object Mentor, Inc.
• Open Source

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

40
FIT FitNesse

- Stand alone web server


- Is a wiki
- Tests written in HTML
- Tests written in wiki text
- Tests are executed on the command line
- Tests are executed from within the wiki
- Tables are executed
- Translates tests into HTML
- Non-table markup is ignored
- Uses FIT to execute tests
- Tables map to Fixtures
- Supports test suites
- Fixtures are code that is aware of the
- Supports variables in tests
system
- Supports test refactoring
- Supplies foundational Fixtures
- Written in Java
- Implementations ported to many
- Supports FIT implementations in any
languages
language

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

41
Thinking in Tables

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

42
Thinking in Tables

Only Tables Execute

Ignored

Executed

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

43
Thinking in Tables

Foundational Table Structure


Name of Fixture

Interaction with Application

• Table structure depends on type of Fixture

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

44
Thinking in Tables

3 Foundation Fixtures

• Column Fixture
• Row Fixture
• Action Fixture

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

45
Thinking in Tables

Column Fixture
package eg;
// Copyright (c) 2002 Cunningham & Cunningham
// Released under the terms of the GNU Genera
Public …

import fit.ColumnFixture;

public class Division extends ColumnFixture {


public float numerator;
public float denominator;
public float quotient() {
return numerator / denominator;
}
}

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

46
Thinking in Tables

Row Fixture
Analogous to comparing
against rows in a
database table
package fitnesse.fixtures;

import fit.RowFixture;
public class EmployeePayRecord {
public int id;
public class EmployeePayRecordsRowFixture extends private double salary;
RowFixture { public EmployeePayRecord(int id,
public Object[] query() throws Exception { double salary){
EmployeePayRecord[] records = new this.id = id;
EmployeePayRecord[2]; this.salary = salary;
records[0] = new EmployeePayRecord(1, 1000); }
records[1] = new EmployeePayRecord(2,2000); public double pay() {
return records; return salary;
} }
public Class getTargetClass() { }
return EmployeePayRecord.class;
}
}

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

47
Thinking in Tables

Action Fixture
• Think GUI window Counter Window
Counter:
public class CountFixture extends Fixture {
private int counter = 0; Counter: 6

public void count() { Count


counter++;
}

public int counter() {


return counter;
}
}

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

48
Exercise #2

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

49
Exercise #2

Login With Tables


• Write the Login test using tables
• Keep in mind what we learned from the
previous exercise
• BOC
• Examples
• Avoid Implementation Details
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

50
Exercise #2

Possible Solution

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

51
FitLibrary

• Extension to FIT
• Written by Rick Mugridge
• Adds some handy Fixtures

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

52
FitLibrary

FitLibrary Fixtures
• ArrayFixture for ordered lists
• SetFixture for unordered lists
• SetUpFixture
• Supports
• Graphics
• Tree structures
• Nested Tables
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

53
FitLibrary

DoFixture

• Broken tables Text

• Highly readable
• Flexibility

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

54
Wiki

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

55
Wiki

What is it?
• A collaborative web site
• Editable by any
• Created by Ward Cunningham
• Every project should have one
• http://c2.com/wiki
• http://en.wikipedia.com
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

56
Wiki

Creating Tests
• Use Wiki syntax to create a page with test tables
• Label the page as a Test Page
• Use a page name of the form Test…
• Turn on the Test property
• Make sure your Fixtures are in the classpath
• Use !path widget
• Mechanics
• !path values are concatenated
• Java command to start FitServer is executed
• Testable HTML is passed to FitServer
• FitServer runs the tests
• Results are passed back to FitNesse
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

57
Wiki

Creating Suites
There are 2 ways to make Suites
• Set the Suite property
• Create a page with the Suite property
• Created test pages inside this page
• When the suite is executed, all child test pages will
be included in the suite execution
• Use the !see widget
• !see <name of test page>
• All “included” tests pages will be included in the
suite execution
Run a Suite by clicking the Suite button
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

58
Hands-on Session
• Conference Proposal Submission Portal
• Some sample Stories
• Should be able to submit new proposal
• Should be able to list all submitted proposal
• Submitting proposal with same title should display appropriate error
message
• Should be able to delete submitted proposal based on the title
• Should be able to delete submitted proposal based on the title
• Should be able to search proposals by title
• Should be able to search proposals by ID
• Should be able to find all proposal by an author's name

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

59
Break

http://www.fitnesse.org

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

60
Patterns

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

61
Patterns

Organizing Tests

• Allowing customers to add new tests


without breaking the build

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

62
Patterns

Version Control

• Keeping the acceptance test in version


control with the code.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

63
Patterns

Cross-Functional Pairing

• Using FitNesse based acceptance tests for


collaboration between cross-functional team
members.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

64
Patterns

ATDD

• Acceptance Test Driven Development

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

65
Patterns

CSTT

• Cleanup, Setup, Test, Teardown

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

66
Patterns

Independent Tests

• Tests shouldn’t depend on each other.


• Tests leave the system in the same state it
started in.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

67
Patterns

Dynamic Stubbing

• Avoiding complications of external systems.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

68
Non-Production
Patterns

Setup/Teardown

• Using non-production light weigh code for


setup and teardown.
• Helps test only what you want to test.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

69
Patterns

Suite Levels

• Creating different levels of suites depending


on the depth/level of feedback desired.
• Smoke, Current Iteration/Sprint, Regression

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

70
Patterns

DRY

• Using !include to avoid repeating yourself.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

71
Patterns

Make it Real

• Write ATs as close as possible to the real


environment.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

72
Patterns

Fixture Evolution

• Allow Fixture implementation to evolve over


time.
• Treat fixtures as first class citizens.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

73
Patterns

At Least One Test/Story

• Every story should have at least one


acceptance test
• Avoid long/multipurpose tests.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

74
Anti-Patterns

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

75
Anti-Patterns

Developer ATs

• Developers writing acceptance tests by


themselves, for themselves.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

76
Anti-Patterns

Unit Testing

• Don’t write ATs at the unit testing level


• Unit tests are implementation specific
• ATs are NOT implementation specific

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

77
Anti-Patterns

QA Testing Tool

• Hard to write tests up front.


• Perhaps only on large projects.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

78
Anti-Patterns

Silver Bullet

• Trying to use FitNesse for all types of


Acceptance Tests
• UI testing
• XML testing

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

79
Anti-Patterns

Test After

• Writing tests after the code is already


written.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

80
Anti-Patterns

Hidden Test Data

• Hiding test data in the fixtures.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

81
Implementation
Anti-Patterns

Dependant ATs

• Making test pages (tables) dependent on


implementation details and data structures.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

82
Anti-Patterns

Logging in Your Fixtures

• Putting log statements or print statements in


the fixture code.
• Fixtures are probably too complicated.

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

83
The End

http://www.fitnesse.org

copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.

84