Lect5 6a Test Debug

Software Engineering
Lecture 5 & 6:
Unit testing & debugging
2013
Outline
What is testing?
Test (stand-alone) procedures:
Black box vs white box testing
Develop and run tests using JUnit
Defensive programming
Debugging
2013
FIT330 Software Engineering
What is testing?
A form of program validation
Validation means to validate our belief that a
program functions correctly:
i.e. satisfies its specification
in practice, we can best increase our confidence
Testing = to use real data to test a program to

meet its specification
2013
Testing method
Test incrementally
Three techniques:
unit testing: test each module in isolation
integration testing: test a group of modules
regression testing: re-run tests after modifications
Two basic steps:
2013
generate test cases
develop & run tests
Test case
Exhaustive testing is impractical
Goal: find a small, representative set of test
cases
Test case is a set of input data values of a
given test unit
Belief: succeeds with TCs succeeds with all
input
2013
Test case
Input domains
int
Test cases
int[]
2013
someMethod(int x, int[] a)
Test stand-alone procedures

Generate test cases:
black-box vs. white-box
Develop and run test:
2013
use JUnit
2a
Generate test cases
Objective: find approximations of the input

domains
Approximation based on representative test
data sets (TDSs)
Two basic steps:
2013
Generate TDSs
Combine TDSs to form test cases
Generate TDSs
Generate TDSs
int
x1
x2
TDS
int[]
a1
a5
a4
2013
a2
a3
Combine TDSs
Generate TDSs
int
Combine TDSs
x1
<x1,a1>
x2
<x2,a1>
...
int[]
<x1,a5>
<x2,a5>
a1
a5
a4
2013
a2
a3
10
Test cases
Generate TDSs
int
Combine TDSs
Test cases
x1
<x1,a1>
x2
<x2,a1>
...
int[]
<x1,a5>
<x2,a5>
a1
a5
a4
2013
a2
a3
11
Generate TDSs
TDS is a sub-set of values of an input domain
One input domain may contain several TDSs
Steps to generate TDSs:
identify the input ranges
define a TDS per range
using representative data values

x1
x2
2013
12
Techniques
Two techniques:
black-box testing (BBT)
glass-box (a.k.a white-box) testing (GBT)
BBT uses program specification

GBT uses program text
2013
13
Black box testing

/**
* @effects
*
if a is null
*
throws NullPointerException
*
else if x is in a
*
returns i such that a[i] = x
*
else
*
throws NotFoundException
*/
public static int search(int[] a, int x)
throws NullPointerException, NotFoundException
How do we test this method

2013
?
14
Other examples: sqrt

/**
* @requires
* @effects
*
*/
public static
2013
x >= 0 && .00001 < epsilon < .001

return n such that
x-epsilon < n^2 <= x+epsilon
float sqrt(float x, float epsilon)
15
isPrime
/**
* @effects
*
if p is a prime
*
returns true
*
else
*
returns false
*/
public static boolean isPrime(int p)
2013
16
BBT: input range criteria

@requires: constraint expressions
@effects:
output conditions
exceptional conditions
Consider relationships between inputs in

@effects
Written using
2013
semi-informal set notation
number range notation

17
search
/**
* @effects
*
if a is null
*
throws NullPointerException
*
else if x is in a
*
returns i such that a[i] = x
*
else
*
throws NotFoundException
*/
public static int search(int[] a, int x)
Ranges:
2013
a: {null}, {[]} + {[x1,...,xn] | xi are integers}
x: {y | y in a} + {y | y not in a}
18
sqrt
/**
* @requires
* @effects
*
*/
public static
x >= 0 && .00001 < epsilon < .001

return n such that
x-epsilon < n^2 <= x+epsilon
float sqrt(float x, float epsilon)
Ranges:
2013
x: [0,+INF)
epsilon: (.00001, .001)
19
isPrime
/**
* @effects
*
if p is a prime
*
returns true
*
else
*
returns false
*/
public static boolean isPrime(int p)
Ranges:
2013
primes: {1,2,3,5,...}
non-primes: {4,6,8,9,...}
20
BBT: TDS formation criteria

For each range use:
data type (incl. array)
typical and atypical data of the type
String data type: treated as an array-type

Other types (later):
2013
iterator
data abstraction
type hierarchy
21
Typical vs. Atypical data

Numeric data type:
typical: a few numbers in the range
atypical: smallest and largest values of the range
Array data type:
2013
typical: arrays containing a few elements
atypical:
null, empty, one-element array
arrays containing known values at specific indices
22
TDS examples: search

Ranges:
a: {null}, {[]} + {[x1,...,xn] | xi are integers}
x: {y | y in a} + {y | y not in a}
TDS(s):
2013
a: {null, [], [1], [3,1], [3,1,4], [3,5,1,4]}
x: {1,2}
23
sqrt
Ranges:
x: [0,+INF)
epsilon: (.00001, .001)
TDS(s):
x: {0, 0.001, 0.01, 0.09, 0.5,

1, 2, 10, 100, 2147483600}
2013
y?
epsilon: {.00002, .0001, .0009}
24
isPrime
Ranges:
primes: {2,3,5,...}
non-primes: {4,6,8,9,...}
TDS(s):
2013
primes: {2,3,5,31,65537}
non-primes: {4,32,65538}
y?
25
BBT pros & cons

Advantages:
ease TC generation
robust against implementation changes
ease result interpretation
Disadvantages:
2013
path incomplete if specification is flawed,

incomplete or not observed
26
Glass box testing

static int maxOfThree(int x,int y,int z) {
if (x > y) {
if (x > z) return x; else return z;
} else {
if (y > z) return y; else return z;
}
}
How do we test this method
2013
27
GBT: input range criteria

Use the logic paths found in:
conditional
loop
recursion
Criteria:
2013
conditional expression
number of iterations (loop & recursion only)
28
Conditional
if P(x)
// do
else if
// do
else
// do
this
Q(x)
that
somethingelse
Range(s):
2013
all x s.t P(x)
all x s.t Q(x)
all x s.t. P(x) && Q(x)

29
Example: conditional
if (x > y) {
} else { // x <= y
}
}
Range(s) for x,y,z:
2013
x > y,z
zx>y
x,z y
x y z
y?
30
deterministic loop
static int someMethod(int n) {
for (int i = 1; i <= n; i++) {
// do something
}
}
Range(s) for n:
2013
(-INF,0]
[1,+INF]
31
non-deterministic loop
static int someMethod(int x) {
while (x > 0) {
// do something with x
}
}
Range(s) for x:
2013
(-INF,0]
[1,+INF)
y?
32
recursion
static int fact(int n) {
if (n < 1)
return -1;
else if (n == 1)
return 1;
else
return n * fact(n-1);
}
Range(s) for n:
2013
(-INF,1)
{1}
(1,+INT)
y?
33
GBT: TDS formation criteria

Depends on the range:
conditional: data type
loop/recursion: number of iterations & termination
Reasonable numbers of iterations:
loop: 0, 1, 2
recursion: 0, 1 (base), 2 (inductive once), 3
(inductive twice)
Termination:
2013
termination values
include exceptional cases

34
TDS example: conditional

if (x > y) {
} else { // x <= y
}
}
Range(s) for x,y,z:
2013
TDS(s) for (x,y,z):
x > y,z
{ (3,2,1), (3,2,2), (3,1,2) }
zx>y
{ (3,2,4), (3,2,3) }
x,z y
{ (1,2,1), (1,2,2), (2,2,1) }
x y z
{ (1,2,3), (1,1,2), (1,2,2),

(1,1,1) }
35
deterministic loop
static int someMethod(int n) {
for (int i = 1; i <= n; i++) {
// do something
}
}
Range(s) for n:
2013
(-INF,0]
[1,+INF]
TDS(s) for n:
{0,1,2}
36
non-deterministic loop
while (x > 0) {
// do something
}
}
Range(s) for x:
2013
(-INF,0]
[1,+INF)
TDS(s) for x:
{0,1,2}
y?
37
recursion
static int fact(int n) {
if (n < 1)
return -1;
else if (n == 1)
return 1;
else
return n * fact(n-1);
}
Range(s) for n:
2013
(-INF,1)
{1}
(1,+INT)
TDS(s) for n:
{-1, 1, 2, 3}
y?
38
conditional & loop (1)

while (x > 0) {
// checks x modulo 10
if (x % 10 == 5)
break;
x--;
}
}
Range(s) for x:
TDS(s) for x:
(-INF,0]
{0, 5,15,1,2}
{ y | y = 10k+5, k 0 }
{ y | y 10k+5, k 0 }
2013
y?
39
conditional & loop (2)

static void myMethod(int n, int m) {
for (int i = 1; i <= n; i++) {
if (pred(i * m))
m++;
}
}
How many logic paths in the worst case
2013
40
conditional & loop (2): TDS

static void myMethod(int n, int m) {
for (int i = 1; i <= n; i++) {
if (pred(i * m))
n = 2
i = {1,2}
m++;
}
}
2013
choose m satisfies one of these:

pred(m) && pred(2m+2)
pred(m) && pred(2m+2)
pred(m) && pred(2m)
pred(m) && pred(2m)
41
GBT pros & cons

Advantages:
useful when specification is flawed, incomplete or

not observed
aid code analysis and debug
Disadvantages:
2013
requires knowledge of the implementation
not robust against implementation changes
42
2b
Develop and run tests
Testing tasks are repetitious

Fully automated testing is difficult
Step 1 is difficult to automate
specifications are not always precise
Step 2 can be automated (our focus here)
2013
using JUnit
43
General approach
Develop test drivers to:
realise the test cases
automate the execution of tests
Test run = run the test driver
2013
44
Test driver
A small program
Specified as a data abstraction
naming convention: xxxTest
Tasks include:
initialise the TDSs and test cases
initialise the expected test results
test each unit
Use assertion to check test results

May form a type hierarchy
2013
45
Example: driver for procedure

One driver per procedure
SquareRootTest
@Test
+ squareRoot()
2013
Num
+<S> sqrt(float,float): float
46
Example: driver for data abstraction

One driver per class
IntSetTest
- tcIntSet: int[][]
- tcX1: int[][]
- tcX2: int[][]
+<S> setUp()
+<S> close()
+ testBasic()
+ testMixed()
+ testMutator()
+ testObserver()
2013
IntSet
- elements: Vector
+ IntSet()
+ insert(int)
+ remove(int)
+ isIn(int): boolean
+ size(): int
+ elements(): Iterator
+ repOK(): boolean
+ toString(): String
47
Implement a test driver

Use a third-party package named JUnit
Two versions:
2013
JUnit 3.5: test drivers as sub-types

JUnit 4.0 (Jdk >= 1.5): test drivers as annotated
procedures
48
Assertion
A boolean statement validated automatically by
the run-time environment
Java keyword: assert
variants: assertEquals, assertArrayEquals
Usage:
2013
validate test results
defensive programming (later)
49
Assertion example
IntSet s = ... // initialise s
assert (s.repOK() == true);
assert that s.repOK is true

otherwise throws an AssertionError
assert (s.size() == 2) : "invalid size "+s.size();
assert that s has two elements

otherwise throws AssertionError
with message invalid size ...
2013
50
Test driver for procedure

Named after the procedure
Brief initial specification only:
no AF and rep invariant
Test method:
is annotated with @Test
throws AssertionError
May be parameterised for each test case
2013
51
Example: SquareRootTest
ch10.junit4.SquareRootTest
Note:
SquareRootTest
arrays are used to initialise

test cases and results
2013
a for loop is used to run each @Test

+ squareRoot()
test case
use static method Assert.assertEquals to
ease assertion test
52
Specification
/**
* @overview A test driver for the Num.sqrt method
*/
public class SquareRootTest {
/**
* @modifies System.out
* @effects
*
for each test case tc = < x,e,r >
y?
*
if |Num.sqrt(x)^2 - r^2| > e
*
throws AssertionError
*
else
*
displays Num.sqrt(x) on the std output
*
*/
@Test
public void squareRoot() throws AssertionError
}2013
53
Implementation
@Test //(timeout = 5000)
public void squareRoot() throws AssertionError {
//... (code omitted) ...
// test cases
float[] tcEps = { 0.00002f, 0.0001f, 0.009f };
float[] tcX = { 0f, 0.001f, 0.01f, 0.09f, 0.5f,
1f, 2f, 10f, 100f,
2147483600f };
// test results
float[] results = new float[tcX.length];
for (int i = 0; i < tcX.length; i++){
results[i] = (float) Math.sqrt(tcX[i]);
};
2013
54
float x,e,r;
for (int i = 0; i < tcX.length; i++) {
x = tcX[i];
r = results[i];
for (int j = 0; j < tcEps.length; j++) {
System.out.println(">>Test case " +
((i*tcEps.length)+j));
e = tcEps[j];
float result = Num.sqrt(x, e);
// assume same delta error b/w two results
assertEquals(r * r, result * result, 2 * e);
System.out.printf("sqrt(%f,%f) = %f " +
"(expected = %f) %n", x, e, result, r);
}
}
} // end squareRoot
2013
55
DEMO
ch10.junit4.SquareRootTest
2013
56
Parameterised test driver

Annotated with
@RunWith(Parameterized.class)
Rep defines a test case and expected output
A constructor to initialise rep with suitable args
A method to set up the test cases:
annotated with @Parameters
static
return type: Collection
Test method operates directly on the rep

2013
57
DEMO
Example
ch10.junit4.
SquareRootParameterisedTest
2013
58
Defensive programming
Being extra careful when programming
Three additional 'checks':
check rep invariant
check the @requires constraints
exhaustive testing of all conditionals
Use assertion to flexibly implement the checks
2013
59
The three 'checks'

Check rep invariant:
implements repOK
Check @requires:
checks input values against the pre-condition
Exhaustive testing of conditionals:
2013
covers all possible cases (including unspecified)
60
Example
String s = Comm.receive();
if (s.equals("deliver")) {
// carry out the deliver request
} else if (s.equals("examine")) {
// carry out the examine request
}
2013
61
Example: defensive
} else {
// handle error case
}
2013
62
Use assertion
} else {
// this can never happen
assert false;
}
2013
63
Enable/disable assertions
Assertions are disabled by default
To enable (disable) use JVM's option -ea (da)
To enable assertions:
java -ea MyProgram
To disable assertions:
java -da MyProgram
2013
64
Defensive programming issues

Require more programming effort
Affect run-time performance
Need to balance:
2013
code quality
cost/time
performance
65
Debugging
To uncover and correct bugs (errors)
Examine the intermediate states
Need test cases that produce the bugs
Efficiency depends on:
2013
design
implementation
documentation (e.g. specifications)
66
Debugging steps
Find bug-producing TCs
Locate buggy code regions
Fix buggy code regions
Retest program (regression testing)
2013
67
Test program
Debug
flow-chart
buggy?
Y
Fix buggy regions
Debugging
End test
2013
68
Form a hypothesis consistent with the test

result
Design and run tests that can refute the
hypothesis
Repeat steps 1 & 2 until a hypothesis is
established
Generate bug-producing TCs from the
hypothesis
2013
use typical/atypical value rule

69
Flow chart (1)

Form a hypothesis
Refute hypothesis via

testing
refuted?
N
Generate bug-producing
TCs
2013
70
Example: (buggy) Num.search

Run SearchTest.search
Run SearchTest.buggySearch
What are the bug-producing TCs
2013
71
Another (buggy) Num.search

Run SearchTest.buggySearch2
What are the bug-producing TCs ?
2013
72
Trace the program with the TCs

Examine the intermediate results top-down
Data abstraction: check procedure groups
Procedure: check @requires @effects
Code region: check variables
Aided by a debugger (e.g. Java debugger)
2013
73
Float chart (2)

Trace the program
with the TCs
Use a
debugger
Examine intermediate results
code
region
found?
Y
2013
74
DEMO
Debug
ch10.Num.search
Use Eclipse's debugger

Debug-runs SearchTest.buggySearch
What is the bug
2013
75
Fix buggy code regions
Carefully analyse each region

Common programming pitfalls:
2013
syntactically correct typing errors
reverse the order of input arguments
loop one index too far
fail to reinitialise a variable
incomplete code copy
incorrect use of parentheses in an expression
76
Debug guidelines (1)

Use a debugger (if one available)
Ensure the right source code
Implements sensible toString methods
A bug may occur far from its first manifestation
Determine the program assumptions
Check carefully the input (against @requires)
Eliminate possible code regions
Get help from others
Take a break!
2013
77
Debug guidelines (2)

Make sure the bug matches its symptoms
Understand why the bug occurred where it is
Determine the impacts of code modifications
2013
78
Summary
Testing is a practical method of validating a program's
correctness
Test cases are generated from specification (BBT)
and/or code (GBT)
Test drivers are data abstractions implemented using
JUnit
Defensive programming involves inserting checks to
detect errors
Debugging is a method to uncover bugs and correct the
affected code regions
2013
79
Questions?
2013
80

Lect5 6a Test Debug

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Lect5 6a Test Debug

Hochgeladen von

Copyright:

Verfügbare Formate

Software Engineering

Black box vs white box testing

Develop and run tests using JUnit

FIT330 Software Engineering

i.e. satisfies its specification

in practice, we can best increase our confidence

Testing = to use real data to test a program to

FIT330 Software Engineering

unit testing: test each module in isolation

integration testing: test a group of modules

regression testing: re-run tests after modifications

Two basic steps:

generate test cases

develop & run tests

FIT330 Software Engineering

FIT330 Software Engineering

FIT330 Software Engineering

Test stand-alone procedures

black-box vs. white-box

Develop and run test:

FIT330 Software Engineering

Generate test cases

Objective: find approximations of the input

Combine TDSs to form test cases

FIT330 Software Engineering

FIT330 Software Engineering

FIT330 Software Engineering

FIT330 Software Engineering

identify the input ranges

define a TDS per range

using representative data values

FIT330 Software Engineering

black-box testing (BBT)

glass-box (a.k.a white-box) testing (GBT)

BBT uses program specification

FIT330 Software Engineering

Black box testing

How do we test this method

FIT330 Software Engineering

Other examples: sqrt

x >= 0 && .00001 < epsilon < .001

FIT330 Software Engineering

FIT330 Software Engineering

BBT: input range criteria

Consider relationships between inputs in

semi-informal set notation

number range notation

a: {null}, {[]} + {[x1,...,xn] | xi are integers}

x >= 0 && .00001 < epsilon < .001

FIT330 Software Engineering

FIT330 Software Engineering

BBT: TDS formation criteria

data type (incl. array)

typical and atypical data of the type

String data type: treated as an array-type

FIT330 Software Engineering

Typical vs. Atypical data

typical: a few numbers in the range

atypical: smallest and largest values of the range

Array data type:

typical: arrays containing a few elements

null, empty, one-element array

arrays containing known values at specific indices

FIT330 Software Engineering