1

Boundary value analysis, Robustness Tetsing , Equivalence Class Testing & Decision Table Testing
There are two types of testing 1. Black box testing ( by giving inputs test the output according to the requirements) 2. White box testing ( testing source code ) Example:- Requirement for age drop down (1-100) If you select the values between 1 -100 it should accept, else it should display error message. But it is not possible to test all the values from 1 to 100(time consuming) To overcome that, two types of black box testing methods are there to derive test cases 1. Boundary value analysis 2. Equivalence class testing

BOUNDARY VALUE ANALYSIS

Boundary value analysis is one of the black box testing methods to derive test cases and to make sure it works for all the values with in the boundary. Boundary value Test Cases are subset of Robustness Test Cases. 1. Boundary value Test cases of single input 2. Boundary value Test cases of two input 3. Boundary value Test cases of three input(additional info) 4. Worst case Boundary Value Test Cases of two inputs 5. Worst case Boundary Value Test Cases of three inputs

1.Boundary value Test cases of single input :Min+ Avg Max|------|----------------------|------------------------|------| Min Max Boundary value test cases of single input Here we are testing within the boundaries (only valid data according to the requirement) We are not testing for invalid data (out of boundary).
Min ---------------------------------- Minimal Min+ ---------------------------------- Just above Minimal Nom ---------------------------------- Average Max ---------------------------------- Just below Maximum Max ---------------------------------- Maximum

Example :Requirement: Age: 1-100 Boundary Value Test Cases are (min, min+1, avg, max-1, max) 1 2 51 99 100 Requirement: ATM PIN: 0000-9999 Boundary Value Test Cases are (min min+1 avg 0000 0001 5000
18 September 2012

max-1 max) 9998 9999

2.Boundary Value Test Cases of two inputs (x1, x2) are :-

{< x1 nom,x2min>,< x1 nom,x2min+>,< x1 nom,x2nom>,< x1 nom,x2max->,< x1 nom,x2max>, <x1min+,x2nom>,<x1min,x2nom>, <x1max,x2nom>,<x1max-,x2nom> } No. of test cases are 4n+1 (n is no of inputs) Give each set of inputs and check whether the output is according to the requirement or not, instead of testing all sets, you have to test the above set of inputs to make sure that it will work for all the values within the range. Example of two inputs: To test whether multiplication is working correctly or not (we need 2 inputs) Requirement: - c= a* b (1<=a<=100, 1<=b<=100) a= {1,2,50,99 ,100} b={1,2,50,99,100}

Boundary value sets of a and b are

Based on the above sets, combinations of test cases (input sets by BVA) are {< x1 nom,x2min>,< x1 nom,x2min+>,< x1 nom,x2nom>,< x1 nom,x2max->,< x1 nom,x2max>, <x1min+,x2nom>,<x1min,x2nom>, <x1max,x2nom>,<x1max-,x2nom> }

{(50,1),(50,2),(50 ,50),(50,99),(50,100),(1,50),(2,50)(99,50)(100,50)} Take first set of inputs and check whether it is according to the requirement or not Give first set input values are 50 and 1 c=50*1

It displays some result that is called actual result, if you know the multiplication(requirement) you can expect the result that is called expected result (if both accepted result and actual result are same you can say first test case is pass else fail) Like above you have to test all sets of inputs one by one, Second set is (50, 2) give inputs and compare actual and expected and decide whether the test case is pass or fail. Same way third set (50, 50), fourth (50, 99)...

18 September 2012

Table format of multiplication test cases (BVA) Inputs A 50 50 50 . B 1 2 50 . Expected Result 50 100 2500 .. Actual Result 50 100 2500 . Status Pass/fail Pass Pass Pass .

3.Boundary value Test cases of three inputs are :{< x1 nom,x2min>,< x1 nom,x2min+>,< x1 nom,x2nom>,< x1 nom,x2max->,< x1 nom,x2max>, <x1min+,x2nom>,<x1min,x2nom>, <x1max,x2nom>,<x1max-,x2nom> } Example :* {x3min , x3 min+1, x3 nom, x3max-1, x3max} (Cartesian product) Requirement: Example triangle problem: 1 x1 100, 1 x2 100, 1 x3 100 are three sides of triangle.

{(50,1),(50,2),(50 ,50),(50,99),(50,100),(1,50),(2,50)(99,50)(100,50)} * {1, 2, 50, 99,100} {(50,1,1)(50,2,1)(50,50,1),(50,99,1)(50,100,1)..} Give each set of inputs and compare actual and expected results Table format of triangle test cases using BVA Inputs x1 50 50 50 x2 1 2 50 x3 1 1 1 .. Expected Result Isosceles Scalene Isosceles . Actual Result Not a triangle Scalene Not a triangle .. Status Pass/fail Fail Pass Fail

4.Worst-case Boundary value analysis Test cases for two inputs:If you are not satisfied with Boundary value Test cases, that means you cant make sure that, The related functionality is working according to the requirements after testing with the above Boundary value limited sets, and then we can go for Worst Case Testing. In case of two inputs, consider all sets produced by the below Cartesian product {x1 min, x1 min+1, x1 nom, x1max-1, x1max} * {x2 min, x2 min+1, x2 nom, x2max-1, x2max}

18 September 2012

These test cases although more comprehensive in their coverage, constitute much more endeavour. To compare we can
n

see that Boundary Value Analysis results in 4n + 1 test case where Worst-Case testing results in 5 test cases. As each variable has to assume each of its variables for each permutation (the Cartesian product) we have 5 to the n test cases.

5.Worst-Case Boundary value Test cases of three inputs triangle problem:Three sides of the triangle are x1, x2, x3 lies between1-200 test cases are

{{ {x1min,x1min+,x1nom,x1max,x1 max-} *{x2min,x2min+,x2nom,x2max,x2 max-}}*

{x3min, x3min+, x3nom, x3max, x3 max-}

} * {x3 min, x3 min+1, x3 nom, x3max-1, x3 max} Total no.of test cases are 5 n (N is no.of inputs)

18 September 2012

ROBUSTNESS TESTING
Robustness testing is an extension of boundary value analysis, to derive test cases and to make sure it will work for all the values (valid and invalid) with in and out of the boundaries. 1. Robustness Test cases of single input 2. Robustness Test cases of two input 3. Robustness Test cases of three input(additional info) 4. Worst case Robustness Test Cases of two inputs 5. Worst case Robustness Test Cases of three inputs 1.Robustness Testing of single input MinMin+ Avg MaxMax+ |------|------|----------------------|------------------------|------|------| Min Max Robustness Test Cases of single input { x1 mn-,x1min,x1min+,x1avg,x1max-,x1max,x1max+ } Example: Requirement of ATM withdraw min amount is 100 max is 10,000 Robustness Test Cases are (Min-1, min, min+1, avg, max-1, max max+1) 0 1 2 51 99 100 101

2.Robustness Test Cases of two inputs a<= x1<=b,c<= x2<=d:-

{< x1 nom,x2min->,< x1 nom,x2min>,< x1 nom,x2min+>,< x1 nom,x2nom>,< x1 nom,x2max->, <x1nom,x2max>,<x1nom,x2max+>,<x1min,x2nom>,<x1min,x2nom>,<x1min+,x2nom><x1max+,x2nom>, <x1max,x2nom>,<x1max-,x2nom> } No.of test cases are 6n+1 (n is no of inputs)

18 September 2012

Give each set of inputs and check whether the output is according to the requirement or not, instead of testing all the values you have to test the above set of inputs to make sure it will work for all the values with in and out of the boundaries(valid and invalid). Example for writing Robustness Test Case of two inputs To test whether addition is working correctly or not (2 inputs) c= a+ b (1<=a<=100, 1<=b<=100) a= {0,1,2,50,99,100,101} b={0,1,2,50,99,100,101} Based on the above sets, combinations of test cases (inputs) are {(50,0)(50,1),(50,2),(50 ,50),(50,99),(50,101),(50,100),(1,50),(2,50)(99,50)(100,50)(101,50)(0,50)} Take first set of inputs and check whether it is according to the requirement or not Give first set input values are 50 and 0 c=50+0

It displays some result that is called actual result, if you know the addition you can expect the result that is called expected result (if both accepted result and actual result are same you can say first test case is pass else fail) Like this you have to test all sets of inputs one by one, second set is (50, 1) give inputs and compare actual and expected and decide whether that is pass or fail. Same way third set (50, 2), fourth (50, 50)... Table format of addition test cases (Robustness Testing) Inputs A 50 50 50 50 . B 0 1 2 50 . Expected Result 50 51 52 100 .. Actual result 50 51 52 100 . Status Pass/fail Pass Pass Pass Pass .

3.Robustness Test Cases In case of three inputs (example triangle problem):{<x1nom,x2min>,<x1nom,x2min>,<x1nom,x2min+>,<x1nom,x2nom>,<x1nom,x2max,<x1nom,x2max+>,<x1min,x2nom>,<x1min,x2nom>,<x1min+,x2nom><x1max+,x2nom>, <x1max,x2nom>,<x1max-,x2nom> } * {x3 min-,x3 min, x3 min+1, x3 nom, x3 max-1, x3 max,x3 max+}

Requirement:1 x1 100, 1 x2 100, 1 x3 100 are three sides of triangle.

Test cases are:{(50,0),(50,1),(50,2),(50 ,50),(50,99),(50,100),(50,101)(0,50),(1,50),(2,50)(99,50)(100,50),(101,50)} * {0, 1, 2, 50, 99,100,101}

18 September 2012

{(50,0,0)(50,1,0)(50,2,0)(50,50,0),(50,99,0)(50,100,0)..} Give each set of inputs and compare actual and expected result Table format of triangle test case (robustness testing) Inputs X1 50 50 50 . X2 0 1 2 . X3 0 0 0 .. Expected Result
Isosceles Scalene Scalene

Actual result
Not a triangle Scalene Scalene

Status Pass/fail
Fail Pass Pass

..

4.WorstCase Robustness Test Cases of two inputs :{ x1 mn-,x1min,x1min+,x1avg,x1max-,x1max,x1max+} * { x2 mn-,x2min,x2min+,x2avg,x2max-,x2max,x2max+ }

These test cases although more comprehensive in their coverage, constitute much more endeavour. To compare we can
n

see that Robustness Testing results in 6n + 1 test case where Worst-Case testing results in 7 test cases. As each variable has to assume each of its variables for each permutation (the Cartesian product) we have 7 to the n test cases. Total no. of Test Cases is 7n

5.Worst-case Robustness Test cases of (three inputs) triangle problem(x1, x2, x3 are the sides):-

{ {x1 mn-,x1min,x1min+,x1avg,x1max-,x1max,x1max+} * { x2 mn-,x2min,x2min+,x2avg,x2max-,x2max,x2max+ } * {x1 mn-,x1min,x1min+,x1avg,x1max-,x1max,x1max+}

18 September 2012

* {x3 min-,x3 min, x3 min+1, x3 nom, x3 max-1, x3 max,x3 max+} Below are the few worst-case Robust ness Test Cases of triangle problem (1<=x1, x2, x3<=200)

Conclusion: The above Test cases are used in the case of single or multiple inputs and inputs should have upper and lower boundaries) Not possible to test all the values so we can use this Black Box Testing methods BVA Test Cases are sub set of Robustness Test Cases, using BVA we are testing within the boundaries (valid inputs), in Robustness Testing we are testing with and with out of the boundaries (valid and invalid). Best method to follow is Robustness testing.

18 September 2012

Equivalence Class Testing

Use the mathematical concept of partitioning into equivalence classes to generate test cases for Functional (Black-box) testing The key goals for equivalence class testing are similar to partitioning: o completeness of test coverage o lessen duplication of test coverage Partitioning : Recall a partitioning of a set, A, is to divide the set A into ( a1, a2, - - - -, an ) subsets such that: o a1 U a2 U - - - - U an = A (completeness) o for any i and j, ai aj = (no duplication)

Equivalence Classes : Equivalence class testing is the partitioning of the (often times, but not always) input variables value set into classes (subsets) using some equivalence relation. There are four types of Equivalence Class Testing types 1. Weak Normal Equivalence Class Testing 2. Strong Normal Equivalence Class Testing 3. Weak Robust Equivalence Class Testing 4. Strong Robust Equivalence Class Testing Equivalence Class Test Cases of single input: In Boundary Value and Robustness Testing we are selecting the inputs at the boundary points and one avg value. For example age- 0-100 in we are considering only -1,0,1, 50, 99,100 and 101 , but it may not accept when you enter the age not at the boundary like 30 , 40, 60, 70 etc., we are not testing these values in BVA or RT to overcome that we can go for Equivalence Class Testing. Consider a numerical input variable, i, whose values may range from -200 through +200. Then a possible partitioning of testing input variable by 4 people may be: o -200 to -100| -101 to 0|1 to 100|101 to 200

Inputs are -150, -50, 90, and 150 Weak Normal Equivalence testing of two inputs: Assumes the single fault or independence of input variables. E.g. If there are 2 input variables, these input variables are independent of each other. Partition the test cases of each input variable separately into different equivalent classes. Choose the test case from each of the equivalence classes for each input variable independently of the other input variable

18 September 2012

10

Strong Normal Equivalence testing of two inputs: This is the same as the weak normal equivalence testing except for Multiple fault assumption or Dependence among the inputs All the combinations of equivalence classes of the variables must be included.

Weak Robust Equivalence testing of two inputs: Up to now we have only considered partitioning the valid input space. Weak robust is similar to weak normal equivalence test except that the invalid input variables are now considered.

18 September 2012

11

Strong Robust Equivalence testing of two inputs: Does not assume single fault - - - assumes dependency of input variables Strong robust is similar to strong normal equivalence test except that the invalid input variables are now considered.

18 September 2012

12

Equivalence Class Test Cases for the Triangle Problem We may define the input test data by defining the equivalence class through viewing the 4 output groups: R1-input sides <a, b, c> do not form a triangle R2-input sides <a, b ,c> form an isosceles triangle R3-input sides <a, b, c> form a scalene triangle R4-input sides <a, b, c> form an equilateral triangle Weak Normal Test Cases of Triangle problem Test Case WN1 WN2 WN3 WN4 A 5 2 3 4 b 5 2 4 1 C 5 3 5 2 Expected Output Equilateral Isosceles Scalene Not a Triangle

Weak Robust Test Cases of Triangle problem Test Case WR1 WR2 WR3 WR4 WR5 WR6 A -1 5 5 201 5 5 b 5 -1 5 5 201 5 C 5 5 -1 5 5 201 Expected Output Value a is not in range Value b is not in range Value c is not in range Value a is not in range Value b is not in range Value c is not in range

Strong Robust Test Cases of Triangle problem Test Case SR1 SR2 SR3 SR4 SR5 SR6 SR7 A -1 5 5 -1 5 -1 -1 b 5 -1 5 -1 -1 5 -1 C 5 5 -1 5 -1 -1 -1 Expected Output Value a is not in range Value b is not in range Value c is not in range Value a,b is not in range Value b,c is not in range Value c,a is not in range Value c,a,b is not in range

18 September 2012

13

Decision Table Testing

Decision Table-Based Testing has been around since the early 1960s; it is used to analyze complex logical relationships between input data. Definition:A Decision Table is the method used to build a complete set of test cases without using the internal structure of the program. In order to create test cases we use a table that contain the input and output values of a program. Such a table is split up into four sections as shown below in fig 2.1.

In fig 2.1 there are two lines which divide the table into four components. The solid vertical line separates the Stub and Entry portions of the table, and the solid horizontal line is the boundary between the Conditions and Actions. These lines separate the table into four portions, Condition Stub, Action Stub, Condition Entries and Action Entries. A column in the entry portion of the table is known as a rule. Values which are in the condition entry columns are known as inputs and values inside the action entry portions are known as outputs. Outputs are calculated depending on the inputs and specification of the program
SAMPLE DECISION TABLE RULES ACTION ENTRIES

CONDITIONS

ACTIONS 2.2 Typical Structure of Decission Table

CONDITION ENTRIES

The above table is an example of a typical Decision Table. The inputs in this given table derive the outputs depending on what conditions these inputs meet. Notice the use of -in the table below, these are known as dont care entries. Dont care entries are normally viewed as being false values which dont require the value to define the output. Figure 2.2 shows its values from the inputs as true(T) or false(F) values which are binary conditions, tables which use binary conditions are known as limited entry decision tables. Tables which use multiple conditions are known as extended entry decision tables

18 September 2012

14

Redundancy Decision Table:When using dont care entries a level of care must be taken, using these entries can cause redundancy and inconsistency within a decision table. An example of a decision table with a redundant rule can be seen in figure 3.3. From the table you can see that there is some conflict between rules 1-4 and rule 9, rules 1-4 use dont care entries as an alternative to false, but rule 9 replaces those dont care entries with false entries. So when condition 1 is met rules 1 -4 or 9 may be applied, luckily in this particular instance these rules have identical actions so there is only a simple correction to be made to complete the following table(we can remove any one rule1-4 or 9).

Figure 3.3 an example of a Redundant Rule

Inconsistency Decision Table :If on the other hand the actions of the redundant rule differ from that of rules 1-4 then we have a problem. A showing this can be seen in figure 3.4. table

Figure 3.4 an example of Inconsistent Rules

From the above decision table, if condition 1 was to be true and conditions 2 and 3 were false then rules 1-4 and 9 could be applied. This would result in a problem because the actions of these rules are inconsistent so therefore the result is nondeterministic and would cause the decision table to fail.

Decision Table for Triangle Problem

As explained above, there are two types of decision table, limited and extended entry tables. Below, in fig 3.1 is an example of a limited entry decision table where the inputs are binary conditions.

Fig 3.1 Decision Table for the Triangle Problem

Rule Counts :Rule counts are used along with dont care entries as a method to test for decision table completeness; we can count the no. of test cases in a decision table using rule counts and compare it with a calculated value. Below is a table which illustrates rule counts in a decision table.

18 September 2012

15

Fig 3.2 an example of Rule Counts in a Decision Table

The table above has a total rule count of 64; this can be calculated using the limited entry formula as its a limited entry table. Number of Rules = 2n (n is no. of conditions) So therefore, Number of Rules =26 = 64 When calculating rule counts the dont care values play a major role to find the rule count of that rule.

Test cases for the triangle problem based on 3.2 diagram

A DT1 DT2 DT3 DT4 DT5 DT6 DT7 DT8 DT9 DT10 DT11 4 1 1 5 2 2 3 3 B 1 4 2 5 2 3 2 4 C 2 2 4 5 3 2 2 5 Expected Output Not a Triangle Not a Triangle Not a Triangle Equilateral Impossible Impossible Isosceles Impossible Isosceles Isosceles Scalene

18 September 2012