Regression Testing

Regression Testing

Srinivasan Desikan

Regression Testing

Regression Testing

Doctor: Congratulations! The stomach ulcer that was bothering you and preventing digestion is now completely cured!
08/27/13

Patient: That is fine doctor, but I have got such a bad mouth ulcer that I cant eat anything and hence there is nothing to digest!
2

Regression Testing

Regression Testing

Regression Testing - Definition Regression testing is selective retesting of the system with an objective to ensure the bug fixes work and those bug fixes have not caused any unintended effects in the system This testing is done to ensure The bug-fixes work And bug-fixes do not create any side-effects
08/27/13 Regression Testing 3

Regression Testing

Regression Testing Types

Dev. team
Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes

Build 1
New Features

Build 2

Build 3 Build 4
New Features

Build 5 Build 6
New Features

Test team test Cycle 1

Reg. 1

test Cycle 2
Time line

Reg. 2 test Cycle 3

Final Reg.
4

08/27/13

Regression Testing

Regression Testing
Regression Testing Types I. Final regression Testing Unchanged build exercised for the min period of cook time (Gold master build) To ensure the same build of the product that was tested reaches the customer More critical than any other type of testing Used to get a comfort feeling on the product prior to release II. Regression testing To validate the product builds between test cycles Unchanged build is recommended but not mandatory Used to get a comfort feeling on the bug fixes, and to carry on with next cycle of testing Also used for making intermediate releases (Beta,Alpha)

08/27/13

Regression Testing

Regression Testing Types

Regression Testing

II. Regression testing To validate the product builds between test cycles Unchanged build is recommended but not mandatory Used to get a comfort feeling on the bug fixes, and to carry on with next cycle of testing Also used for making intermediate releases (Beta,Alpha)

08/27/13

Regression Testing

Regression Testing

Methodology explained

1: What is needed for selecting regression test cases? 2. Performing smoke tests 3: Classifying test cases 4: Selecting test cases 5: Resetting test cases for regression testing & phases of testing 6: How to conclude results 7: Popular strategies 8: Some guidelines 9: exercise
08/27/13 Regression Testing 7

Regression Testing

Types Regular regression Final regression

What? Selective re-testing to ensure Defect fixes work No side effects

Why? Defects creep in due to Changes Defect fixes may cause existing functionality to fail

When? When set of defect fixes arrive after formal testing for those areas completed Performed in all test phases
8

08/27/13

Regression Testing

Regression Testing What is needed for selecting test cases? Bug fixes and how it affect the system Area of frequent defects Area which has undergone many/recent code changes Area which is highly visible to the users Area which have risks are more Core features of the product which are mandatory requirements of the customer

08/27/13

Regression Testing

Regression Testing Points to remember

Selection depends more on the criticality of bug fixes than the criticality of the defect itself More positive test cases than negative test cases for final regression Constant set of regression test cases may not work always

08/27/13

Regression Testing

10

Classifying test cases Regression Testing

Note The order of test execution is priority 0, 1 & 2 , Priority helps in entry, exit criteria Priority0 Test cases Sanity test cases which checks basic functionality Run for acceptance tests and for major change Delivers very high project value

08/27/13

Priority 1 Test cases

Uses

Priority 2 Test cases

Uses

the basic and normal setup Delivers high project value High probability of getting selected for regression
Regression Testing

extended

setup Moderate project value Some functional and all nonfunctional test cases fall into this category
11

Regression Testing Performing initial smoke test

Smoke testing ensures that the basic functionality works and indicate that the build can be considered for further testing

Defect is cumulative effect of people, product, process and roles. Regression should take care of all these aspects Regression Testing 08/27/13 12

Criteria

Regression Testing Selecting test cases

a. Bug fixes work b. No side effects

Defect fixes Legend

P0 Priority 0 P1 Priority 1 P2 Priority 2 - Very few - All - Subset

Criticality

Impact

Low

Med

High

P0 P1 P2
08/27/13

P0 P1 P2
Regression Testing

P0 P1 P2
13

Regression Testing How to conclude results

Test case result (Current build) FAIL Test case result (Previous build) PASS

Remarks

Regression failed and Apply RESET guidelines and proceed after getting new build Bug fixes are working and Continue your regression to find side effects Bug fixes not working or not provided or Wrong selection If you are satisfied with work around then result marked as PASS else FAIL (Check with Prog mgmt) This test case could have been included for finding side effects or Wrong selection

PASS

FAIL

FAIL

FAIL

Pass with work around PASS

FAIL

PASS

08/27/13

Everyone monitors regression test results as it not only talk about defects and but also their fixes
Regression Testing

18

Regression Best Testing Practices 5. Detect defects, protect your product from

defects and defect fixes

08/27/13

Regression Testing

23

Regression Testing

08/27/13

Regression Testing

24

Regression Testing

08/27/13

Regression Testing

25