Subject: STQA Academic Year: 2017-18

Chapter 3. Managing the Test Process

Topics: Efficient Test Suite Management: minimizing the test suite and its benefits, test suite
minimization problem.

Efficient Test Suite Management:

Minimizing the Test Suite:

Software testing is most expensive phase of development. It becomes unfeasible to execute all
the test cases. Test case minimization techniques are used to minimize the testing cost in terms of
execution time, resources etc. The purpose of test case minimization is to generate representative
set from test suite that satisfy all the requirements as original test suite with minimum number of
test cease. Main purpose of test case minimization techniques is to remove test cases that become
redundant and obsolete over time. Several techniques have been purposed in literature. These
techniques can be categorized as Heuristics, Genetic Algorithm, Integer Linear Programming
based techniques. This paper presents a survey on the work that has been done in test case
minimization.

Benefit of Test Suite:

Automation means taking your manual tests and automating them using a tool and
language of your choice. As a tester, I have also looked into automated testing and to make
things clearer for us, I have drawn up a list of points as to why I feel projects will benefit from
automated testing:

1. Save Time – This is one of the biggest benefits for me, especially when it comes to
regression testing. As we all know, regression testing is the retesting of the application
when new features have been introduced or a change is made to an existing feature that

Prof. Sunil A. Sushir Department of Information Technology

Subject: STQA Academic Year: 2017-18
has been previously tested (the change can be as a result of change request, defect fix,
refactoring). The aim of regression testing is to ensure the application still works as
expected and in order for us to verify this, we need to run all test scripts associated with
the change. There is a risk here, that due to time constraints we may not run all tests
associated to the change which may result in undiscovered defects.
2. Speed – As automated tests are run by tools, these are run much faster than human users
which adds to the first benefit of saving time.
3. Repeatability – the same tests can be re-run in exactly the same manner eliminating the
risk of human errors such as testers forgetting their exact actions, intentionally omitting
steps from the test scripts, missing out steps from the test script, all of which can result in
either defects not being identified or the reporting of invalid bugs (which can again, be
time consuming for both developers and testers to reproduce)
4. Maintenance of the test suite – From my experience I have found myself in situations
where the test suite has become out of date – new functionality has been introduced or
existing features have been changed in the way they work and the test cases are no longer
up to date as the tester(s) has had no time to go back and update the test scripts. When
tests are automated and run after each build, those that are out of date will fail hence,
forcing the tester to go back and fix the test script – this process ensures the test scripts
are kept up to date and quality of the software is maintained
5. Reusable – the automated tests can be reused on different versions of the software, even
if the interface changes.
6. Increase Coverage – testers can create a test suite with tests to cover every feature
within the application.
7. Cost Reduction – This can be of benefit when the number of resources required for
regression testing are reduced.

Prof. Sunil A. Sushir Department of Information Technology

Subject: STQA Academic Year: 2017-18
Test Suite Minimization Problem:

1) Test suite is a container that has a set of tests which helps testers in executing and reporting
the test execution status. It can take any of the three states namely Active, In progress and
completed.
2) A Test case can be added to multiple test suites and test plans. After creating a test plan, test
suites are created which in turn can have any number of tests.
3) Test suites are created based on the cycle or based on the scope. It can contain any type of
tests, viz - functional or Non-Functional.
4) Test suite structure is an essential items of a system test plan
5) Test Suite Structure:

• Detail test groups and subgroups are outlined in the test suite structure section based on
the test categories identified in the test approach section.
• Test objectives are created for each test group and subgroup based on the system
requirements and functional specification.
• If some existing test cases, automated or manual, need to be run as regression tests, those
test cases must be included in the test suite.

Prof. Sunil A. Sushir Department of Information Technology

Subject: STQA Academic Year: 2017-18

Fig. Test Suit Minimization

6) If each test case represents a piece of a scenario, such as the elements that simulate a
completing a transaction, use a test suite. For instance, a test suite might contain four test cases,
each with a separate test script:
Test case 1: Login
Test case 2: Add New Products
Test case 3: Checkout
Test case 4: Logout
7) Test suites can identify gaps in a testing effort where the successful completion of one test
case must occur before you begin the next test case. For instance, you cannot add new products
to a shopping cart before you successfully log in to the application. When you run a test suite in
sequential mode, you can choose to stop the suite execution if a single test case does not pass.
Stopping the execution is useful if running a test case in a test suite depends on the success of
previous test cases.

Prof. Sunil A. Sushir Department of Information Technology

Subject: STQA Academic Year: 2017-18
8) Test suites often need to adapt to the software that it is intended to test. The core software
changes and grows, and as such, its test suite also needs to change and grow. However, the test
suites can often grow so large as to be unmaintainable.
9) Maintaining the suite becomes too expensive. Once we have a suite in place, we have to
maintain it. As the size of the suite grows, the amount of maintenance of existing test grows. It
grows in proportion to the number of tests in the suite and the rate of change of the underlying
software being tested. As the amount of maintenance grows, so does its expense.
10) We can developed techniques to assist in the maintenance of these test suites, specifically by
allowing for test-suite reduction (while preserving coverage adequacy) and test-suite
prioritization.

Prof. Sunil A. Sushir Department of Information Technology