Software Testing

Books:

1. Software testing: Principles and practices

By: Srinivasan Desikan and Gopalaswamy Ramesh,
Pearson Education

2. Software testing in the real world

By: Edward Kit, Pearson Education

3. System validation and verification

By: Jeffrey O. Grady, CRC Press

4. Software testing: A Craftsman Approach

By: Paul C. Jorgensen, CRC Press
1
 Overview
In a typical testing project, someone gives you a program and
asks you to test it.
• The program may (or may not) come with a specification and
documentation.
Documents vary in accuracy and completeness.
 You create tests, run them, and report the progress and
results of your work.
 New versions of the program come to you. You might
reuse some old tests, check whether bugs have been
fixed, and (or) try new tests.

Eventually, the product is either cancelled or put into production.

2
 What is testing?

A technical investigation of the product under test conducted to

provide stakeholders with quality-related information.

Defining Testing

A technical
 means including experimentation, logic, mathematics, models,
tools (testing-support programs), and tools (measuring instruments,
event generators, etc.)

investigation
 An organized and thorough search for information.
 This is an active process of inquiry. We ask hard questions
(run hard test cases) and look carefully at the results.

3
of the product under test
• The product includes the data, the documentation, the
hardware, whatever the customer gets. If it doesn’t all work
together, it doesn’t work.

stakeholders
 Someone who has a vested interest in the success of the testing e
 Someone who has a vested interest in the success of the product.

with quality-related information

 The information of interest is often about the presence
(or absence) of bugs.
 Other types of information are sometimes more vital to the
stakeholders

4
Information Objectives
Find important bugs, to get them fixed

Assess the quality of the product

Help managers make release decisions

Block premature product releases

Help predict and control costs of product support

Check interoperability with other products

Find safe scenarios for use of the product

Assess conformance to specifications 5
 Certify the product meets a particular standard

Ensure the testing process meets accountability standards

Minimize the risk of safety-related lawsuits

Help clients improve product quality & testability

Help clients improve their processes

Evaluate the product for a third party

6
The six essentials of software testing:

Software Testing: Means by which people, methods,

measurements, tools and equipments are integrated to test the
software product.
1. The quality of the test process determines the success of
test effort:
Quality and effectiveness of software testing are determined
primarily by the quality of test processes used.

2. Prevent defect migration by using early life cycle testing

techniques:
More than half the errors are usually introduced at the
requirement phase.
Effective test program prevents the migration of errors from
any development phase to any subsequent phase.
7
3. The time for software testing tool is now:
Testing tool for every platform:
Capture/play back tools,
Tools for client/server,
Structural coverage, Etc.

4. A real person must take responsibility for improving the test

process:
Tools must be used with test process including effective test planning
and design.
To improve the testing process someone must plan and manage the
progress.

5. Testing is a professional discipline requiring trained, skilled

people:
It is not entry level job or stepping stone to other thing.
It is a proffession.
8
6. Cultivate a positive team attitude of creative
destruction:

Testers supply the information like:

Can we make this product fail to do what we expect it to do?

It is going to break as we can always find errors.

But is it breaking at reasonable boundaries?
Is it stressing or breaking acceptably or unacceptably,
given its criticality?
Do we know that we have covered all the important
possibilities?
After addressing all the questions only we can release the
product 9
 Objective of testing
• To show that a product do what it should’t
do.
• To show that a product does’t do what it
should

10
The State of art and the state of practice

 SW testing yet to become fundamental component of

university syllabus

Training in the industry is there but not properly defined

Published papers giving state of art methods are unproven in

the field (many well proven methods are unused in industry)
and omit real world dimension like return on investment.

11
History Aspects:

Early days:
Testing was “debugging” (fix the bugs), performed by
developers.
Few dedicated resources for testing that too were very late in
the development process.

1957:
Testing distinguished from debugging.
Testing was still an after development activity.

1970:
Software engineering term used more often.
1972:
First formal conference on testing was held at the University
of North California.
12
1979:
 Myers defined testing as “The process of executing a
program with the intent of finding errors”.
 BUT IN THE REAL WORLD OF INDUSTRY: Testing got
dumped when the schedules and the budget becomes tight.
 Testing –too late to be done properly

1980:
“Quality” was buzzword
Various groups formed some standards like:

IEEE (Institute of Electrical and Electronics engineers)

ANSI (American National Standards Institute)
ISO (International Standards Organization)

1990:
Testing tools came.
13
Where are we now?
1980:
SEI(S/w Engineering Institute) at Carnegie Mellon University
developed CMM for US dept. of defense. 1987: 5 level CMM was
proposed to measure level of s/w maturity.
75% of world s/w development is at level 1.
Very few are at level 5.

How should testing be positioned?

Early testing is the key to success of testing effort.
Testing has its own life cycle.

Testing is still subordinate to development.

Result: Little or no action based on tester’s input.
14
Now things are improving.
Establishing a practical perspective

Ultimate goal of testing: Quality Maintenance and Satisfaction

of customers.

What and why?

Mistake: A human action that produces incorrect result.
(Mistake produces fault or bug)
Fault: An incorrect step, process or data definition in a computer
program. The outgrowth of a mistake. (Potentially leads to
a failure)
Failure: An incorrect result. The result of the fault.

Error: The amount by which the result is incorrect

15
Where are errors?
Errors are concentrated in earlier stages of development
process. (Fig. 1.1)

Design, 27%

Design
Code
Requirements, Other
56% Code, 7% Requirements

Other, 10%

Fig. 1.1 Defect Distribution

16
The cost of errors:
All are costly.
Most expensive: Undetected errors and errors detected late.

Undetected: Migrate to other stages and cause failure in the field fin

Historical definitions of testing:

2. Establishing confidence that a program does what it is
supposed to do (Hetzel, 1973).

2. The process of executing a program or system with the intent of

finding error (myers 1979)
3. Detecting specification errors and deviations from the
specification.
4. Any activity aimed at evaluating an attribute or capability of a
program or system (hetzel, 1983)
17
5. The measurement of software quality (hetzel, 1983)
6. The process of evaluating a program or system.
• Verifying that a system satisfies its specified requirement or
identifying difference between expected and actual results.
• Confirming that a program performs its intended function
correctly.

The IEEE/ANSI definitions of testing

3. The process of operating a system or component under specified
conditions, observing or recording the result and making an
evaluation of some aspect of the system or component.
2. The process of analyzing a software item to detect the difference
between existing and required conditions and to evaluate the
features of the software items.

18