Software

Engineering

Softwa


re
Testing
Version 1.0


“ Te stin g ca n o n ly sh o w th e p re se n ce o f e rro rs , n o t
th e ir a b se n ce s . ” - E d sg e r D ijk stra

Rushdi Shams, Lecturer, Dept of CSE, KUET 1

The testing process
Component testing
 Testing of individual program components;
 Usually the responsibility of the component developer
(except sometimes for critical systems);
 Tests are derived from the developer’s experience.
System testing
 Testing of groups of components integrated to create a
system or sub-system;
 The responsibility of an independent testing team;
 Tests are based on a system specification.

Rushdi Shams, Lecturer, Dept of CSE, KUET 2

Testing phases

Component System
testing testing

Software developer Independent testing team

Rushdi Shams, Lecturer, Dept of CSE, KUET 3

Testing process goals
Validation testing
 To demonstrate to the developer and the system
customer that the software meets its requirements;
 A successful test shows that the system operates as
intended.
Defect testing
 To discover faults or defects in the software where its
behaviour is incorrect or not in conformance with its
specification;
 A successful test is a test that makes the system
perform incorrectly and so exposes a defect in the
system.

Rushdi Shams, Lecturer, Dept of CSE, KUET 4

The software testing
process

Test Test Test Test

cases data results reports

Design test Prepare test Run program Compare results

cases data with test da
ta to test cases

Rushdi Shams, Lecturer, Dept of CSE, KUET 5

Testing policies
Only exhaustive testing can show a program is
free from defects. However, exhaustive testing
is impossible,
Testing policies define the approach to be used in
selecting system tests:
 All functions accessed through menus should be
tested;
 Combinations of functions accessed through the same
menu should be tested;
 Where user input is required, all functions must be
tested with correct and incorrect input.

Rushdi Shams, Lecturer, Dept of CSE, KUET 6

System testing
Involves integrating components to create a
system or sub-system.
May involve testing an increment to be
delivered to the customer.
Two phases:
Integration testing - the test team have access
to the system source code. The system is
tested as components are integrated.
Release testing - the test team test the
complete system to be delivered as a black-
box.

Rushdi Shams, Lecturer, Dept of CSE, KUET 7

Integration testing
Involves building a system from its components
and testing it for problems that arise from
component interactions.
Top-down integration
 Develop the skeleton of the system and populate it with
components.
Bottom-up integration
 Integrate infrastructure components then add
functional components.
To simplify error localisation, systems should be
incrementally integrated.

Rushdi Shams, Lecturer, Dept of CSE, KUET 8

Incremental integration
testing
A T1

T1
A
T1 T2
A B
T2

T2 B T3

T3
B C
T3 T4
C
T4

D T5

Test sequence 1 Test sequence 2 Test sequence 3

Rushdi Shams, Lecturer, Dept of CSE, KUET 9

Regression Testing
When incremental integration testing is done
automatically (with software tools), the test is
called regression testing.
www.junit.org


Rushdi Shams, Lecturer, Dept of CSE, KUET 10

Rushdi Shams, Lecturer, Dept of CSE, KUET 11
Testing approaches
Architectural validation
 Top-down integration testing is better at discovering
errors in the system architecture.
System demonstration
 Top-down integration testing allows a limited
demonstration at an early stage in the development.
Test implementation
 Often easier with bottom-up integration testing.
Test observation
 Problems with both approaches. Extra code may be
required to observe tests.

Rushdi Shams, Lecturer, Dept of CSE, KUET 12

Release testing
The process of testing a release of a system
that will be distributed to customers.
Primary goal is to increase the supplier’s
confidence that the system meets its
requirements.
Release testing is usually black-box or
functional testing
Based on the system specification only;
Testers do not have knowledge of the system
implementation.

Rushdi Shams, Lecturer, Dept of CSE, KUET 13

Black-box testing
Inputs causing
anomalous
Input test da
ta Ie behaviour

System

Outputs which reveal

the presence of
Output testesults
r Oe defects

Rushdi Shams, Lecturer, Dept of CSE, KUET 14

Acceptance Testing
When release testing is done in presence of
clients, that time, s/he may confirm whether
s/he accepts the release or not.
This type of release testing is called
acceptance testing

Rushdi Shams, Lecturer, Dept of CSE, KUET 15

Testing guidelines
Testing guidelines are hints for the testing team to
help them choose tests that will reveal defects in
the system
 Choose inputs that force the system to generate all
error messages;
 Design inputs that cause buffers to overflow;
 Repeat the same input or input series several times;
 Force invalid outputs to be generated;
 Force computation results to be too large or too small.

Rushdi Shams, Lecturer, Dept of CSE, KUET 16

Performance testing
Part of release testing may involve testing the
emergent properties of a system, such as
performance and reliability.
Performance tests usually involve planning a
series of tests where the load is steadily
increased until the system performance
becomes unacceptable.

Rushdi Shams, Lecturer, Dept of CSE, KUET 17

Stress testing
Exercises the system beyond its maximum design
load. Stressing the system often causes defects
to
come to light.
Stressing the system test failure behaviour..
Systems should not fail catastrophically. Stress
testing checks for unacceptable loss of service or
data.
Stress testing is particularly relevant to distributed
systems that can exhibit severe degradation as a
network becomes overloaded.

Rushdi Shams, Lecturer, Dept of CSE, KUET 18

Sanity Test vs Smoke Test
Sanity test determines whether it is reasonable
to proceed with further test.
Smoke test determines whether it is possible to
proceed with further test.
A software smoke test determines whether the
program launches and whether its interfaces
are accessible and responsive (for example,
the responsiveness of a web page or an input
button).
If the smoke test fails, it is impossible to
conduct a sanity test.  
Rushdi Shams, Lecturer, Dept of CSE, KUET 19
Sanity Test Vs Smoke Test
sanity test exercises the smallest subset of
application functions needed to determine
whether the application logic is generally
functional and correct (for example, an
interest rate calculation for a financial
application).
If the sanity test fails, it is not reasonable to
attempt more rigorous testing.
Both are ways to avoid wasting time and effort
by quickly determining whether an
application is too flawed to merit any rigorous
testing. Rushdi Shams, Lecturer, Dept of CSE, KUET 20
Testing scenario
A student in Scotland is studying American History and has been asked to write a paper
on ŌFrontier mentality in the American West from 1840 to 1880Õ.To do this, she needs to
find sources from a range of libraries. She logs on to the LIBSYS system and uses the
search facility to discover if she can access original documents from that time. She
discovers sources in various US university libraries and downloads copies of some of
these. However, for one document, she needs to have confirmation from her university
that she is a genuine student and that use is for non-commercial purposes. The student
then uses the facility in LIBSYS that can request such permission and registers her
request. If granted, the document will be downloaded to the registered libraryÕs server
and printed for her. She receives a message from LIBSYS telling her that she will receive
an e-mail message when the printed document is available for collection.

Rushdi Shams, Lecturer, Dept of CSE, KUET 21

System tests

1 . T es t the login mec hanism us ing c orre ct and in co rrect logins to che c k
tha t v alid users are a c cep ted and inv alid us ers are reje cted.
2 . T es t the search fa cility u sing d if fere nt queries a gains t known sources to
che ck tha t the se arch me cha nism is ac tua l y finding do cu me nts.
3 . T es t the sy st em pre se ntation fa c liity to ch eck th at in forma tion about
do cu me nts is displayed prop erly.
4 . T es t the me chanism ot requ es t perm ission for downlo ading.
5 . T es t the e-mail respons e indic a itng that the downlo aded doc um en t is
ava ila ble .

Rushdi Shams, Lecturer, Dept of CSE, KUET 22