UNIT -IV

TESTING AND MAINTENANCE

1
Software Testing

Testing is the process of

exercising a program with the
specific intent of finding errors
prior to delivery to the end
user.

2
What Testing Shows
errors
requirements conformance

performance

an indication
of quality

3
 Strategic Approach
• To perform effective testing, you should conduct effective technical
reviews. By doing this, many errors will be eliminated before testing
commences.
• Testing begins at the component level and works "outward" toward
the integration of the entire computer-based system.
• Different testing techniques are appropriate for different software
engineering approaches and at different points in time.
• Testing is conducted by the developer of the software and (for large
projects) an independent test group.
• Testing and debugging are different activities, but debugging must
be accommodated in any testing strategy.

4
 V&V

• Verification refers to the set of tasks that

ensure that software correctly implements a
specific function.
• Validation refers to a different set of tasks that
ensure that the software that has been built is
traceable to customer requirements. Boehm
[Boe81] states this another way:
– Verification: "Are we building the product right?"
– Validation: "Are we building the right product?"

5
Who Tests the Software?

developer independent tester

Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality

6
 Testing Strategy
System engineering

Analysis modeling
Design modeling

Code generation Unit test

Integration test
Validation test

System test

7
 Testing Strategy
• We begin by ‘testing-in-the-small’ and move toward
‘testing-in-the-large’
• For conventional software
– The module (component) is our initial focus
– Integration of modules follows
• For OO software
– our focus when “testing in the small” changes from an
individual module (the conventional view) to an OO class
that encompasses attributes and operations and implies
communication and collaboration
8
•
Strategic Issues
Specify product requirements in a quantifiable manner long before testing
commences.
• State testing objectives explicitly.
• Understand the users of the software and develop a profile for each user
category.
• Develop a testing plan that emphasizes “rapid cycle testing.”
• Build “robust” software that is designed to test itself
• Use effective technical reviews as a filter prior to testing
• Conduct technical reviews to assess the test strategy and test cases
themselves.
• Develop a continuous improvement approach for the testing process.

9
Unit Testing

module
to be
tested

results

software
engineer
test cases

10
Unit Testing
module
to be
tested
interface
local data structures
boundary conditions
independent paths
error handling paths

test cases
11
Unit Test Environment
driver
interface
local data structures

Module boundary conditions

independent paths
error handling paths

stub stub

test cases

RESULTS
12
Integration Testing Strategies
Options:
• the “big bang” approach
• an incremental construction strategy

13
Top Down Integration
A
top module is tested with
stubs

B F G

stubs are replaced one at

a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D E

14
Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E

cluster
15
Sandwich Testing
A
Top modules are
tested with stubs

B F G

Worker modules are grouped into

builds and integrated
D E

cluster
16
 Regression Testing
• Regression testing is the re-execution of some subset of tests that have
already been conducted to ensure that changes have not propagated
unintended side effects
• Whenever software is corrected, some aspect of the software
configuration (the program, its documentation, or the data that
support it) is changed.
• Regression testing helps to ensure that changes (due to testing or for
other reasons) do not introduce unintended behavior or additional
errors.
• Regression testing may be conducted manually, by re-executing a
subset of all test cases or using automated capture/playback tools.

17
 Smoke Testing
• A common approach for creating “daily builds” for product software
• Smoke testing steps:
– Software components that have been translated into code are integrated
into a “build.”
• A build includes all data files, libraries, reusable modules, and engineered
components that are required to implement one or more product functions.
– A series of tests is designed to expose errors that will keep the build from
properly performing its function.
• The intent should be to uncover “show stopper” errors that have the highest
likelihood of throwing the software project behind schedule.
– The build is integrated with other builds and the entire product (in its
current form) is smoke tested daily.
• The integration approach may be top down or bottom up.

18
High Order Testing
• Validation testing
– Focus is on software requirements
• System testing
– Focus is on system integration
• Alpha/Beta testing
– Focus is on customer usage
• Recovery testing
– forces the software to fail in a variety of ways and verifies that recovery is properly
performed
• Security testing
– verifies that protection mechanisms built into a system will, in fact, protect it from
improper penetration
• Stress testing
– executes a system in a manner that demands resources in abnormal quantity,
frequency, or volume
• Performance Testing
– test the run-time performance of software within the context of an integrated system

19
Debugging: A Diagnostic Process

20
The Debugging Process

21
 Debugging Effort
time required
to diagnose the
symptom and
time required determine the
to correct the error cause
and conduct
regression tests

22
 Symptoms & Causes
symptom and cause may be
geographically separated

symptom may disappear when

another problem is fixed

cause may be due to a

combination of non-errors

cause may be due to a system

or compiler error

symptom cause may be due to

assumptions that everyone
cause believes

symptom may be intermittent

23
Consequences of Bugs
infectious
damage

catastrophic
extreme
serious
disturbing

annoying
mild

Bug Type

Bug Categories: function-related bugs,

system-related bugs, data bugs, coding bugs,
design bugs, documentation bugs, standards
violations, etc.
24
Debugging Techniques

brute force / testing

backtracking

induction

deduction

25
 Correcting the Error
• Is the cause of the bug reproduced in another part of the program? In many
situations, a program defect is caused by an erroneous pattern of logic that
may be reproduced elsewhere.
• What "next bug" might be introduced by the fix I'm about to make? Before the
correction is made, the source code (or, better, the design) should be
evaluated to assess coupling of logic and data structures.
• What could we have done to prevent this bug in the first place? This question is
the first step toward establishing a statistical software quality assurance
approach. If you correct the process as well as the product, the bug will be
removed from the current program and may be eliminated from all future
programs.

26
 Final Thoughts
• Think -- before you act to correct
• Use tools to gain additional insight
• If you’re at an impasse, get help from
someone else
• Once you correct the bug, use regression
testing to uncover any side effects

27
Testabilit
y
• Operability—it operates cleanly
• Observability—the results of each test case are readily
observed
• Controllability—the degree to which testing can be
automated and optimized
• Decomposability—testing can be targeted
• Simplicity—reduce complex architecture and logic to
simplify tests
• Stability—few changes are requested during testing
• Understandability—of the design

28
What is a “Good” Test?
• A good test has a high probability of
finding an error
• A good test is not redundant.
• A good test should be “best of breed”
• A good test should be neither too
simple nor too complex

29
 Internal and External Views
• Any engineered product (and most other things)
can be tested in one of two ways:
– Knowing the specified function that a product has
been designed to perform, tests can be conducted that
demonstrate each function is fully operational while
at the same time searching for errors in each function;
– Knowing the internal workings of a product, tests can
be conducted to ensure that "all gears mesh," that is,
internal operations are performed according to
specifications and all internal components have been
adequately exercised.

30
Test Case Design
"Bugs lurk in corners
and congregate at
boundaries ..."
Boris Beizer

OBJECTIVE to uncover errors

CRITERIA in a complete manner

CONSTRAINT
with a minimum of effort and time

31
 Exhaustive Testing

loop < 20 X

14
There are 10 possible paths! If we execute one
test per millisecond, it would take 3,170 years to
test this program!!
32
 Selective Testing
Selected path

loop < 20 X

33
Software Testing
white- black-
box box
methods
methods

Methods

Strategies

34
White-Box
Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...
35
Why Cover?
logic errors and incorrect assumptions
are inversely proportional to a path's
execution probability

we oftenbelievethat a path is not

likely to be executed; in fact, reality is
often counter intuitive

typographical errors are random; it's

likely that untested paths will contain
some

36
Basis Path Testing
First, we compute the cyclomatic
complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

37
 Cyclomatic
Complexity
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.

modules

V(G)

modules in this range are

more error prone
38
Basis Path Testing
1
Next, we derive the
independent paths:

2 Since V(G) = 4,
there are four paths
3 Path 1: 1,2,3,6,7,8
4
5 6
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive test
7 cases to exercise these
paths.

39
Basis Path Testing Notes
you don't need a flow chart,
but the picture will help when
you trace program paths

count each simple logical test,

compound tests count as 2 or
more

basis path testing should be

applied to critical modules

40
 Deriving Test Cases
• Summarizing:
– Using the design or code as a foundation,
draw a corresponding flow graph.
– Determine the cyclomatic complexity of the
resultant flow graph.
– Determine a basis set of linearly independent
paths.
– Prepare test cases that will force execution of
each path in the basis set.

41
 Graph
Matrices
• A graph matrix is a square matrix whose size
(i.e., number of rows and columns) is equal to
the number of nodes on a flow graph
• Each row and column corresponds to an
identified node, and matrix entries correspond
to connections (an edge) between nodes.
• By adding a link weight to each matrix entry,
the graph matrix can become a powerful tool
for evaluating program control structure
during testing

42
 Control Structure Testing
• Condition testing — a test case design method
that exercises the logical conditions contained
in a program module
• Data flow testing — selects test paths of a
program according to the locations of
definitions and uses of variables in the
program

43
 Data Flow Testing
• The data flow testing method [Fra93] selects test paths of a program
according to the locations of definitions and uses of variables in the
program.
– Assume that each statement in a program is assigned a unique
statement number and that each function does not modify its
parameters or global variables. For a statement with S as its statement
number
• DEF(S) = {X | statement S contains a definition of X}
• USE(S) = {X | statement S contains a use of X}
– A definition-use (DU) chain of variable X is of the form [X, S, S'], where S
and S' are statement numbers, X is in DEF(S) and USE(S'), and the
definition of X in statement S is live at statement S'

44
 Loop Testing

Simple
loop
Nested
Loops

Concatenated
Loops Unstructured
Loops
45
Loop Testing: Simple Loops
Minimum conditions—Simple Loops
1. skip the loop entirely
2. only one pass through the loop
3. two passes through the loop
4. m passes through the loop m < n
5. (n-1), n, and (n+1) passes through
the loop
where n is the maximum number
of allowable passes

46
Loop Testing: Nested Loops
Nested Loops
Start at the innermost loop. Set all outer loops to their
minimum iteration parameter values.
Test the min+1, typical, max-1 and max for the
innermost loop, while holding the outer loops at their
minimum values.
Move out one loop and set it up as in step 2, holding all
other loops at typical values. Continue this step until
the outermost loop has been tested.
Concatenated Loops
If the loops are independent of one another
then treat each as a simple loop
else* treat as nested loops
endif*
for example, the final loop counter value of loop 1 is
used to initialize loop 2.
47
Black-Box Testing
requirements

output

input events

48
Black-Box Testing
• How is functional validity tested?
• How is system behavior and performance tested?
• What classes of input will make good test cases?
• Is the system particularly sensitive to certain input values?
• How are the boundaries of a data class isolated?
• What data rates and data volume can the system tolerate?
• What effect will specific combinations of data have on
system operation?

49
 Graph-Based Methods
To understand object Directed link object
the objects that #1 (link weight) #2

are modeled in
software and Undirected link
Node weight
(value
the Parallel links
)

relationships object
#
that connect 3

these objects (a)

In this context, we
new menu select generates document
consider the term
file (generation time  1.0 sec) window
“objects” in the
broadest possible allows editing
context. It of Attributes:
is represented as
encompasses data
contains
objects, traditional
document background color: white
components text color: default color
tex
(modules), and t or preferences
object-oriented
(b)
elements of 50
computer software.
Equivalence Partitioning

user output FK
queries formats input
mouse
picks data
prompts

51
Sample Equivalence Classes
Valid data
user supplied commands
responses to system prompts
file names
computational data
physical parameters
bounding values
initiation values
output data formatting
responses to error messages
graphical data (e.g., mouse picks)

Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place
52
 Boundary Value Analysis

user output FK
queries formats input
mouse
picks data
prompts

output
input domain domain
53
 Comparison Testing
• Used only in situations in which the reliability of
software is absolutely critical (e.g., human-
rated systems)
– Separate software engineering teams develop
independent versions of an application using the
same specification
– Each version can be tested with the same test data
to ensure that all provide identical output
– Then all versions are executed in parallel with real-
time comparison of results to ensure consistency

54
 Orthogonal Array Testing
• Used when the number of input parameters
is small and the values that each of the
parameters may take are clearly bounded

Z Z

Y Y
X X
One input item at a time L9 orthogonal array

55
 Software Maintenance
• Software is released to end-users, and
– within days, bug reports filter back to the software engineering
organization.
– within weeks, one class of users indicates that the software must
be changed so that it can accommodate the special needs of their
environment.
– within months, another corporate group who wanted nothing to
do with the software when it was released, now recognizes that
it may provide them with unexpected benefit. They’ll need a few
enhancements to make it work in their world.
• All of this work is software maintenance

56
 Maintainable Software
• Maintainable software exhibits effective modularity
• It makes use of design patterns that allow ease of understanding.
• It has been constructed using well-defined coding standards and
conventions, leading to source code that is self-documenting and
understandable.
• It has undergone a variety of quality assurance techniques that have
uncovered potential maintenance problems before the software is
released.
• It has been created by software engineers who recognize that they
may not be around when changes must be made.
– Therefore, the design and implementation of the software must “assist” the
person who is making the change

57
 Software Supportability
• “the capability of supporting a software system over its whole
product life.
– This implies satisfying any necessary needs or requirements, but also
the provision of equipment, support infrastructure, additional software,
facilities, manpower, or any other resource required to maintain the
software operational and capable of satisfying its function.” [SSO08]
• The software should contain facilities to assist support personnel
when a defect is encountered in the operational environment (and
make no mistake, defects will be encountered).
• Support personnel should have access to a database that contains
records of all defects that have already been encountered—their
characteristics, cause, and cure.

58
 Reengineering

Business
processes

IT Software
systems Reengineering applications

59
 Business Process Reengineering
• Business definition. Business goals are identified within the
context of four key drivers: cost reduction, time reduction,
quality improvement, and personnel development and
empowerment.
• Process identification. Processes that are critical to achieving
the goals defined in the business definition are identified.
• Process evaluation. The existing process is thoroughly
analyzed and measured.
• Process specification and design. Based on information
obtained during the first three BPR activities, use-cases are
prepared for each process that is to be redesigned.
• Prototyping. A redesigned business process must be prototyped
before it is fully integrated into the business.
• Refinement and instantiation. Based on feedback from the
prototype, the business process is refined and then instantiated
within a business system.

60
 Business Process Reengineering
Business
definition

Refinement
&
Instantiation

Prototyping

Process
identification
Process
Specification
and Design

Process
Evaluation

61
 BPR Principles
• Organize around outcomes, not tasks.
• Have those who use the output of the process perform the
process.
• Incorporate information processing work into the real work
that produces the raw information.
• Treat geographically dispersed resources as though they were
centralized.
• Link parallel activities instead of integrated their results.
When different
• Put the decision point where the work is performed, and build
control into the process.
• Capture data once, at its source.

62
Software Reengineering
Forward inventory
engineering analysis

Data document
restructuring restructuring

code reverse
restructuring engineering

63
 Inventory
Analysis
• build a table that contains all applications
• establish a list of criteria, e.g.,
– name of the application
– year it was originally created
– number of substantive changes made to it
– total effort applied to make these changes
– date of last substantive change
– effort applied to make the last change
– system(s) in which it resides
– applications to which it interfaces, ...
• analyze and prioritize to select candidates for
reengineering

64
 Document Restructuring
• Weak documentation is the trademark of many legacy
systems.
• But what do we do about it? What are our options?
• Options …
– Creating documentation is far too time consuming. If the system
works, we’ll live with what we have. In some cases, this is the
correct approach.
– Documentation must be updated, but we have limited resources.
We’ll use a “document when touched” approach. It may not be
necessary to fully redocument an application.
– The system is business critical and must be fully redocumented.
Even in this case, an intelligent approach is to pare documentation
to an essential minimum.

65
 Reverse Engineering
dirty source code

restructure
code

clean source code processing

extract
abstractions interface

initial specification database

refine
&
simplify

final specification

66
 Code Restructuring
• Source code is analyzed using a restructuring tool.
• Poorly design code segments are redesigned
• Violations of structured programming constructs
are noted and code is then restructured (this can
be done automatically)
• The resultant restructured code is reviewed and
tested to ensure that no anomalies have been
introduced
• Internal code documentation is updated.

67
 Data Restructuring
• Unlike code restructuring, which occurs at a relatively low level
of abstraction, data structuring is a full-scale reengineering
activity
• In most cases, data restructuring begins with a reverse
engineering activity.
– Current data architecture is dissected and necessary data models are
defined (Chapter 9).
– Data objects and attributes are identified, and existing data
structures are reviewed for quality.
– When data structure is weak (e.g., flat files are currently
implemented, when a relational approach would greatly simplify
processing), the data are reengineered.
• Because data architecture has a strong influence on program
architecture and the algorithms that populate it, changes to the
data will invariably result in either architectural or code-level
changes.

68
 Forward
Engineering
1. The cost to maintain one line of source code may be 20 to 40
times the cost of initial development of that line.
2. Redesign of the software architecture (program and/or data
structure), using modern design concepts, can greatly facilitate
future maintenance.
3. Because a prototype of the software already exists, development
productivity should be much higher than average.
4. The user now has experience with the software. Therefore, new
requirements and the direction of change can be ascertained with
greater ease.
5. CASE tools for reengineering will automate some parts of the
job.
6. A complete software configuration (documents, programs and
data) will exist upon completion of preventive maintenance.

69
 Economics of Reengineering-I
• A cost/benefit analysis model for reengineering
has been proposed by Sneed [Sne95]. Nine
parameters are defined:
• P1 = current annual maintenance cost for an application.
• P2 = current annual operation cost for an application.
• P3 = current annual business value of an application.
• P4 = predicted annual maintenance cost after reengineering.
• P5 = predicted annual operations cost after reengineering.
• P6 = predicted annual business value after reengineering.
• P7 = estimated reengineering costs.
• P8 = estimated reengineering calendar time.
• P9 = reengineering risk factor (P9 = 1.0 is nominal).
• L = expected life of the system.

70
 Economics of Reengineering-II
• The cost associated with continuing maintenance of a candidate application
(i.e., reengineering is not performed) can be defined as
Cmaint = [P3 - (P1 + P2)] x L

• The costs associated with reengineering are defined using the following
relationship:
Creeng = [P6 - (P4 + P5) x (L - P8) - (P7 x P9)] `

• Using the costs presented in equations above, the overall benefit of

reengineering can be computed as
cost benefit = Creeng - Cmaint

71
 Coding Practice -1

Coding principles and concepts are

closely aligned programming style,
programming languages, and
programming methods.

72
 Coding Practice - 2
• Before you write one line of code, be sure
you:
• Understand of the problem you’re trying to solve.
• Understand basic design principles and concepts.
• Pick a programming language that meets the
needs of the software to be built and the
environment in which it will operate.
• Select a programming environment that provides
tools that will make your work easier.
• Create a set of unit tests that will be applied once
the component you code is completed.

73
 Coding Practice - 3
• As you begin writing code, be sure you:
• Constrain your algorithms by following structured programming
[Boh00] practice.
• Consider the use of pair programming
• Select data structures that will meet the needs of the design.
• Understand the software architecture and create interfaces that are
consistent with it.
• Keep conditional logic as simple as possible.
• Create nested loops in a way that makes them easily testable.
• Select meaningful variable names and follow other local coding
standards.
• Write code that is self-documenting.
• Create a visual layout (e.g., indentation and blank lines) that aids
understanding.

74
 Refactoring
Refactoring is:
•Restructuring (rearranging) code...
•...in a series of small, semantics-preserving
transformations (i.e. the code keeps working)......in
order to make the code easier to maintain and
modify
•we need to keep the code working
•we need small steps that preserve semantics
•we need to have unit tests to prove the code
works

75
 Contd…
You should refactor:
•Any time that you see a better way to do
things.
•―Better‖ means making the code easier to
understand and to modify in the future
•You can do so without breaking the code

76
 Contd…
Unit tests are essential for this
•switch statements are very rare in properly designed
object-oriented code
• Therefore, a switch statement is a simple and easily
detected ―”bad smell”
• Of course, not all uses of switch are bad
• A switch statement should not be used to distinguish
between various kinds of object
•There are several well-defined refactoring for this case
•The simplest is the creation of subclasses

77
78