MES 3053

Software Testing and Quality

Chapter 1
Software Verification & Validation
 Learning Objectives
 Explain the role of verification and validation in
software quality
 Describe the distinction between verification and
validation
 Describe the importance of verification and
validation

2
 Software Verification and Validation
• What is it?
• Did you see it at any of SDLC stage?
• Why do we need it?
• Can software be developed without it?
 The Definition of Quality
Webster’s dictionary defines quality as:

“the essential character of something,

inherent or distinguishing character, degree or
grade of excellence”
 An Introduction
 Software quality is something that everyone wants.
– Managers know that they want high quality.
– Software developers know that they want to produce a
quality product.
– Users insist that software work consistently and be
reliable.
 The objectives of software verification and validation are to
qualify the quality of software program.
 How to measure?
– By measuring the attributes and capabilities of the
software against expectation and applicable standards.
 Software Quality
In computer literature, there are two accepted
meanings of quality:
– Meeting the specifications (Producer View)
– Meeting customer’s requirement specifications
(Customer View)
Example: usability, portability and reusability
Usability The ease of use and learnability of software application
Portability The capability of the application to be executed across a diverse range of
hardware
Reusability The ability to transfer constructed software-component in one system into
another application.
 How to determine the quality?
The requirements must be measurable in order
to determine either the software meet the
requirement or not.
Quality demands requirement specifications in
sufficient detail in the software can be
quantitatively measured against those
specifications.
Quality cannot be achieved ONLY by assessing an
already completed product. Thus, it is important
to prevent quality defects from the beginning.
 Manage Quality through QA and QC
 QA – Quality Assurance
– Process oriented.
– Preventing bugs.
– Its activities are focused on the process used to create the
deliverable.
– makes sure you are doing the right things, the right way.
 QC – Quality Control
– Product oriented.
– Finding bugs.
– Its activities are focused on the deliverable itself.
– makes sure the results of what have done are what had been
expected.
 The costs of Prevention
 The sooner the defects are located and corrected, the less
costly it will be in the long run.
 Prevention costs consist of action taken to prevent defects
from occurring in the first place.
 However, if defects discovered after the product has been
released it can be very devastating as it may damage the
organization’s reputation or loss future sales.
 Therefore greatest payback is with prevention.
– Emphasis on prevention costs
– Reduces the number of defects that go to the customer
undetected.
– Improves product quality
– Reduces the cost of production and maintenance.
 Software Verification and Validation
 Software Verification and Validation (V&V) is a
software engineering practice that provides
confidence that the system was built adequately and
will meet the needs of the system.
 In other words, V&V is the process of checking that a
product, service, or system meets specifications and
that it fulfills its intended purpose.
 Verification and validation activities complement
each other.
 Verification takes place before validation, and not vice
versa.

10
 Software Verification
 The process of evaluating software to determine whether the products of a
given development phase satisfy the conditions imposed at the start of that
phase. [IEEE-STD-610].
 It is a quality assurance process that is used to evaluate whether or not a
product, service, or system complies with regulations, specifications, or
conditions imposed at the start of a development phase.
 Are we building the product right?
 This can be objective if the specification are sufficiently precise.
 Three approaches to verification:
 Experiment with the program (i.e. testing)
 Inspect the program (i.e. review)
 Reason about the program (i.e. formal specification)

11
 Software Validation
 The process of evaluating software during or at the
end of the development process to determine
whether it satisfies specified requirements. [IEEE-STD-
610]
 Does the software do what the user really required?
 Are we building the right product?
 Is a high-level checking activity.
 This is difficult to determine and involves subjective
judgments.
 It includes usability testing and user acceptance test.

12
 Verification & Validation

Concept Validation
System

Verification
 Verification or Validation?
 It depends on the property.
 For example: elevator response
…if a user press a request button at floor i, an available
elevator must arrive at floor i soon…
[SOON is a subjective quantity]
Thus, it can only be validated, but not verified.
… if a user press a request button at floor i, an available
elevator must arrive at floor i within 30 seconds…
[30 seconds is a precise quantity]
Thus, it can be verified.

14
The Role of V&V in Software Evolution
 The evolution of a software can be done in many ways,
depending on the development approach.
 The development approach determines the specific
intermediate documents/products to be created.
 For any given project, V&V objectives must be
identified for each of the products created.
 Type of products:
– Requirements
– Specifications
– Designs
– Implementation
– Changes

15
 The Role of V&V in Software Evolution
• V&V objectives:
– Correctness
– Consistency
– Necessity
– Sufficiency
– Performance

16
 The V&V Approach
 Five categories of V&V approaches are:
– Reviews
○ Walkthrough, technical review, inspection, audits
– Software testing
○ Module testing, integration testing, system testing, etc.
– Proof of correctness (program verification)
○ The use of formal specification in verifying the implementation.
– Simulation and prototyping
○ Technique for analyzing the expected behavior of a software.
– Requirements tracing.
○ Technique for ensuring that the product addresses each of its
requirements.
○ Technique: traceability matrix

17
 Review
 A process of meeting during which a work product, or set of work
products is presented to project personnel, managers, users,
customers, or other interested parties for comment or approval.
 The review team made up of a leader, a secretary and members.
 It can be formal or informal
– Formal review explicit and definite rules of procedures.
– Informal review has no predefined procedures.
 Three kinds of formal review are:
– Technical review
– Inspection
– Walkthrough
– Audits
 Proof of Correctness
Formal proof that attempts to demonstrate
logically that software is correct.
It is often difficult to justify because of the
additional effort required.
– Difficult to express software requirements and designs
in terms of mathematical notation. Also, mathematics
not appropriate for large applications.
Protocols and secure systems are two areas
where formal methods have been successfully
implemented.
 An Example of Formal Proof

Pre-condition

Post-condition

Definition:

An algorithm is correct if and only if it halts and the pre-condition combined

with the conditions arising from execution lead to the post-condition.

20
 Tracing
• Tracing is the act of establishing the
relationship between two or more products of
development process.
• For example, to establish the relationship
between a given requirement and the
function(s) that implements the requirement.
• To support traceability, all requirements and
components have to be identified.
An Example of Requirement
Traceability Matrix

22
 Software Testing
Software testing is a process of verifying and
validating that a software application meets the
requirement specifications and customer’s
requirement specifications.
 Software testing activities can and should be
carried out throughout the development.
Tests can be derived from requirements and
specifications, design artifacts or the source code.
Most types of tests can only be run after some
part of the software is implemented.
 Testing and debugging
Testing and debugging are distinct
processes.
Testing is concerned with establishing the
existence of defects in a program.
Debugging can occur because of a consequence
of a successful testing.
Debugging attempts to find the cause of an
error and fix it.
To find the cause of an error, the error must be
located first.
 The Debugging Process

Test Specification Test

results cases

Locate Design Repair Retest

error error repair error program
 The Process of Software V&V
Is a whole life-cycle process – V&V must be
applied at each stage in the software process.
Has two principal objectives:
– The discovery of defects in a system;
– The assessment of whether or not the system is
useful and able to be used in an operational
situation.

26
 The Goal of Software V&V
V&V should establish confidence that the
software is fit for purpose.
This does NOT mean completely free of
defects.
Rather, it must be good enough for its
intended use
– the type of use will determine the degree of
confidence that is needed.

27
 The V&V Confidence
Depends on system’s purpose, user
expectations and marketing environment
 Software function
○ The level of confidence depends on how critical the
software is to an organisation.
 User expectations
○ Users may have low expectations of certain kinds of
software.
 Marketing environment
○ Getting a product to market early may be more
important than finding defects in the program.

28
 Software V&V Tools
Tools are computer programs or software that
enhance the process of developing software.
Thus, software V&V tools are meant to enhance
the process of software V&V in particular.
Tools for reviewing:
– General administrative tools
– Static analyzers
– Configuration management tools
– Reverse engineering tools
 Software V&V Tools …cont.
Tools for testing
– Static analyzer
– Test case generator
– Test oracle generator
– Test harnesses
– Debuggers
– Coverage analyzers
– Test management tools
 The Importance of
V&V
Case #1 Toyota President Akio Toyoda surrounded by
reporters after a meeting at the Transport Ministry in
Tokyo on Feb. 9, 2010

Asked to comment in English during an "emergency" press conference held last Friday on the eight
million cars being recalled worldwide due to sticky gas pedals, Toyota president Akio Toyoda, taking
the public stand for the first time since the first recalls were announced in the U.S. a couple of weeks
ago, implored his audience to have faith that he could turn the company's turn of bad luck around.
"Please believe me," he said.
It's a request that's getting harder by the day. In his second hastily arranged meeting with the media
in five days broadcast live nationally, Toyoda said yesterday that the corporation, a source of Japanese
national pride which has gone unfathomably awry over the last several weeks, is recalling 437,000
2010 Prius and other hybrid models (plug-in Priuses, Lexus HS250h sedan and the SAI) around the
world because of a computer glitch in the braking system. Over 220,000 of the cars will be
recalled in Japan.
Read more: http://www.time.com/time/world/article/0,8599,1963313,00.html

31
The Importance of
V&V
Case #2

13 July 2010
• Share price fell by 4.2%
• RM 32.5 billion wiped off its value http://www.telegraph.co.uk/technology/apple/7888760/Apple-iPhone-4-
• Recall will cost RM 5.1 billion company-loses-6.5bn-after-bad-review.html

Apple just admitted that there's an issue with the iPhone 4's reception. Upon
investigation, they were stunned to find that the formula that they use to
calculate how many bars of signal strength to display is totally wrong. The
formula, in many instances, mistakenly displays 2 more bars than it should for a
given signal strength. For example, we sometimes display 4 bars when we
should be displaying as few as 2 bars. Users observing a drop of several bars
when they grip their iPhone in a certain way are most likely in an area with very
weak signal strength, but they don't know it because we are erroneously
displaying 4 or 5 bars. Their big drop in bars is because their high bars were
never real in the first place.
32
 The Importance of V&V
Case #3
MOSCOW (AFP) – A programming error may have caused
a rocket carrying three Russian navigation satellites to
fail to reach orbit, a Russian space agency source told the
RIA Novosti news agency on Monday. "A number of
specialists think that mistakes in the programming
of the onboard computer system of the Proton
rocket led to the engines sending the rocket too
high and onto a faulty trajectory," the source said.
The rocket carrying satellites for the Russian government
navigation system Glonass, failed to reach the correct
orbit Sunday after blasting off from Baikonur
cosmodrome, the Russian space agency said. The
satellites crashed into the Pacific Ocean near Hawaii, the
spacecraft corporation RKK Energiya said in a statement,
confirming earlier reports. "The Proton launch rocket
functioned abnormally, sending the three Glonass
satellites and the upper-stage booster rocket on the http://news.yahoo.com/s/afp/20101206/wl_asia_afp/r
wrong trajectory and they fell into the Pacific Ocean ussiaspacescience
1,500 kilometers northwest of Honolulu," the statement
said.

33
 Summary
In this chapter you have learned how to:
 Explain the role of verification and validation in software
quality
 Describe the distinction between verification and
validation
 Describe the importance of verification and validation

34