Software Quality and Testing

(SCSJ3343)
 Lecturers Details

Dr. Rohayanti Hassan(Sec. 03)

Webpage: http://comp.utm.my/rohayanti
Phone: 07-5538719
Email: rohayanti@utm.my
 Course Schedule
Sec. 3 - Perdana
Thursday (8.30 am 11.00 am) @ BK4, N28.

UTMSPACE
Refer schedule @ BK4, B11, FAB.
 Attendance Policies

42 hours of lectures over 14 weeks

80% of attendance is compulsory W4 (16 hrs), W7
(28hrs), W9 (36hrs)
Warning letter(s) will be issued
If less than 80% - not allowed to sit for final!
 Course Policy
Be PUNCTUAL !
Will not be allowed to enter the class after 10 minutes!
80% attendance rule applies.
NO Plagiarism for assignments and project
0 mark will be given.
Etiquette. Please keep all cell phones, pagers, and other
electronic devices (including laptops) turned OFF during
class. If your activities during class are deemed disruptive,
you will be asked to leave. And please dress appropriately.
 Course Learning Outcomes
i. Apply established software quality and testing concepts, the
fundamental test process, test approaches, and principles to
support test objectives.
ii. Design tests by using established techniques
iii. Execute test cases according to test design, and analyze and
report on the results of tests.
iv. Ability to do work effectively in a team and lead the team in
organising and illustrating the test activities throughout the
software testing life cycle

Please fill in the entry survey at the end of this lecture.

URL available on the e-learning
Assessment
 Mid-Semester Exam

Date & Day: Friday ( 3/11/17 )

Time: 9 am
Venue
Sec 01 BK1
Sec 02 BK 2
Sec 03 (UNSRI) BK3
SOFTWARE QUALITY ASSURANCE

INTRODUCTION
 Basic Questions

What is the point of SQA?

How necessary is SQA?

How good is software?

Is software production different from producing

other products? If so, how and why?
 Objectives

What is Software Quality?

Why is Software Quality important?
What is Software Quality Assurance ?
Software Quality factors
Elements of Software Quality Assurance
Development and quality plans
Process Maturity models
 What is Software?

Software is:
Computer programs, procedures, and possibly associated
documentation and data pertaining to the operation of a
computer system.
[IEEE]
 Differences between Software and
Other Products

1. Product Complexity
2. Product Visibility
3. Product Development Process
 "Software Crisis"

term coined by DoD years ago

Programmers
demand

supply

Time
Problem Today: complexity of problems addressed by
software has outpaced improvements in software
creation process
Quality Products
 Software vs. Quality

Software is:
Computer programs, procedures, and possibly
associated documentation and data pertaining to
the operation of a computer system.
Quality:
The combination of all 4 components is needed to
assure the quality of the development process as
well as the ensuing long years of maintenance.
Software: Good & Bad
 The Big Question

Q: How do we assure quality?

A: We have a good process.

 What is Quality?
Popular Views about Quality:
quality related to luxury, class and taste
quality is related to quality of life.
I know about quality when I see the product.

Professional Views about Quality:

Conformance to requirements.
Fitness for use.

Business Beam (Pvt.) Limited 21

 Two aspects of Software Quality
q Intrinsic product quality, often limited to defect
rate and reliability.

Q Customer Satisfaction, often includes product

quality and process quality.

Business Beam (Pvt.) Limited 22

 Pressman's definition of
"Software Quality"
Conformance to explicitly stated functional and
performance requirements, explicitly documented
development standards, and implicit characteristics
that are expected of all professionally developed
software.
 IEEE Definition of
"Software Quality"

1. The degree to which a system, component, or

process meets specified requirements.

2. The degree to which a system, component, or

process meets customer or user needs or
expectations.
 Role of Customer
Customers dont buy products, they buy assurances
that their expectations with the products would met.

You also, can only sell assurances in other words,

Quality.

Customer is in absolute position to judge the quality.

Business Beam (Pvt.) Limited 25

 Quality Assurance

Quality assurance is a system of activities designed to

ensure production that meets pre-established
requirements and standards.

Business Beam (Pvt.) Limited 26

 IEEE Definition of
"Software Quality Assurance"

1. A planned and systematic pattern of all actions

necessary to provide adequate confidence that an item
or product conforms to established technical
requirements.
2. A set of activities designed to evaluate the process by
which the products are developed or manufactured.
Contrast with quality control.
 SQ vs SQA

SQ: The degree to which a component, system or

process meets specified requirements and/or
user/customer needs and expectations. [IEEE]
SQA: Part of quality management focused on
providing confidence that quality requirements will
be fulfilled. [ISO 9000]
 Quality Assurance vs. Quality Control

Quality control and quality assurance activities serve

different objectives
Quality control activities are only a part of the total
range of quality control assurance activities
 More Definitions

"software error"
"software fault"
"software failure"

types of errors
1. code error
2. procedure error
3. documentation error
4. software data error
 Nine causes of Software Errors
Faulty requirements definition.
Client-developer communication failures.
Deliberate deviations from software requirements.
Logical design errors.
Coding errors.
Non-compliance with documentation and coding
instructions.
Shortcomings of the testing process.
User interface and procedure errors.
Documentation errors.
 When are defects introduced?

The majority of defects are introduced in earlier

phases.
Requirements are the top factor in a projects success or
failure.

Phase % of % effort to
defects fix
Requirements 56 82
Design 27 13
Code 7 1
Other 10 4
 Cost of fixing defects
Relative cost of fixing defects
benchmark: cost at requirements phase = 1

Phase found Cost ratio

Requirements 1
Design 35
Coding 10
Unit / integration testing 15 - 40
System / acceptance testing 30 70
Production 40 - 1000
 Importance of Software Quality
Software is a major component of computer systems (about 80% of the
cost) used for
Communication (e.g. phone system, email system).
Health monitoring.
Transportation (e.g. automobile, aeronautics).
Economic exchanges (e.g. e-commerce).
Entertainment.
etc.
Software defects are extremely costly in term of
money
reputation
loss of life
 Importance of Software Quality
Notable consequences of software errors:
1988 shooting down of Airbus 320 by the USS Vincennes - cryptic and
misleading output displayed by tracking software.
1991 patriot missile failure - inaccurate calculation of time due to computer
arithmetic errors.
London Ambulance Service Computer Aided Dispatch System several deaths.
On June 3, 1980, the North American Aerospace Defense Command (NORAD)
reported that the U.S. was under missile attack.
First operational launch attempt of the space shuttle, whose real-time
operating software consists of about 500,000 lines of code, failed -
synchronization problem among its flight-control computers.
9 hour breakdown of AT&T's long-distance telephone network - caused by an
untested code patch.
 Importance of Software Quality
Ariane 5 explosion June 4, 1996

First flight of the European Ariane 5 launcher crashed about 40

seconds after takeoff.
Cost was about half a billion dollars.
Explosion was the result of a software error:
Uncaught exception due to floating-point error: conversion from a
64-bit integer to a 16-bit signed integer applied to a larger than
expected number.
Module was re-used without proper testing from Ariane 4 .
Error was not supposed to happen with Ariane 4.
No exception handler.
 Importance of Software Quality

Mars Climate Orbiter - September 23, 1999

Mars Climate Orbiter, disappeared as it began to orbit
Mars.
Cost about $US 125-million.
Failure due to error in a transfer of information between a
team in Colorado and a team in California.
One team used imperial units (e.g., inches, feet and pounds) while
the other used metric units for a key spacecraft operation.
 Importance of Software Quality

Mars Polar Lander - December, 1999

Mars Polar Lander, disappeared during landing on Mars.
Failure most likely due to unexpected setting of a single
data bit.
Defect not caught by testing.
Independent teams tested separate aspects.
 Importance of Software Quality
Internet viruses and worms
Blaster worm (US$ 525 million)
Sobig.F (US$ 500 million 1 billion)
Exploit well known software vulnerabilities
Software developers do not devote enough effort to
applying lessons learned about the causes of
vulnerabilities.
Same types of vulnerabilities continue to be seen in
newer versions of products that were in earlier
versions.
Usability problems
 Importance of Software Quality
Monetary impact of poor software quality (Standish
group - 1995)
175,000 software projects/year - Average Cost per
project
Large companies - $US 2,322,000
Medium companies - $US 1,331,000
Small companies - $US 434,000
31.1% of projects canceled before completed.
cost $81 billion
 Importance of Software Quality
52.7% of projects exceed their budget - costing
189% of original estimates.
cost $59 billion
16.2% of software projects completed on-time
and on-budget (9% for larger companies).
Large companies - delivered systems have
approximately only 42% of originally-proposed
features and functions.
78.4% of smaller companies projects get
deployed with at least 74.2% of their original
features and functions.
Software Quality Assurance
 SQA Activities
Assuring, with acceptable levels of confidence,
conformance to functional technical requirements.
Assuring, with acceptable levels of confidence,
conformance to manageria requirements of
scheduling and budgets.
Initiating and managing activities for the
improvement and greater efficiency of software
development and SQA activities.
 SQA Responsibilities for a Project
Review of documents developed by development team.
Track the compliance with standards.
Development of QA Plan (test plan + test cases).
Implementation of test cases (Black Box or Glass Box Testing).
Management of bug repository.
Participating in code and design reviews.

Business Beam (Pvt.) Limited 44

 SQA Across the Life Cycle

Typical SDLC phases and relevant Artifacts SQA Responsibilities

Requirements Collection Requirement Specs Reviews

Analysis Functional Specs Reviews

Architecture & Design Design Specs Reviews

Development Code & Executables

Implement
Test Cases
Testing

Deployment Deployment Docs Review

Business Beam (Pvt.) Limited 45

 SQA Artifacts for a Project

Dev. Artifacts SQA Artifacts SQA Artifacts

(more)
Requirement Specs RS Reviews

Functional Specs FS Reviews

Test Plan

Design Specs DS Reviews

Test Cases

Code & Executables Bug Reports

More Test Cases

Deployment Docs DD Reviews

Business Beam (Pvt.) Limited 46

Software Quality Factor
 Question

To know that quality has improved, it would be

helpful to be able to measure quality.
How can we measure quality?
 Requirements
SoftwareRequirements
A condition or capability needed by a user to solve a
problem or achieve an objective that must be met or
possessed by a system or system component to satisfy
a contract, standard, specification, or other formally
imposed document. (IEEE)
RequirementEngineering
The development and use of costeffective technology
for the elicitation, specification and analysis of the
stakeholder requirements which are to be met by
software intensive systems (Berry and Ryan, 1999)
 Comprehensive Requirements

Comprehensive definition of requirements that will

cover all aspects of software use throughout all
stages of the software life cycle.

Factor models define the broad spectrum of software

requirements.
 Software Quality Factors

Different Models:
McCalls Factor Model (11 Factors)
Evans and Marciniak factor model (12 Factors)
Deutsch and Willis factor model (15 Factors)
ISO/IEC 9126 (10 Factors)
McCalls Factor Model
Deutsch & Willis Factor Model
Deutsch & Willis Factor Model (cont.)
ISO/IEC 9126: Internal and External Quality
 ISO/IEC9126 Sub-characteristics
Quality Sub-characteristics
Functionality Suitability
Accuracy
Interoperability
Security
Portability Installability
Coexistence
Adaptability
Replaceability
Efficiency Resource Utilization
Performance
Reliability Maturity
Fault tolerance (robustness)
Recoverability
Usability Understandability
Learnability
Operability
Attractiveness
Maintainability Analyzability
Changeability
Stability
Testability
 In-class Exercise

What quality characteristic and its relevant sub-characteristic

that best fits the following requirements:
If the customer provides a How Did You Hear About Us
identifier code during the application process, retrieve
customer information from GloboRainBQW.
In case of a disconnection of the Telephone Bankers
workstation from the system while dealing with a customer,
the system shall restore the work to the same state when
connected.
All pages shall be self contained with all control information
built into the page such as that the user does not have to
leave the screen to get help.
 Summary

Relationship of software, software quality & SQA

The need of software quality factor
Different software quality factors
 Formation of Group

Refer to your e-learning

Ice breaking session
Problem Solving 1