Software Quality

and Testing
Computer Engineering Department
Indus University

UNIT I
Background and Motivation

Software Quality
What is? : Software Quality is degree to which a

system, system component, or process meets

specified requirements or meets customer expectation
Why is required ? : Improvement in software quality
improves customer satisfaction
How to measure ? : By softwares usability,
availability, reliability, maintainability etc.
IBM measure quality with CUPRIMDSO
Hewlett-Packard focus on FURPS

How to improve ? : Adhering to requirements,

delivering quickly without bugs

Software Quality Factors

ISO 9126 Quality Factors:
Functionality
Reliability
Usability
Efficiency
Maintainability
Portability
HP Quality Factors (FURPS):
Functionality
Usability
Reliability
Performance
Supportability

MaCalls Quality Factors

Ethical Basis for Software Quality

Quality is responsibility of everyone
Business decision for cost versus quality
Communication to end user about quality of

software
Testing for applications thoroughly:
Functionality: Positive versus Negative tests
Security
Performance
Environmental

Total Quality Management Principles

TOTAL QUALITY MANAGEMENT
CONTINUOUS IMPROVEMENTS

Customer
Focus

Process
Improveme
nt

Hunan
Side of
Quality

Metrics, Models, Measurements and Analyses

Software Process Models &

Methodologies
Waterfall Development Model
Prototyping Approach
Spiral Model
Iterative Development Process Model
Object-Oriented Development Process
Cleanroom Methodology
Agile Methodology

Software Process Models Waterfall

Development Model
Requirements Gathering and Analysis
Architectural Design
HLD /
I0

LLD / I1

Code /
I2

UT

Integration
Component Test
System Test Functionality, Regression,

Performance
Early Customer Feedback and Beta Test Programs
Release

Software Process Models Waterfall

Development Model
Model is used when requirements are known by

customer
Model is used for significantly larger and complex
system e.g. implementing HR system company wide
Completion of one cycle leads to final product
Produce good quality product due to known
requirements and validation is done at each step in
development cycle
Takes significant amount of time to market
Higher cost
Requirement modifications will become costly

Software Process Models

Prototype Approach
Requirements
Gathering and Analysis
Quick Design
Refining Design &
Prototype

Building Prototype

Customer Evaluation of Prototype

Customer Satisfied
Full-Scale Development

Software Process Models

Prototype Approach
Model is extremely useful when requirements

are not known by customer

Model is very useful to know GUI designs to
customer
Shorter time to market
Requirement modification can be implemented
If prototype is not used then effort is wasted
May lead to higher cost if several prototypes
are build before the final one
Quality may be compromised

Software Process Models

Spiral Model

Software Process Models

Spiral Model
Modified and Improved version of Waterfall

Model
Relies heavily on prototyping and risk
management
More flexible then waterfall model
Verification is done at each phase
Used in many modern systems which involves
reuse, object-oriented development, rapid
prototyping etc
If prototype is not used then effort is wasted

Software Process Models Iterative

Development Model

Software Process Models

Iterative Development Model
It is one of the Agile methodology which are extremely

important in current development environment

Each Iteration provide additional functionalities
Customer are able to see finished product components at
very early stage of development
Depending on feedback from customer functionality can
be modified in subsequent iteration
Cost management is better
Lower risk
Wrong architectural decision in early iteration can cause
major roadblocks in subsequent iterations
There may be duplication of effort in terms of testing

Software Process Models Object

Oriented Development Process
Model the essential system (MVC

architecture)
Derive candidate-essential classes
Constrain essential model
Derive additional classes
Synthesis classes
Define interfaces
Complete design
Implement the solution

Software Process Methodology

Cleanroom Methodology

Software Process Methodology

Agile Methodology
Alternative to waterfall, or traditional

sequential development.
Uses incremental, iterative work cadences,
known as sprints which are 1 to 4 weeks long
Agile team should include a customer
representative for efficient face-to-face
communication
Very short feedback loop and adaptation cycle
Quality focus
SCRUM is most common agile methodology

Quality Standards, Methodologies

The SEIs Capability Maturity Model (CMM &

CMMI)
The SPR Assessment (Software Productivity
Research)
The Malcolm Bladrige Assessment
ISO 9000 (International Standard
Organization)

Quality Standards, Methodologies

The SEIs CMM
Quality standards specific for software development
CMM Standard is used to evaluate process maturity
Five Maturity Levels: Initial, Repeatable, Defined, Managed

and Optimizing
Initial (Level 1): Chaotic: Unpredictable cost, schedule and
quality performance
Repeatable (Level 2): Intuitive: Cost and quality are not
reliable, some control over schedule. Key Process Areas:
Requirements Management
Software project planning and oversight
Software subcontract Management
Software quality assurance
Software Configuration Management

Quality Standards, Methodologies

The SEIs CMM Cont..
Defined (Level 3): Qualitative: Reliable cost and schedule but quality is

unpredictable Key Process Areas:

Organizational process improvements
Organizational process definition
Training Program
Integrated Software Management
Intergroup Coordination
Peer Reviews

Managed (Level 4): Quantitative: reasonable control over quality, cost and

schedule. KPA area:

Process measurement and analysis
Quality management

Optimizing (Level 5): Continuous process improvements and automation. KPA

are:
Defect prevention
Technology innovation
Process change management

Quality Standards, Methodologies

The SPR Assessment
Quality standards specific for software

development
SPR Standard is used to uncover issues with
productivity, quality and user satisfaction.
It is sometimes used in conjunction with SEIs CMM
Five point scale: 1-Excellent, 2-Good, 3-Average, 4Below Average, 5-Poor
Has automated tool (CHECKPOINT) to for collecting
a resource planning and quality projection data
Standard collect qualitative productivity and
quantity data from each project

Quality Standards, Methodologies

The SPR Assessment Cont
Software quality and metrics covered by assessment
Quality and productivity measurements
Pretest defect removal experience among programmers
Testing defect removal experience among programmers
Project quality and reliability targets
Pretest defect removal at the project level
Project testing defect removal
Post-release defect removal
Finding are divided in 5 streams:
Findings about projects and software products assessed
Findings about the software technologies used
Findings about the software processes used
Findings about the ergonomics and work environment for staff
Findings about personnel and training for management and staff

Quality Standards, Methodologies

The Malcolm Baldrige Assessment
Quality standards used for all industry (not just

software), especially in USA considered as most

prestigious award. AKA MBNQA.
7 examination criteria: Leadership, Information
and Analysis, Strategic quality planning, Human
resource utilization, Quality assurance of
products and services, Quality results,
Customer satisfaction
System scores based on 3 evaluation
dimensions : approach, deployment and results

Quality Standards, Methodologies

ISO 9000
Registration standard to show industry that

organization is following certain quality standards

Requires audit passing in 20 major elements for
software companies to get ISO certification.
For software development it collect product metrics as
well as process metrics
Corrective actions and document control are the area
of nonconformance most of the time
Very strong requirements for documentation:
ownership, versions, approval, distribution
Described as say what you do, do what you say, &
prove it

Quality Improvements with

Methodologies
CMM, CMMI & ISO Focuses on process

improvement and assessment

SPR & Six sigma Focuses on producing zero
defect products
Malcolm Baldrige & TQM Management
strategies to embed quality awareness in all
organizational process
These methods drive quality improvements by
process improvements, product improvements
and people based improvements

Information Engineering
Defined as generation, distribution, analysis and use of

information in systems
Enables effective communication of information
throughout enterprise
Helps improve people skills, procedure and systems and
hence the overall quality
Two types of IE variant:
Data Processing driven: Enable IS Departments to develop

information systems that satisfied the information needs.

CASE tools available today support this DP-driven
Business driven : Enable strategic business planning. It
was designed for rapid change in the client/server, objectoriented environment of the business-driven 1990s

Information Engineering Cont

Data Processing driven stages :
Information Strategy Planning
Outline Business Area Analysis
Detailed Business Area Analysis
Business System Design
Technical Design
Construction
Transition
Business driven stages:
Strategy

Analysis
Strategic Modeling
Tactical and Operational Modeling
Activity Modeling
Process Modeling
Code Generation

Software Tools: Information Engineering Facility (IEF) from Texas Instruments

Measuring Customer Satisfaction

What to measure ?
Measure overall satisfaction
Measurement for loyalty
Attribution satisfaction product/feature feedback
Measure exit/abandonment rate
Net promoter score likelihood of referral by customer
Things going wrong
How to measure ?
Request that your customers fill in a survey
Ask the right questions in survey
Familiarize yourself with customer expectation and meet them on

regular basis
Discover customer satisfaction by recognizing where you are going
wrong
Find out who has more satisfied customers then you and why

Software Quality Engineering

Define the skills and knowledge necessary to

perform software quality engineering tasks

Understand the software life cycle
Determine how to evaluate software quality
activities and processes and determine
whether they meet their intended purpose
Choose applicable standards and meeting
techniques

Defining Quality Requirements

Product versus process requirements
For defining quality requirements all different

types of requirements must be defined :

Architectural requirements
Business requirements
User (stakeholder) requirements
Functional (solution) requirements
Quality-of-service (non-functional) requirements
Implementation (transition) requirements

Defining Quality Requirements

Characteristics of good/quality requirements
Unitary (Cohesive) : Address one and only one thing
Complete : Fully stated with no missing information
Consistent : Does not contradict with other requirement and fully
consistent with other documentation
Non-conjugated (Atomic): Does not contain conjunctions e.g.
Password must contain 8 characters long with at least 1 special
character from & ( ) *, instead it should be 1) Password must contain
8 characters. 2) Password must have at least 1 special character from
&()*
Traceable: Must trace back to business need
Current : Should not be obsolete with passing of time
Unambiguous : It is subject to one and only one interpretation
Specify Importance/Priority : Criticality of the requirement
Verifiable : Requirement is testable

Management Issues for Software

Quality
Each stage of SDLC is prone quality issues e.g.
Requirements are ambiguous, not complete etc
Version Control of software is not maintained
System architecture cannot handle load of user
base
Management of Quality issue is to capture the

quality issues, process them by initiating quality

improvement activities and monitoring until
resolution.
SAP QIM is one of the tool that provides the
solution

Data Quality Assurance & Control

Data quality assurance(QA) improves data quality by :
Profiling of data to discover inconsistencies and anomalies
Performing datacleansing activities
Data quality control(QC) is the process of controlling the usage of

data with known quality measurements for an application or a process

Data QA processes provides following information to Data QC :
Severity

of inconsistency
Incompleteness
Accuracy
Precision
Missing / Unknown

E.g. if Data QC finds too many inconsistencies in data from QA, it will

stop data being sent to process or application needed the same so

that incorrect processing / function does not happen

Benchmarking and Certification

Benchmarking is process of developing requirements and

setting goals to measure your performance against that

of best-in class
Requires research into best practices
Goes beyond industry standard
Based on learning from others

Steps for the benchmarking process:

1.Planning
2.Analysis
3.Integration
4.Action
5.Maturity

Benchmarking and Certification

Cont
Certification is process of confirming that :
measurements from your organizations product
or process is performing in accordance with
benchmark
product is functioning and performing as per
requirements
processes are being followed that was said to
be followed