CSE3308/DMS/2005/24

Monash University - School of Computer Science and Software Engineering

Software Engineering: Analysis and Design - CSE3308

Software Quality

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.1

Lecture Outline
u What u How

is Software Quality? Quality Activities

can we measure Software Quality?

u Software

v Software Quality Assurance v Software Quality Planning v Software Quality Control u Quality

Standards Quality Standards

v AS3563 and ISO9000 u Beyond

v Capability Maturity Model v Personal Software Process

CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.2

Page 1

What is Software Quality?

The problem of quality management is not what people dont know about it. The problem is what they think they do know. Quality has much in common with sex. Everybody is for it, (under certain conditions, of course). Everyone feels that they understand it, (even though they wouldnt want to explain it.) Everyone thinks execution is only a matter of following natural inclinations (after all, we do get along somehow.) And, of course, most people feel that problems in these areas are caused by other people (if only they would take the time to do it right.) Crosby 1979
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.3

Definitions of Quality
u Quality is fitness for use - Juran v those product features which meet the needs of the customers and thereby provide product satisfaction v freedom from deficiencies u Quality

is conformance to requirements and zero defects - Crosby u Quality is the totality of characteristics that bear upon its ability to satisfy stated or implied needs - ISO9000
v stated needs - specified as requirements by a customer v implied needs - identified and defined by the company providing the product

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.4

Page 2

Defining Software Quality

u Software

Quality is conformance to:

v explicitly stated functional and performance requirements v explicitly documented development standards v implicit characteristics that are expected of all professionally developed software u Software

Quality is ambiguous, subjective and multidimensional

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.5

The need for more detailed measures of quality

u Previous

definitions are high-level and difficult to measure are difficult to specify are usually incomplete

u Requirements u Requirements u Experiment:

Compare software warranties with hardware warranties

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.6

Page 3

The Hardware Warranty

Company X warrants that the products it manufactures and sells are free from defects in materials and workmanship. If any product fails to operate properly, the company will repair the defective product and restore it to normal operation without charge

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.7

The Software Warranty

Company Xs sole obligation under this warranty will be to provide support services described in our current software support policy. The company does not warrant that the licensed software is free from defects or that the support services will correct any defects that might exist.

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.8

Page 4

McCalls Software Quality Factors

Maintainability Flexibility Testability Portability Reusability Interoperability

Product revision

Product transition

Product operations Correctness Reliability Efficiency Integrity Usability

CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.9

The ilities -Software Quality Attributes - Product Operation

u

Correctness
v Does it do what I want? v Tool - Use Cases

Reliability
v Does it do it accurately all of the time? v Tool - Formal methods

Efficiency

v Will it run on my hardware as well as it can? v Tool Good algorithmic design, appropriate language (even Assembler in some cases) v Is it secure? v Tool - Java

Integrity Useability
v Is it designed for the user? v Tool - User-centred design

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.10

Page 5

The ilities -Software Quality Attributes - Product Revision

u Maintainability v Can I fix it? v Tool - Encapsulation u Flexibility v Can I change it? v Tool - Encapsulation u Testability v Can I test it? v Tool - Interfaces
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.11

The ilities -Software Quality Attributes - Product Transition

u Portability v Will I be able to use it on another machine? v Tool - Java, ANSI C, (and other emerging technologies) u Reusability v Will I be able to reuse some of the software? v Tool - OO class libraries (e.g. GUI components), function libraries (e.g. Numerical Recipes in C) u Interoperability v Will I be able to interface it with another system? v Tool - CORBA, DCOM

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.12

Page 6

Metrics for Quality

u Many

quality factors are difficult or impossible to measure directly

v Need to develop indicative measures of the quality factors v Unfortunately many of these measures are still subjective

u Two methods are: v McCalls checklist approach (see download on website) v Hewlett-Packards FURPS
v Functionality
feature set, capabilities of program, generality of delivered functions, security of overall system

v Useability
human factors, overall aesthetics, consistency and documentation

v Reliability
frequency and severity of failure, accuracy of output, MTTF, ability to recover from failure, predictability

v Performance
processing speed, response time, resource consumption, throughput, efficiency

v Supportability
extensibility, adaptability, serviceability testability, compat ibility, configurability , ease of installation, ease of problem localization

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.13

Software Quality Activities

u Quality Assurance v The production of organisational procedures and standards which lead to high-quality software u Quality Planning v Choosing appropriate procedures and standards and tailoring them for a specific software project u Quality Control v Ensuring that procedures and standards are followed by the software development team

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.14

Page 7

Software Quality Assurance

u How

an organisation defines the methods by which it will achieve quality u Organisation must define or select standards u Quality of the product is influenced by the quality of the process u Standards need to be embedded in the software development process u All this will be documented in a Quality Manual u The relationship between software process and product quality is complex and poorly understood
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.15

Assuring Quality
u Need

to ask four questions u WHAT attributes of the product manifest quality in your context? u HOW is quality to be measured? u WHEN do we evaluate the product and the process? u WHO is responsible for carrying out the process? u Quality relies on people, but we can greatly help or hinder people from producing quality
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.16

Page 8

Quality Principles
u Try

to prevent defects from being introduced u Ensure that defects that get in are detected and corrected as early as possible
u Establish u Measure

and eliminate the causes as well as the symptoms of defects quality characteristics audit work for compliance with standards and procedures

u Independently

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.17

The Quality Plan

u Is

specific to a project u Produced early in the life of a project u Should set out desired product qualities u Should define how the quality is to be assessed u Should indicate which standards are to be applied u Indicate how compliance to the standards is monitored and assured u May define new standards
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.18

Page 9

Contents of Software Quality Plan from ISO9000

u u u u u u u u u u u

Management responsibility Quality system Contract review Design control Quality control Purchasing Customer supplied info Configuration management Process control Inspection and testing Inspection and testing equipment

u u u u u u u u u

Control of non-conforming product Corrective action Handling, storage, packing and delivery Quality records Internal quality audits Training Software maintenance Statistical techniques Control of the development environment
Lecture 10A.19

CSE3308 - Software Engineering: Analysis and Design, 2005

Quality Control
u Ensuring

that all of the above gets done! u Tyranny is not effective in ensuring that the work is done u People must see a clear benefit to them in performing the above activities u Embedding the procedures in day-to-day work is essential u Reviews are one of the major tools for quality control

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.20

Page 10

Quality Standards
u u

Major standards used in Australia ISO9000

v international set of quality standards v ISO9000-3:1997 is a specific subset aimed at software development. It may be superceded by the latest ISO9000 standard

HB 90.9-2000
v Provides guidelines for the software industry on quality management systems (QMS) complying with ISO9001:2000 (Quality Management Systems requirements) v Specifies requirements for a QMS for a software development organization that
needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements, and aims to enhance customer satisfaction through the effective application of the system, including processes for continual improvement of the system and the assurance of conformity to customer and applicable regulatory requirements.

v Governments and other large organizations often require quality certification, such as ISO9000 v earlier standards superceded by this include AS 3563 and AS 3905.8
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.21

Quality Standards (2)

u Organisations

must be certified to be granted ISO9000 status u Certification is granted by independent auditing groups (e.g. Standards Australia) u Certification is not cheap (approximately $500,000 for a medium-sized company) u Certification might not bring any benefits to the company in terms of quality, but could in terms of marketing u Certification is an on-going process; audits are carried out annually

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.22

Page 11

Quality Standards (3)

u To get benefits from quality standards v must use sound management techniques v must aim to improve the process v employees must participate actively u Quality

standards are of greatest value to those organisations which dont already have formal development processes u They dont replace individual skills and abilities u They can only be as good as the work practices which they document
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.23

Beyond Quality Standards

u Quality

standards are only a necessary first step towards a software development environment which produces quality software u We need to define what sub-processes are necessary in the overall process u The Capability Maturity Model (CMM) documents this for organisations u The Personal Software Process (PSP) does this for individual developers

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.24

Page 12

The Capability Maturity Model

u

Developed by the Software Engineering Institute (SEI), based on work by Watts Humphrey
v Now evolved into the CMMI family of models which are still very similar (e.g. http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr029. pdf)

5 Levels
v Level 1: Initial v Level 2: Repeatable v Level 3: Defined v Level 4: Managed v Level 5: Optimising

At each level Key Process Areas (KPAs) provide goals and example practices

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.25

Process Maturity Levels

Optimising Managed Process Control

Process measurement Defined

Process Definition Repeatable

Initial

Basic Management Control

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.26

Page 13

Level 1: Initial
u Process u Few

is ad hoc and sometimes chaotic depends upon individual effort

processes are defined Process Areas

u Success u Key

v None u Too

many software development organisations are in this category

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.27

Level 2: Repeatable
u Basic

project management processes defined u Process discipline in place to repeat successes u Change is inherently dangerous at this level u Key Process Areas
v Configuration management v Quality assurance v Sub-contract management v Project tracking v Project planning v Requirements management
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.28

Page 14

Level 3: Defined
u Process

is documented, standardised and integrated into a standard software process u All projects use this standard software process u Measurement is still qualitative u Key Process Areas
v Peer reviews v Intergroup coordination v Software product engineering v Integrated software management v Training program v Software process definition and focus
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.29

Level 4: Managed
u Detailed

measures of software process and product quality are collected u Process and product are quantitatively controlled u Data gathering should be automated u Key Process Areas
v Quality management v Quantitative process management

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.30

Page 15

Level 5: Optimising
u Continuous

process improvement is enabled u There is quantitative feedback from the process and from piloting innovative ideas and technologies u Move from a purely product focus to focusing on process as well u Key Process Areas
v Process change management v Technology change management v Defect prevention

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.31

The Personal Software Process

u Mirrors

the CMM for an individual u designed to help you become a better software engineer u requires research, motivation and study to work u Framework for
v why you make errors and how you find them v determining the quality of your reviews v determining the types of errors you make u Developed

by Watts Humphrey
Lecture 10A.32

CSE3308 - Software Engineering: Analysis and Design, 2005

Page 16

PSP0
u PSP0

- Baseline Process

v your current process with some basic measurements and a reporting format v time recording v defect recording v defect type standard u PSP0.1 v a coding standard v size measurement v process improvement proposal
CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.33

PSP1
u PSP1.0

- Personal Planning Process

v adds planning steps to PSP0: test report size and resource estimation u PSP1.1 v to establish a performance rate for future planning v PSP1.0 enhanced by adding: task planning schedule planning

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.34

Page 17

PSP2
u PSP2.0

Personal Quality Management Process

v adds review techniques to PSP1 to help you find defects early: design reviews code reviews defect rates are typically 1 per 5-12 lines of code, do you know what yours is?

u PSP2.1 v establishes design completeness criteria: design templates

CSE3308 - Software Engineering: Analysis and Design, 2005 Lecture 10A.35

PSP3
u PSP3

- Cyclic Personal Process u For large programs - 10,000 LOC u Sub-divide into PSP2-sized modules u Enhance the base module in iterative cycles u In each iteration do a complete PSP2 including design, code, compile, test u Effectively scale up from base module to large program, if each increment is of high quality

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.36

Page 18

Overview of CMM and PSP

u CMM

sets out the principal practices for managing the processes in large-scale software development sets out the principal practices for defining, measuring and analysing an individuals own processes

u PSP

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.37

References
u Pressman,

Roger S., Software Engineering: A Practitioners Approach, McGraw-Hill, 2000 (Ch. 19). u Capability Maturity Model for Software (SWCMM), The Software Engineering Institute (SEI), Carnegie Mellon University. http://www.sei.cmu.edu/cmm/cmm.html u The Personal Software ProcessSM (PSPSM ), The Software Engineering Institute (SEI), Carnegie Mellon University. http://www.sei.cmu.edu/tsp/psp.html

CSE3308 - Software Engineering: Analysis and Design, 2005

Lecture 10A.38

Page 19