Biswajit Biswas

Table of contents

Abstract ____________________________________________________________ 3
Abbreviations _______________________________________________________ 4
Introduction _________________________________________________________ 5
Model Description____________________________________________________ 5
Calculations ________________________________________________________ 5
Other Usage ________________________________________________________ 7
Plotting of Gompertz curve ____________________________________________ 8
Imperatives of the model ______________________________________________ 8
Conclusions ________________________________________________________ 9
Abstract

This paper discusses a reliability model, which has been successfully used for
predicting reliability of a software system. This model is based on a numerical
distribution model based on “Rayleigh Equation” belonging to Weibull family of
curves.

Taking input from the upstream processes in terms of defect density and defect
injection rate, it is possible to estimate the defect rates to be unearthed at
downstream processes, e.g. final stages of testing. This estimate is useful for
obvious reasons, as it helps to envisage the software system much before it is
shipped. SPC methods for defect management and growth curves used for
determining test-stop stage can also be effectively used vis-a-vis this model as it
gives apriori information of the parameter that needs to be controlled.

This model has been successfully used by TEL in more than 20 live projects in
last 18 months and used by us to predict the outgoing defect rate. The
predictions at various stages have been pretty accurate. This paper also
discusses the imperatives of the model, the methods of estimating organizational
constants, which goes into defining the model for this purpose.
Abbreviations

HLD - High Level Design

KLOC - Kilo Lines of Code
LLD - Low Level Design
PDF - Probability Distribution function
SPC - Statistical Process Control
TEL - Tata Elxsi Limited
Introduction

Software reliability models are used to estimate the reliability or the number of
latent defects of the software product when it is available to the customers. Such
an estimate is important for two reasons:

i) Quantitative statement on outgoing quality of the software product.

ii) For resource planning for the software maintenance stage.

Model Description

The number of defects recovered during various life cycle stages of a project
conforms to a numerical distribution, which is represented by Rayleigh equation.
Estimation of overall defect density of the entire project can be obtained by
carrying out non-linear regression analysis using this equation with observed
defect data for design reviews and code reviews. Estimation for number of
defects for any stage (e.g., Unit Testing) thereafter can be obtained through
PDF.
The nature of curve indicates the pattern of defect removal rate in the life cycle of
the project. The steeper it is, the less defect prone it is when delivered to
customer, on the other hand, if it is flatter, it indicates inefficient defect removal
rate and hence lot of defects leaked to customer.

Rayleigh Model, among the family of Weibull distribution, is found to be most

suitable for predicting reliability of software product. It predicts the expected
value of defect density at different stages of life cycle of the project, once
parameters (total number of defects or total cumulative defect rate and peak of
the curve in terms of unit of time) for the curve are decided.

The PDF of the curve can be given as:

F(t)= f(K, tm, t) where tm is the peak of the curve

t is actual time unit
K is cumulative defect density

Process control can be exercised using these predicted values of defect density
at various stages of testing.

Calculations

Two parameters K, tm are estimated prior to plotting the Rayleigh curve for entire
range. Minimum three points are needed to estimate these parameters using
Non-linear regression analysis. Parameters are considered to be valid if
following statistical derivations are in conformance.
a) Standard error of estimate
b) Proportion of variance – R2
c) Durbin-Watson test for autocorrelation

Once two parameters are established, graph is plotted for entire range.

t t = TUT

Fig: Rayleigh Distribution

Defect density at Life cycle stage t= TUT (at unit testing) is = K * y (at t= TUT)

Thus, defect density is predicted at any stage of the project by substituting the
value of “t” and getting the “Y” axis value, multiplied by K.

Following time scale is used in the model for carrying out Non-linear regression
analysis.

Stage Time scale

HLD 0.5
LLD 1.5
Implementation 2.5

Unit Testing 3.5

Integration Testing 4.5
System Testing 5.5

Table: Mapping of stages to time scale

Below is the plot of Rayleigh curve plotted for a live project.

Rayleigh Curve

35
30 LLD 28.9
Defects /KLOC

25 CODING
23
20
15 UT
10 10.5
5 IT
HLD 3.5
0 1.4
05
45
85
25
65
05
45
85
25
65
05
45
85
25
65
05
45
85
0.
0.
0.
1.
1.
2.
2.
2.
3.
3.
4.
4.
4.
5.
5.
6.
6.
6.
Life cycle stages

Fig: Rayleigh Plot for a TEL project

The red line indicates the actual defect density observed as against the predicted
values (brown smooth curve) obtained through theoretical model.
Observed defect density closely matches with the defect density predicted by the
model. Outgoing quality of the software

The curve indicates the defect density at the time of system testing as 0.68
defects/KLOC or 21 defects.

Other Usage

The process of bug-detection, during a planned testing process against test

cases, follows a pattern called S-curve or more suitably Gompertz curve. Plot of
cumulative bug/trouble count against those theoretically predicted patterns with
control boundaries shows the consistency and effectiveness of the testing
process.
Plotting of Gompertz curve

Estimation for expected number of defects (k) is done by using Ralyeigh

equation.
Upper and Lower control limits of Gompertz curve are calculated using equation:

t
b
Y= k* ( a )

where ‘a’ and ‘b’ are model parameters, values of which depend on type of
software.

f t

Fig: Gompertz curve

Imperatives of the model

The basic assumptions made in using Rayleigh models are:

i) Error injection rate at various stages is constant.
ii) Defect removal effectiveness remains more or less unchanged.

This means, the upstream processes of the life cycle like Design reviews and
Code Reviews need to be followed consistently.
In-flight control charts used for measuring consistency of review processes are
necessary for this purpose.

So, the implied need of this model is to have a very high level of process maturity
and process capability. Implementation of the model is not going to be a success
unless these implied needs are satisfied.
Conclusions

This has been very effectively implemented in our organization since last 18
months in over 20 projects. The outcome has been very successful prediction of
outgoing quality.
This is also reflected in our customer feedback.

Effective implementation of this model calls for overall consistency in all software
engineering activity, which reaps its benefit all through the life cycle stage in
terms of effective reviews and effective testing. To recollect the success, the
projects where this has been used for predicting out going quality and has
matched to the prediction, we have achieved peer review effectiveness in excess
of 75% and testing effectiveness in excess of 95%.

This model is one of the main tool for Quantitative process management and
Statistical process management as its very use itself calls for on-line process
under statistical control.

References:

[1] Stephen H. Kan - Metrics & Models in Software Quality Engineering.