STUDY OF FIVE MAJOR PROBLEMS,

REASONS AND SOLUTIONS OF A SOFTWARE

QUALITY ASSURANCE DEPT: A
QUALITATIVE ANALYSIS
Habibullah Rafay 12-2176
Sheharyar Mohsin 12-2030
Waleed Arshad 12-2021
Dept. of Computer Science
FAST-NU
Karachi
May 8, 2015

Contents
1 Introduction
1.1 Review of Related Literature .
1.2 Statement of the problem . . .
1.3 Objective of the study . . . . .
1.4 Significance of study . . . . . .
1.5 Limitation and Delimitations of
1.6 Research Methodology . . . . .
1.7 Sampling Technique . . . . . .
1.8 Research Instrument . . . . . .
1.9 Ethical Considerations . . . . .
1.10 Research Procedure . . . . . .

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

3
3
5
5
6
6
6
6
6
6
7

2 Findings
2.1 Basic Problems of Software Quality Assurance
2.2 Constantly Changing Requirements . . . . . . .
2.3 Preference of Software . . . . . . . . . . . . . .
2.4 Testing Scheme . . . . . . . . . . . . . . . . . .
2.5 Importance of Software Quality Assurance . . .

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

7
7
7
7
7
8

. . . .
. . . .
. . . .
. . . .
study
. . . .
. . . .
. . . .
. . . .
. . . .

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

3 Discussion

4 Conclusion

9
1

5 Recommendation

10

Letter of Transmittal
April 21st, 2015
Nazia Imam
Lecturer, FAST-NU,
Shah Latif town, National Highway,
Karachi
Respected Mam,
As assigned to us on March 4th, 2015, we are submitting the attached research entitled STUDY OF FIVE MAJOR PROBLEMS, REASONS AND
SOLUTIONS OF A SOFTWARE QUALITY ASSURANCE DEPT: A QUALITATIVE ANALYSIS.
This research discusses the five common major problems faced by a software quality assurance department in an organization or a software house.
Also discussed are the reasons and solutions of the problems which are inferred through the analysis of the qualitative information received through
personnel.
We hope you find this report satisfactory.
Sincerely yours,
Habibullah Rafay 12-2176
Sheharyar Mohsin 12-2030
Waleed Arshad 12-2021

Abstract
Software quality assurance departments have been a major cause in the
booming success of IT industry. Today, this department has a vital role in
almost every software house. Problems that are being faced by the SQA
Dept. is a major study conducted by IT professionals nowadays. This
article highlights the major problems faced by SQA dept. In this article,
we have acquired the data by interviewing officials from multiple firms. The
research deduced that sometimes the requirements by the client are unclear
and vague. Also, the client changes its requirements frequently when the
build is in its final phases. Furthermore, for any firm, the SQA team restricts
itself to the domain of its firm only, causing lack of diversity. Hence the
results conclude that major problems for an SQA team comprises of the
fact that requirements for any software, by the client, is the key. In order
to reduce the over work that a SQA team does, it is necessary that the
requirements from the client are concrete and stable, and it must remain
constant until the build is released.

Introduction

There are numerous problems that software houses face. Some of which
include the development sector, while others include a place and reputation
in the market. These problems affect the overall performance and the aims
of a software house or an organization. There are some prior problems that
need to be looked upon at first. One of those types are the problems faced
by the SQA dept. of a software house or an organization. The SQA dept. is
the most critical and important dept. of any organization. It deals with the
development sector as well as the client sector of the organization. In short,
the SQA dept. acts as a bridge between the Client and the Organization.
The problems faced by the SQA dept. are more considerable than all of the
other problems of an organization. Consequently a major chunk of problems
faced by any organization is the problems faced by SQA dept. From the
reputation in the market to the overall productivity and performance of the
organization SQA plays a vital role. Therefore, considering the problems of
SQA becomes the foremost interest of this research. The paper discusses five
of the major problems that an SQA dept. faces throughout the development
phase until the project is completed or the product is sold. The paper also
discusses the reasons and the inferred solutions of the problems.

1.1

Review of Related Literature

The quality assurance Department in a software house has recently been a

focused criterion in the field of research. The quality assurance department

is responsible for testing the specific software to extreme quality standards

set by the organization or the software house. However there are certain
problems that a SQA department faces during their testing of the software.
The problems that occur in the SQA department can be divided into many
categories. They could be in terms of resources, time, cost, and quality of the
user end management. There has been already a handsome amount of work
done in highlighting the issues and problems faced by the SQA departments.
Many software houses and other organizations face a lot of issues in quality
assurance of their projects. Shortcuts in testing, Less testing time, correct
errors later attitude, Lack of user involvement, Poor documentation, Poor
planning, Lack of management, Inadequate knowledge of software product
and poor testability results in poor quality assurance [1]. All of them are
key problems and must be considered in any software development to ensure
its long life. Quality Assurance team must have to focus on agreement made
with customer to check requirements with respect to customers objective and
make sure that everything is working properly as per user requirements.
Quality assurance fails due to lack of quality. Lack of Quality is a major
reason for a failure of any project [2] and it leads developer and project managers to close the project. Examples of many big projects had been specified
those examples prove that many large projects ended and were dumped because of lack of quality assurance. Benchmark failure is also discussed in this
research. Benchmark is basically software designed and developed to ensure
the quality of a project. Benchmark also fails due to non-termination of
failure projects and lack of vision. Many project managers did not mention
there failures due to the fair to lose their job. This is the reason because of
which benchmark fails. He concluded that software houses and organization
must have to create separate department for quality assurance and to get
rid of the wastage of time and cost.
When testing software certain aspects should be dealt with care.
There must be no shortcuts in testing procedures [3].The team should be
experienced and hard working. No shortcuts should be used in the testing
schedule. The team must follow the schedule very rigorously. The time that
is assigned to a testing schedule, it should be followed very strictly. There
should be no reduction in testing time. There must be a healthy coordination between the development team and the Testing team. The testing
team must not be responsible for identifying the problems and then telling
the development team to deal with such problems. The development team
should cater these problems before hand. The documentation of the user
and the system should be very distinct. Testing should have a sound knowledge of the domain of the application. The staff of the QA team should be
competent and qualified.
Functional and non-functional requirements have to be taken care of too
[4]. To build better software the testing should ensure that the software
development does not conflict with the inception phase. Choice of tools and
4

models to support the testing phase throughout is also an essentiality. Lack

of measures to test the iterative development also leads to the failure of large
projects.
Software Quality assurance is still a challenge in many areas and still there
are problems that need to be considered and sorted out. In some organizations, SQA suffers due to the denying of the process flow of the testing
procedures which mostly does not affect the product in terms of short time
period but shows its effect after a long time.
Human mistakes are also the reason for the problems faced by the SQA
departments. Management of resources and not utilizing them properly is
something that is due to the lack of human training. Some organizations do
not take SQA as an essential part of their work flow therefore less funds are
spent on SQA which is why they face difficulties with their products in the
market.
The following research intends to highlight five major problems, irrespective of the categories of the problems, faced by an SQA department of an
organization or a software house and propose suitable solutions to these
problems.

1.2

Statement of the problem

Software Quality Assurance department provides a bridge between the developers and the users. For a team of developers when developing certain
product, the consumer requirements are the key aspect. The user requirements are given to the developer which in turn follows a step by step procedure to develop a product. This development has multiple stages. Once
a product is in its build phase, the QA Dept. may play its role, starting to
test various aspects of the software.
Furthermore, in which phase the product is available to test, is very crucial
decision. To start the unit tests or to test the complete project, is the decision taken by the development team. When a product is given to QA Dept.,
the QA Dept. tests the product for every possible outcome in the market.
Thus making the product ready to encounter any real time failures.
To test a product to its limits, the QA Dept. faces various problems. The
purpose of this study is to identify the major problems faced by the Software
Quality Assurance Department while testing products.

1.3

Objective of the study

The objective of this study is to find the problems faced by the Quality
Assurance Department.
Cooperation of developers and Quality assurance officers

 To determine which problems are mostly occurred while testing software.

To help developers avoid the errors which are overlooked while developing the software.
Fundamental approaches that are used by the SQA depts.
How the rectification in products costs.

1.4

Significance of study

The significance of this study can be expressed in various aspects. First,

it will help the developers to identify the glitches and errors which were
left in the development phase. The study will provide an insight to the
upcoming SQA developers about the problems and methods to overcome
those problems while testing various products.

1.5

Limitation and Delimitations of study

This report was restricted as a low population sample. This was due to the
fact that the research was conducted in two software houses only. Therefore,
this report can be generalized at a very low population.

1.6

Research Methodology

This was a qualitative research. Semi-structured open ended interviews were

conducted.

1.7

Sampling Technique

The population of this research were the Managers of SQA departments of

two software houses in Karachi. This was a non-random convenient sample.

1.8

Research Instrument

A set of 6 independent instruments were created to ensure proper reasoning

of problems faced by SQA departments. From each software house two
personnel were interviewed.

1.9

Ethical Considerations

Proper permissions were taken from respective software houses before visiting their SQA departments. An email was sent first to ask for a formal visit.
The visit and interview was conducted after they granted visit permission.
The interviews were audio recorded with their permission.

1.10

Research Procedure

The question of the research came up with the idea of Problems Faced by
SQA. The questions then extended to the boundary of five major problems
faced by the SQA department. To make this report more effective, it was
decided to infer some solutions from the qualitative analysis that would be
done through the data that would be collected by the interviewees. Qualitative analysis was done on the open ended data that was collected through
the SQA personnel. The interview was conducted with two personnel from
each software house and the interview was recorded.

2
2.1

Findings
Basic Problems of Software Quality Assurance

The very basic problems faced by the SQA dept in testing the software
are the unclear requirements. When the client specifies his/her needs, it is
unclear for the very first time. Conducting multiple meetings decides the
one point where the requirements are frozen and both parties i.e. the client
and the developers agrees upon certain criteria. But this has to be done
very formally and systematically. Using the proper standardized methods
for gathering Functional andNon-Functional requirements of client.

2.2

Constantly Changing Requirements

Once the requirements are defined and the developing team starts on a
project, it is very hard to alter requirements. The client should understand
the fact that changing certain requirement after the development phase is
costly, both for the developer and the client.
Although developers welcome change, but in turn it affects various dimensions of the product, hence the developer now has to cater unnecessary work
flow.

2.3

Preference of Software

Various software houses and companies have different expertise. The organization prefers to test only those software in which they have strong knowledge. The organization may test their own as well as external products, but
of the same domain.

2.4

Testing Scheme

There are multiple testing techniques in software world. The technique

which we have found out is the integration based testing. This technique
involves testing the software in its development phase.

2.5

Importance of Software Quality Assurance

The SQA dept not only helps the client but also helps the developers. Fixing
the product before being launched into the market, avoiding the slightest of
chance of any major loss is indeed a benefit to the digital world.

Discussion

The problems faced by the SQA department are very subtle as there are certain reasons why SQA face such problems during their testing and quality
assurance. The unclear requirements make is difficult for the SQA team to
test the software upto the mark. The constantly changing requirements add
to the difficulty of testing phase as it becomes nearly impossible to entertain
changes after the build has been released. The preference of the software
and the choice of the testing scheme are major factors that affect the testing
of every other software.
The lack of formal meetings between the client and the software house team
is the main reason behind the unclear requirements for the very first time
which builds a major hurdle for the SQA team to test the software accordingly.
When the project is being finalized on paper it is very probable that the
percentage of the change is not decided in the early meetings, which results
in the client asking to change modules and major parts in the software which
adds another pile of hurdles over the previous one.
The preference of the software that an SQA team tests depends upon the
domain of the types of software that the software house builds. Due to lack
of expertise in multiple types of software domains it restricts the SQA team
to test the software remaining in a particular domain, which leaves the SQA
team with short boundaries.
In order to overcome some of the problems the SQA teams use Integration
based testing so that the testing is done along with the development phase
so as to reduce the burden of work and entertain any minor changes that a
client eventually asks for.
To overcome unclear requirements problem, which is the first and foremost
problem of a n SQA team, it is required to have early meetings before the
start of the actual project, so that every requirement, either functional or
non-functional, is crystal clear and the development side and the SQA team
need not to make any assumptions of approach the client in between the
testing or development phase.
A percentage of change is to be setin early meetings, before the start of
project. It is now a good practice to entertain the client with a fixed percentage of minor changes that the client can change. In most cases, early
meetings decide a value of around 10-15 percent of minor changes that could
be entertained in a particular project. This helps the development and SQA
8

team to be prepared for any changes within that 10-15 percent of the area
in the project. As a result they develop and test in such a way that they
can easily make changes possible.

Conclusion

There are FIVE major problems faced by an SQA department of a software

house. Due to an alarming effect of these problems on the marketplace of the
software house companies, new strategies are being implemented to cover up
the SQA problems. The unclear requirements create most of the problems
for the SQA team. When the requirement documents are incomplete or
ambiguous, it is difficult for the SQA team to test the software accordingly.
Furthermore, change in requirements at mid or final stages of the development adds to the already unclear requirement problem of the SQA team.
Moreover, a software house is restricted in its domain of the preference of
software used for testing due to lack of expertise in multiple domains. The
SQA team cannot use that software which is out of their companys domain.
Whereas the most widely used technique for testing is the Integrated Testing Technique which can be applied or used with all kinds of software and
any platform for testing. The unclear requirements are mainly due to early
misunderstandings and lack of communication which leads to the difficulty
in testing phases to the SQA team. After all the study and analysis it can
be inferred that early meetings and proper documentation will lead to a
better understanding of the project and will cause fewer problems for the
SQA during the testing phases. Clearly stating every objective and every
requirement on paper will reduce the hurdles for the SQA team.
Another issue that is encountered by the SQA is the constantly changing requirements from the client side. When the development is in its final stages,
the client requires a major change in the functionality of the project. This
costs the most to any project for any software house. The client has to be
given a flexibility of change in the product. The company cannot go for a
fixed and un-flexible deal in the beginning. To resolve this issue, a particular
amount of change can be decided in the early meetings. This will ease the
development and SQA team in developing and testing the software according to that particular amount of change that can be done. The company
can keep a space for change from the very beginning. This will reduce the
sudden unexpected requirement of change from the client.
The study revealed that most software houses use integrated testing technique. Previously the testing was started after a complete build was released.
This resulted in major problems to the SQA whenever a bug has to be reported, which in turn affected the development side as well. The SQA team
will then have to test a completely new build again after that particular
bug was fixed, and this chain goes on. To recover this issue most software

houses now use integrated testing technique, which unites the developers
with the SQA team and the testing begins along with the development. After successful developments of a module or executable chunk of code, it is
sent to the SQA team to be tested. The basic benefit of this technique is the
feasibility in fixing bugs to the development side which used to be a menace
when testing was done after the release of the build.

Recommendation

To increase the possibility of successful early meetings it is highly recommended to set up several meetings before the actual development is started.
And it is also recommended for the SQA team to act as a bridge between
the development team and the client. Making notes at the meetings and
carrying out written agreements with the client will help in a creation of a
better requirement document.
As mentioned earlier in the conclusion that a particular percentage of the
change should be decided before the start of the project, it is highly recommended that the percentage should be kept around 10 to 12 percent and it
must be kept clear in the beginning that the changes are to be minor changes
and not the major ones. Keeping it fewer than 12 will make it within the
possibility range of the development team to implement minor changes in
the project.
The code should be readily tested as soon as a part of it is completed or
made executable by the development team. As the development is now done
in small executable modules, so it is recommended to the development team
to forward the module to the SQA for testing and debugging, rather than
sending the build after the modules have been integrated.

References
[1] Improvement of Key Problems of Software Testing in Quality Assurance
Nayyar Iqbal, M. Rizwan Jameel Qureshi
Department of Computer Science
COMSATS Institute of Information Technology
Lahore Pakistan
[2] Problems in Software Quality Assurance and Reasons
Mohammed Alshammri
Faculty of Engineering and Information Technology, University of Technology, Sydney
Sydney, NSW 2007, Australia
[3] Software quality challenges and practice recommendations
10

Aya R. Elgebeely
Application Developer
IBM
[4] Critical Issues in Software Quality Assurance: An Exploratory Study
William D. Burg, University of Alabama, Birmingham
Sanjay K. Singh, University of Alabama, Birmingham

11