Beruflich Dokumente
Kultur Dokumente
A test strategy
A test strategy is an outline that describes the testing portion of the software development cycle.
It is created to inform project managers, testers, and developers about some key issues of the
testing process. This includes the testing objective, methods of testing new functions, total time
and resources required for the project, and the testing environment.
In the test strategy is described how the product risks of the stakeholders are mitigated in the test
levels, which test types are performed in the test levels, and which entry and exit criteria apply.
The test strategy is created based on development design documents. The system design
document is the main one used and occasionally, the conceptual design document can be referred
to. The design documents describe the functionalities of the software to be enabled in the
upcoming release. For every set of development design, a corresponding test strategy should be
created to test the new feature sets.
Contents
• 1 Test Levels
• 2 Roles and Responsibilities
• 3 Environment Requirements
• 4 Testing Tools
• 5 Risks and Mitigation
• 6 Test Schedule
• 7 Regression Test Approach
• 8 Test Groups
• 9 Test Priorities
• 10 Test Status Collections and Reporting
• 11 Test Records Maintenance
• 12 Requirements traceability matrix
• 13 Test Summary
• 14 See also
• 15 References
Test Levels
The test strategy describes the test level to be performed. There are primarily three levels of
testing: unit testing, integration testing, and system testing. In most software development
organizations, the developers are responsible for unit testing. Individual testers or test teams are
responsible for integration and system testing.
Page
1
Software Testing 2010
Testing strategies should be reviewed by the developers. They should also be reviewed by test
leads for all levels of testing to make sure the coverage is complete yet not overlapping. Both the
testing manager and the development managers should approve the test strategy before testing
can begin.
Environment Requirements
Environment requirements are an important part of the test strategy. It describes what operating
systems are used for testing. It also informs clearly the necessary OS patch levels and security
updates required. For example, a certain test plan may require service pack 2 installed on the
Windows XP OS as a prerequisite for testing.
Testing Tools
There are two methods used in executing test cases: manual and automation. Depending on the
nature of the testing, it is usually the case that a combination of manual and automated testing is
the most optimal testing method. Planner should find the appropriate automation tool to reduce
total testing time.
Test Schedule
A test plan should make an estimation of how long it will take to complete the testing phase.
There are many requirements to complete testing phases. First, testers have to execute all test
cases at least once. Furthermore, if a defect was found, the developers will need to fix the
problem. The testers should then re-test the failed test case until it is functioning correctly. Last
but not the least, the tester need to conduct regression testing towards the end of the cycle to
make sure the developers did not accidentally break parts of the software while fixing another
part. This can occur on test cases that were previously functioning properly.
The test schedule should also document the number of tester available for testing. If possible,
assign test cases to each tester.
Page
2
Software Testing 2010
It is often difficult to make an accurate approximation of the test schedule since the testing phase
involves many uncertainties. Planners should take into account the extra time needed to
accommodate contingent issues. One way to make this approximation is to look at the time
needed by the previous releases of the software. If the software is new, multiplying the initial
testing schedule approximation by two is a good way to start.
Test Groups
From the list of requirements, we can identify related areas, whose functionality is similar. These
areas are the test groups. For example, in a railway reservation system, anything related to ticket
booking is a functional group; anything related with report generation is a functional group.
Same way, we have to identify the test groups based on the functionality aspect.
Test Priorities
Among test cases, we need to establish priorities. While testing software projects, certain test
cases will be treated as the most important ones and if they fail, the product cannot be released.
Some other test cases may be treated like cosmetic and if they fail, we can release the product
without much compromise on the functionality. This priority levels must be clearly stated. These
may be mapped to the test groups also.
Page
3
Software Testing 2010
test leader and the project manager, along with all the team members, in a central location. This
may be stored in a specific directory in a central server and the document must say clearly about
the locations and the directories. The naming convention for the documents and files must also
be mentioned.
Ideally each software developed must satisfy the set of requirements completely. So, right from
design, each requirement must be addressed in every single document in the software process.
The documents include the HLD, LLD, source codes, unit test cases, integration test cases and
the system test cases. Refer the following sample table which describes Requirements
Traceability Matrix process. In this matrix, the rows will have the requirements. For every
document {HLD, LLD etc}, there will be a separate column. So, in every cell, we need to state,
what section in HLD addresses a particular requirement. Ideally, if every requirement is
addressed in every single document, all the individual cells must have valid section ids or names
filled in. Then we know that every requirement is addressed. In case of any missing of
requirement, we need to go back to the document and correct it, so that it addressed the
requirement.
Test Summary
The senior management may like to have test summary on a weekly or monthly basis. If the
project is very critical, they may need it even on daily basis. This section must address what kind
of test summary reports will be produced for the senior management along with the frequency.
The test strategy must give a clear vision of what the testing team will do for the whole project
for the entire duration. This document will/may be presented to the client also, if needed. The
person, who prepares this document, must be functionally strong in the product domain, with a
very good experience, as this is the document that is going to drive the entire team for the testing
activities. Test strategy must be clearly explained to the testing team members right at the
beginning of the project.
White box testing strategy deals with the internal logic and structure of the code.
White box testing is also called as glass, structural, open box or clear box testing.
The tests written based on the white box testing strategy incorporate coverage of
the code written, branches, paths, statements and internal logic of the code etc.
In order to implement white box testing, the tester has to deal with the code and
hence is needed to possess knowledge of coding and logic i.e. internal working of
Page
4
Software Testing 2010
the code. White box test also needs the tester to look into the code and find out
which unit/statement/chunk of the code is malfunctioning.
Unit Testing:
The developer carries out unit testing in order to check if the particular module or
unit of code is working fine. The Unit Testing comes at the very basic level as it is
carried out as and when the unit of the code is developed or a particular
functionality is built.
Statement Coverage:
In this type of testing the code is executed in such a manner that every statement
of the application is executed at least once. It helps in assuring that all the
statements execute without any side effect.
Branch Coverage:
No software application can be written in a continuous mode of coding, at some
point we need to branch out the code in order to perform a particular functionality.
Branch coverage testing helps in validating of all the branches in the code and
making sure that no branching leads to abnormal behavior of the application.
Security Testing:
Security Testing is carried out in order to find out how well the system can protect
itself from unauthorized access, hacking – cracking, any code damage etc. which
deals with the code of application. This type of testing needs sophisticated testing
Page
5
Software Testing 2010
techniques.
Mutation Testing:
A kind of testing in which, the application is tested for the code that was modified
after fixing a particular bug/defect. It also helps in finding out which code and which
strategy of coding can help in developing the functionality effectively.
Besides all the testing types given above, there are some more types which fall
under both Black box and White box testing strategies such as: Functional testing
(which deals with the code in order to check its functional performance),
Incremental integration testing (which deals with the testing of newly added code in
the application), Performance and Load testing (which helps in finding out how the
particular code manages resources and give performance etc.) etc.
Black Box Testing is not a type of testing; it instead is a testing strategy, which does
not need any knowledge of internal design or code etc. As the name "black box"
suggests, no knowledge of internal logic or code structure is required. The types of
testing under this strategy are totally based/focused on the testing for requirements
and functionality of the work product/software application. Black box testing is
sometimes also called as "Opaque Testing", "Functional/Behavioral Testing" and
"Closed Box Testing".
The base of the Black box testing strategy lies in the selection of appropriate data
as per functionality and testing it against the functional specifications in order to
check for normal and abnormal behavior of the system. Now a days, it is becoming
common to route the Testing work to a third party as the developer of the system
knows too much of the internal logic and coding of the system, which makes it unfit
to test the application by the developer.
Various testing types that fall under the Black Box Testing strategy are: functional
testing, stress testing, recovery testing, volume testing, User Acceptance Testing
(also known as UAT), system testing, Sanity or Smoke testing, load testing, Usability
testing, Exploratory testing, ad-hoc testing, alpha testing, beta testing etc.
These testing types are again divided in two groups: a) Testing in which user plays a
role of tester and b) User is not required.
Page
6
Software Testing 2010
Functional Testing:
In this type of testing, the software is tested for the functional requirements. The
tests are written in order to check if the application behaves as expected.
Stress Testing:
The application is tested against heavy load such as complex numerical values,
large number of inputs, large number of queries etc. which checks for the
stress/load the applications can withstand.
Load Testing:
The application is tested against heavy loads or inputs such as testing of web sites
in order to find out at what point the web-site/application fails or at what point its
performance degrades.
Ad-hoc Testing:
This type of testing is done without any formal Test Plan or Test Case creation. Ad-
hoc testing helps in deciding the scope and duration of the various other testing and
it also helps testers in learning the application prior starting with any other testing.
Exploratory Testing:
This testing is similar to the ad-hoc testing and is done in order to learn/explore the
application.
Usability Testing:
This testing is also called as ‘Testing for User-Friendliness’. This testing is done if
User Interface of the application stands an important consideration and needs to be
specific for the specific type of user.
Smoke Testing:
This type of testing is also called sanity testing and is done in order to check if the
application is ready for further major testing and is working properly without failing
up to least expected level.
Recovery Testing:
Recovery testing is basically done in order to check how fast and better the
application can recover against any type of crash or hardware failure etc. Type or
extent of recovery is specified in the requirement specifications.
Volume Testing:
Volume testing is done against the efficiency of the application. Huge amount of
data is processed through the application (which is being tested) in order to check
Page
7
Software Testing 2010
Alpha Testing:
In this type of testing, the users are invited at the development center where they
use the application and the developers note every particular input or action carried
out by the user. Any type of abnormal behavior of the system is noted and rectified
by the developers.
Beta Testing:
In this type of testing, the software is distributed as a beta version to the users and
users test the application at their sites. As the users explore the software, in case if
any exception/defect occurs that is reported to the developers.
Which simply can be put as: A type of testing where we explore software, write and
execute the test scripts simultaneously.
Exploratory testing is a type of testing where tester does not have specifically
planned test cases, but he/she does the testing more with a point-of-view to explore
the software features and tries to break it in order to find out unknown bugs.
A tester who does exploratory testing, does it only with an idea to more and more
Page
8
Software Testing 2010
understand the software and appreciate its features. During this process, he/she
also tries to think of all possible scenarios where the software may fail and a bug
can be revealed.
While testing, if a tester comes across a bug, as a general practice, tester registers
that bug with the programmer. Along with registering the bug, tester also tries to
make it sure that he/she has understood the scenario and functionality properly and
can reproduce the bug condition. Once programmer fixes the bug, tester runs a test
case with the same scenario replication in which the bug had occurred previously. If
tester finds that the bug is fixed, he/she again tries to find out if the fix can handle
any such same type of scenario with different inputs.
For an example, lets consider that a tester finds a bug related to an input text field
on a form, where the field is supposed to accept any digit other than the digits from
1 to 100, which it fails to and accepts the number 100. Tester logs this bug to the
programmer and now is waiting for the fix. Once programmer fixes the bug, it sends
it across to the tester so as to get it tested. Tester now will try to test the bug with
same input value (100: as he/she had found that this condition causes application to
fail) in the field. If application rejects the number (100) entered by the tester,
he/she can safely close the defect.
Now, along with the above given test input value, which had revealed the bug,
tester tries to check if there is any other value from this set (0 to 100), which can
cause the application to fail. He/she may try to enter values from 0 to 100, or may
be some characters or a combination of character and numbers in any order. All
these test cases are thought by the tester as a variation of the type of value he/she
had entered previously and represent only one test scenario. This testing is called
exploratory testing, as the tester tried to explore and find out the possibility of
Page
9
Software Testing 2010
• Exploratory testing can uncover bugs, which are normally ignored (or hard to
find) by other testing strategies.
• It helps testers in learning new strategies, expand the horizon of their
imagination that helps them in understanding & executing more and more
test cases and finally improve their productivity.
• Exploratory testing helps tester in confirming that he/she understands the
application and its functionality properly and has no confusion about the
working of even a smallest part of it, hence covering the most important part
of requirement understanding.
• As in case of exploratory testing, we write and execute the test cases
simultaneously. It helps in collecting result oriented test scripts and shading
of load of unnecessary test cases which do not yield and result.
• Exploratory testing covers almost all type of testing, hence tester can be sure
of covering various scenarios once exploratory testing is performed at the
highest level (i.e. if the exploratory testing performed can ensure that all the
possible scenarios and test cases are covered).
Many people approached me for help with likes of the following questions:
-- I am new to Testing field and I do not know anything about it, will I be able to
secure a good future and survive in this field?
-- I hear about terminologies but cannot relate them logically, what can help me in
getting well versed with all the relevant terminologies of testing field?
Page
10
Software Testing 2010
I will try to address each and every question (and also all the other similar type of
questions asked by my readers over the period of time) with the help of following
scenarios and hope that it will help my readers.
I am totally new to this field, where and what should I begin with?
This seemingly stupid question is actually a very valid query, as many of the testers
are from the non-computer background. It indeed can be very hard for any new
person to understand the terminologies if that person is entering the IT field for the
very first time.
Once you start understanding, what is a program? What does it mean by execution?
What are the different languages? Etc. you gain enough momentum and
understanding which helps you in dealing with any new terminology in the
computer world. As you finish it, you should also take an overview of various
development methodologies used in the software development field such as Water-
Fall model, spiral model, incremental-integration model etc.
After you finish your first step, you should start looking into various terminologies
and definitions used in software testing field such as various types of testing,
testing strategies, methodologies etc. Even though most of the
definitions/terminologies are very simple and quite easy to understand, you should
try to create a small definition directory, which will be handy for you anytime you
wish to go through.
As soon as you are done with the above specified things, you should understand
what is a bug/error and life-cycle of a bug (For this you can go through my articles
"Software Testing – An Introduction", "Software Testing – Contents of a Bug",
"Software Testing – Bug Life Cycles" and "Software Testing – Bug And Statuses Used
During A Bug Life Cycle"). At this point, you will have enough knowledge to be able
to take a plunge in the vast field of Software Testing.
Page
11
Software Testing 2010
If you are testing a system without knowing the requirements, its just like searching
for something in this world and you don’t know what you are exactly expected to
look for and identify. This way whatever you come across, you will think that it’s the
same thing that you have been searching entire world for. And we have to admit
that it surely can be your mistake. If someone has asked you to test a system
without giving you the proper requirements, then almost everything you see in the
system will seem to be a bug for an example if the requirement says that there
should be a submit button on the login form but as the requirement has not been
specified, you surely will say that the submit button is a mistake of developer and it
should not be there (even when logically we know there WILL be a submit button on
the login form) and hence you will count it as a bug.
On other hand, if you really want to test a system and no matter what are the
requirements, then you surely can go ahead and do so. But then you will have to
use only monkey testing. The reason for this is again the same … you don’t know
anything about the system! So whatever you LOGICALLY think can be a potential
bug in the system, you can report it.
Anyone can test a system, even if a normal person brings a Television at his house
he can find out a normal defect that like defective power cable by trying to turn the
Television On, and mark my words … EVERYONE WILL DO THE SAME THING AT
FIRST. Then what’s the difference in that normal person and you as a tester (or say
GOOD Tester)? Here you can distinguish yourself from others; you can try to search
for more bugs in the system.
Page
12
Software Testing 2010
It also depends on the type of project you are working in, if your work demands you
to understand and use an automation tool then in that case you will have to surely
learn it.
A test case is a set of conditions or variables and inputs that are developed for a
particular goal or objective to be achieved on a certain application to judge its
capabilities or features.
It might take more than one test case to determine the true functionality of the
application being tested. Every requirement or objective to be achieved needs at
least one test case. Some software development methodologies like Rational
Unified Process (RUP) recommend creating at least two test cases for each
requirement or objective; one for performing testing through positive perspective
and the other through negative perspective.
1. Information
Information consists of general information about the test case. Information
incorporates Identifier, test case creator, test case version, name of the test
case, purpose or brief description and test case dependencies.
2. Activity
Activity consists of the actual test case activities. Activity contains
information about the test case environment, activities to be done at test
case initialization, activities to be done after test case is performed, step by
step actions to be done while testing and the input data that is to be supplied
for testing.
3. Results
Results are outcomes of a performed test case. Results data consist of
information about expected results and the actual results.
Page
13
Software Testing 2010
Test cases should be designed and written by someone who understands the
function or technology being tested. A test case should include the following
information -
Designing test cases can be time consuming in a testing schedule, but they are
worth giving time because they can really avoid unnecessary retesting or
debugging or at least lower it. Organizations can take the test cases approach in
their own context and according to their own perspectives. Some follow a general
step way approach while others may opt for a more detailed and complex approach.
It is very important for you to decide between the two extremes and judge on what
would work the best for you. Designing proper test cases is very vital for your
software testing plans as a lot of bugs, ambiguities, inconsistencies and slip ups can
be recovered in time as also it helps in saving your time on continuous debugging
and re-testing test cases.
As a tester tests an application and if he/she finds any defect, the life cycle of the
defect starts and it becomes very important to communicate the defect to the
developers in order to get it fixed, keep track of current status of the defect, find
out if any such defect (similar defect) was ever found in last attempts of testing etc.
For this purpose, previously manually created documents were used, which were
circulated to everyone associated with the software project (developers and
testers), now a days many Bug Reporting Tools are available, which help in tracking
and managing bugs in an effective way.
It’s a good practice to take screen shots of execution of every step during software
testing. If any test case fails during execution, it needs to be failed in the bug-
reporting tool and a bug has to be reported/logged for the same. The tester can
choose to first report a bug and then fail the test case in the bug-reporting tool or
fail a test case and report a bug. In any case, the Bug ID that is generated for the
Page
14
Software Testing 2010
At the time of reporting a bug, all the mandatory fields from the contents of bug
(such as Project, Summary, Description, Status, Detected By, Assigned To, Date
Detected, Test Lead, Detected in Version, Closed in Version, Expected Date of
Closure, Actual Date of Closure, Severity, Priority and Bug ID etc.) are filled and
detailed description of the bug is given along with the expected and actual results.
The screen-shots taken at the time of execution of test case are attached to the bug
for reference by the developer.
After reporting a bug, a unique Bug ID is generated by the bug-reporting tool, which
is then associated with the failed test case. This Bug ID helps in associating the bug
with the failed test case.
After the bug is reported, it is assigned a status of ‘New’, which goes on changing as
the bug fixing process progresses.
If more than one tester are testing the software application, it becomes a possibility
that some other tester might already have reported a bug for the same defect
found in the application. In such situation, it becomes very important for the tester
to find out if any bug has been reported for similar type of defect. If yes, then the
test case has to be blocked with the previously raised bug (in this case, the test
case has to be executed once the bug is fixed). And if there is no such bug reported
previously, the tester can report a new bug and fail the test case for the newly
raised bug.
If no bug-reporting tool is used, then in that case, the test case is written in a
tabular manner in a file with four columns containing Test Step No, Test Step
Description, Expected Result and Actual Result. The expected and actual results are
written for each step and the test case is failed for the step at which the test case
fails.
This file containing test case and the screen shots taken are sent to the developers
for reference. As the tracking process is not automated, it becomes important keep
updated information of the bug that was raised till the time it is closed.
(Please Note: The above given procedure of reporting a bug is general and not
based on any particular project. Most of the times, the bug reporting procedures,
values used for the various fields used at the time of reporting a bug and bug
tracking system etc. may change as par the software testing project and company
requirements.)
Page
15
Software Testing 2010
Complete list of contents of a bug/error/defect that are needed at the time of raising
a bug during software testing. These fields help in identifying a bug uniquely.
When a tester finds a defect, he/she needs to report a bug and enter certain fields,
which helps in uniquely identifying the bug reported by the tester. The contents of a
bug are as given below:
Project: Name of the project under which the testing is being carried out.
Subject: Description of the bug in short which will help in identifying the bug. This
generally starts with the project identifier number/string. This string should be clear
enough to help the reader in anticipate the problem/defect for which the bug has
been reported.
Description: Detailed description of the bug. This generally includes the steps that
are involved in the test case and the actual results. At the end of the summary, the
step at which the test case fails is described along with the actual result obtained
and expected result.
Summary: This field contains some keyword information about the bug, which can
help in minimizing the number of records to be searched.
Assigned To: Name of the developer who is supposed to fix the bug. Generally this
field contains the name of developer group leader, who then delegates the task to
member of his team, and changes the name accordingly.
Test Lead: Name of leader of testing team, under whom the tester reports the bug.
Detected in Version: This field contains the version information of the software
application in which the bug was detected.
Closed in Version: This field contains the version information of the software
application in which the bug was fixed.
Date Detected: Date at which the bug was detected and reported.
Expected Date of Closure: Date at which the bug is expected to be closed. This
depends on the severity of the bug.
Actual Date of Closure: As the name suggests, actual date of closure of the bug
i.e. date at which the bug was fixed and retested successfully.
Page
16
Software Testing 2010
Priority: Priority of the bug fixing. This specifically depends upon the functionality
that it is hindering. Generally Medium, Low, High, Urgent are the type of severity
that are used.
Severity: This is typically a numerical field, which displays the severity of the bug.
It can range from 1 to 5, where 1 is high severity and 5 is the lowest.
Status: This field displays current status of the bug. A status of ‘New’ is
automatically assigned to a bug when it is first time reported by the tester, further
the status is changed to Assigned, Open, Retest, Pending Retest, Pending Reject,
Rejected, Closed, Postponed, Deferred etc. as per the progress of bug fixing
process.
Bug ID: This is a unique ID i.e. number created for the bug at the time of reporting,
which identifies the bug uniquely.
Test Case Failed: This field contains the test case that is failed for the bug.
Any of above given fields can be made mandatory, in which the tester has to enter
a valid data at the time of reporting a bug. Making a field mandatory or optional
depends on the company requirements and can take place at any point of time in a
Software Testing project.
(Please Note: All the contents enlisted above are generally present for any bug
reported in a bug-reporting tool. In some cases (for the customized bug-reporting
tools) the number of fields and their meaning may change as per the company
requirements.)
(Right from the first time any bug is detected till the point when the bug is fixed and closed, it is
assigned various statuses which are New, Open, Postpone, Pending Retest, Retest, Pending
Reject, Reject, Deferred, and Closed. For more information about various statuses used for a bug
Page
17
Software Testing 2010
during a bug life cycle, you can refer to article ‘Software Testing – Bug & Statuses Used During
A Bug Life Cycle’)
There are seven different life cycles that a bug can passes through:
Page
18
Software Testing 2010
This way, any bug that is found ends up with a status of Closed, Rejected, Deferred
or Postponed.
Software testing is all about caring for software quality. It helps in deploying error free software,
in time. Software testing helps to find out the deviations of software developed from the result
expected. The quality analyst engineers assure that the software testing reveals appropriate
results with respect to what the software was intended for ?
A software testing company works to deliver quality assured software applications – be it big or
small project. Software testing company analysis the difficulties keeping in view that each
project developed is unique. A project may have undergone several alterations, while its way to
deployment. Bugs get incorporated due to many factors. Starting form miscommunication,
defecting software coding, use of software development tools, timely delivery pressures, and the
ever changing requirements, all can contribute for flawed software. No matter what the case may
be, what counts is the defects should be fixed before the customer makes use of it.
Software testing company has a wide area to work on. The IT industry may be flooded with a
Page
19
Software Testing 2010
variety of software testing services, not to mention the wide diversity found in their baptism
scheme. There is compatibility testing, functional testing, performance testing, load testing,
regression testing, stress testing, and unit testing among a few to mention. A reliable software
testing company realizes the need for validating the software product not just once, but also
repeated validation for each new release of the software product or website.
A reputed software testing company follows specific guidelines in accordance to the client’s
requirements. They ensure to comply with the quality system with regards to software validation.
Software testing company follows the bug testing processes, to the max. The head quality analyst
(project manager) of software testing company is responsible for project planning and scheduling
it. Monitoring and reporting the project of software testing is handled with utmost diligence. For
each project, the software testing is different. This depends on the size, scope and usability of the
project. Each factor looked on with minute details prove worth in delivering a quality software
output after software testing services.
Software testing has positive impacts on quality software. It helps clients business and makes the
software developed more usable and more user-friendly.
Page
20
Software Testing 2010
can keep you ahead and provide you with quality results.
With the beginning of your project software testing phases begins, with the software
development life cycle testing goes along with it on various stages. Testing at client
place and at your end as BETA & ALPHA TESTING is done to provide the bug free
product to your consumers.
So it's always advisable to go for the professional software testing company which
goes with the process and identifies the bugs making your product error free. A
Company which can understand the importance of your project deadlines & meet
your expectations.
Software development is an industrious mission that is incomplete and trivial without Software
Testing as this has a major significance in the Software Development Life Cycle. Software
Development Life Cycle is the set of all the technical procedures required in the development of
software.
Software testing company evaluates the quality and checks the potency of the program for the
required output. Software testing is the most important functionality in the Software
Development life Cycle as it exhibits all the mistakes and errors in the developed software.
Without rectifying theses errors, which in the technical words are called the "bugs" the software
development is not considered to be complete. Hence, Software testing is an important parameter
for the assured quality of the product.
All software testing companies in and across India follow two methods of Software Testing, i.e.
manual testing and automated testing. Though when any of these methods are used by the
Software Testing Companies, they further branch out in various types of testing standards like
the White box testing, Black box testing, Incremental integration testing, Unit testing, Integration
testing, Functional testing, System testing, End-to-end testing, etc.depending on the given set of
requirements and cases.
Along with the modifications and changes in the technologies there is tremendous pressure on
the software development companies to render the best and faultless services. The software
testing companies in India take up this responsibility to provide an error less software
development to all its customers in a predetermined frame of time and as well be cost effective.
To survive and make one’s corporate identity its essential for all the Software development
companies to go through the process of software testing and follow the latest trends and methods
Page
21
Software Testing 2010
An efficient Software Testing is not only displaying the errors and rectifying them but also
underlining the requirements of the customer throughout the product cycle.
Software testing companies also need to introduce information that is new to project community
so as to keep them much aware of the standards and parameters followed by these companies.
However not all the companies may be well equipped with the in-house testing professionals, at
this point of time offshore software testing companies come in the scenario. Offshore software
testing companies’ confirm the correctness, completeness and compatibility of the software and
find all the missing links in the products.
The process of Software testing starts from the beginning of the project, where all the software
developers and programmers examine all the aspects of the data provided by the clients and dig
out all the flaws and issues. They also provide ideas and views to enhance the quality of the
software. This phase of testing is known as alpha testing. The other phase that appears in an
offshore Software Testing by the clients as and when they visit or use the software designed and
developed carries on Beta testing.
Its always advisable for the software to go through these phases as the errors and mistakes can be
recognized from both the ends and their rectifications can also be done simultaneously. Software
Testing companies have to take all the desired measures and make use of all the latest techniques
to perform this task. As it is very important to be sure that it does not only works but the
functionality also meets the requirement of the customers.
Deepraj Srivastava works to help all those who want information about the Multimedia
development services, content writing and articles writing for Software regarding category. To
know more about Software Development Company, e-commerce services , Testing services,
2D/3D animation,Search engine optimization Software Testing Company visit:
www.commediait.com
A Closer View at the Software Testing Company
Software testing company plays a vital role in the software development as it
ensures the accuracy and the totality of the software developed by a software
development company.
I would like to make a note that the following checklists are defined in most generic form and do
not promise to cover all processes that you are be required to go through and follow during your
work. There may be some processes which are completely missed out from the lists or it may
also contain processes which you don’t need to follow in your form of work.
Page
22
Software Testing 2010
• Check the scripts assigned to you: This is the first and foremost process in the list.
There is no specific logic used to assign scripts to testers who should execute them all,
but you may come across practices where you will be assigned script based on your work
load for the day or your skill to understand and execute it in least possible time.
• Check the status/comments of the defect in the Test Report Tool: Once you unveil a
bug, its very important to keep track of the status of it as you will have to re-test the bug
once it is fixed by a developer. Most of the times, general practice is to confirm if any fix
to a bug is successful as it also makes it sure that the tester can proceed with other tests
involving deeper side of that particular functionality. Sometimes, it also addresses issues
related to understanding of functionality of the system for example: if a tester registered a
defect, which is not an actual bug as per the programming/business logic. Then in that
case, a comment from developer might help in understanding the mistake committed by
the tester.
• Update the test data sheet with all values which are required such as user name,
functionality, test code etc.
• Use naming conventions defined as testing standards to define a bug appropriately.
• Take screen prints for the script executed using naming conventions and provide test data
that you used for the testing. The screen prints will help other testers and developers to
understand how the test was executed and it will also serve as a proof for you. If possible,
try to explain the procedure you followed, choice of data and your understanding etc.
• If your team is maintaining any type of tracking sheet, do not forget to update all the
tracking sheets for the bug, it’s status, time and date found, severity etc.
• If you are using a test reporting tool, do not forget to execute the script in the tool. Many
test reporting tools require scripts to be executed in order to initiate the life cycle of a
bug. For example Test Director needs script to be executed till the step where it the test
script failed, other test steps before failed test step are declared as passed.
• Update the tracking sheets with current status, status in reporting tools etc. if it is required
to be updated after you execute the script in the reporting tool.
• Check if you have executed all the scripts properly and updated the test reporting tool.
• After you complete your days work, it is better to do a peer to peer review. This step is
very important and often helps in finding out missing steps/processes.
• First of all, confirm with your test lead if the defect is valid.
• Follow the appropriate naming conventions while logging defects.
• Before submitting the defect, get it reviewed by Work Lead/Team Lead.
• Give appropriate description and names in the defect screen prints as per naming
conventions.
• After submitting defects attach the screen prints for the defect on Test Reporting Tool.
Page
23
Software Testing 2010
• Note down the defect number/unique identifier and update the test tracking sheet with
appropriate information.
• Maintain a defect log, defect tracking sheet, screen prints dump folder etc. for a backup.
• Confirm with your test lead/work lead if the scripts are really blocked due to an existing
bug.
• Block scripts with an active defect (Defect status: New/Assigned/Fixed/Reopen)
• Update the current script/defect in test reporting tool and tracking sheets with the defect
number/unique identifier, which is blocking the execution of the script or testing of the
defect.
• If a defect is retested successfully, then unblock all scripts/defects blocked by it.
At the end of day, send an update mail to your Team Lead/Work Lead which should include the
following:
Usability Testing:
As the term suggest, usability means how better something can be used over the purpose it has
been created for. Usability testing means a way to measure how people (intended/end user) find
it (easy, moderate or hard) to interact with and use the system keeping its purpose in mind. It is a
standard statement that "Usability testing measures the usability of the system".
Page
24
Software Testing 2010
Any changes suggested by the tester at the time of usability testing, are the most crucial points
that can change the stand of the system in intended/end user’s view. Developer/designer of the
system need to incorporate the feedbacks (here feedback can be a very simple change in look and
feel or any complex change in the logic and functionality of the system) of usability testing into
the design and developed code of the system (the word system may be a single object or an entire
package consisting more than one objects) in order to make system more and more presentable to
the intended/end user.
Developer often try to make the system as good looking as possible and also tries to fit the
required functionality, in this endeavor he may have forgotten some error prone conditions which
are uncovered only when the end user is using the system in real time.
Usability testing helps developer in studying the practical situations where the system will be
used in real time. Developer also gets to know the areas that are error prone and the area of
improvement.
In simple words, usability testing is an in-house dummy-release of the system before the actual
release to the end users, where developer can find and fix all possible loop holes.
The outcome/feedback is noted down based on observations of how the user is using the system
and what are all the possible ways that also may come into picture, and also based on the
behavior of the system and how easy/hard it is for the user to operate/use the system. User is also
asked for his/her feedback based on what he/she thinks should be changed to improve the user
interaction between the system and the end user.
Over the time period, many people have formulated various measures and models for performing
usability testing. Any of the models can be used to perform the test.
Page
25
Software Testing 2010
• Usability test can be modified to cover many other types of testing such as functional
testing, system integration testing, unit testing, smoke testing etc. (with keeping the main
objective of usability testing in mind) in order to make it sure that testing is done in all
the possible directions.
• Usability testing can be very economical if planned properly, yet highly effective and
beneficial.
• If proper resources (experienced and creative testers) are used, usability test can help in
fixing all the problems that user may face even before the system is finally released to the
user. This may result in better performance and a standard system.
• Usability testing can help in uncovering potential bugs and potholes in the system which
generally are not visible to developers and even escape the other type of testing.
Usability testing is a very wide area of testing and it needs fairly high level of understanding of
this field along with creative mind. People involved in the usability testing are required to
possess skills like patience, ability to listen to the suggestions, openness to welcome any idea,
and the most important of them all is that they should have good observation skills to spot and
fix the problems on fly.
Security Testing of any developed system (or a system under development) is all about finding
out all the potential loopholes and weaknesses of the system, which might result into loss/theft of
highly sensitive information or destruction of the system by an intruder/outsider. Security
Testing helps in finding out all the possible vulnerabilities of the system and help developers in
fixing those problems.
Need of Security Testing
• Security test helps in finding out loopholes that can cause loss of important information
and allow any intruder enter into the systems.
• Security Testing helps in improving the current system and also helps in ensuring that the
system will work for longer time (or it will work without hassles for the estimated time).
• Security Testing doesn’t only include conformance of resistance of the systems your
organization uses, it also ensures that people in your organization understand and obey
security policies. Hence adding up to the organization-wide security.
• If involved right from the first phase of system development life cycle, security testing
can help in eliminating the flaws into design and implementation of the system and in
turn help the organization in blocking the potential security loopholes in the earlier stage.
This is beneficial to the organization almost in all aspects (financially, security and even
efforts point of view).
Page
26
Software Testing 2010
Now a day, almost all organizations across the world are equipped with hundreds of computers
connected to each other through intranets and various types of LANs inside the organization
itself and through Internet with the outer world and are also equipped with data storage &
handling devices. The information that is stored in these storage devices and the applications that
run on the computers are highly important to the organization from the business, security and
survival point of view.
Any organization small or big in size, need to secure the information it possesses and the
applications it uses in order to protect its customer’s information safe and suppress any possible
loss of its business.
Security testing ensures that the systems and applications used by the organizations are secure
and not vulnerable to any type of attack.
(Please Note: The above given list of security types is based on the Open Source
Security Testing Methodology Manual of Pete Herzog and the Institute for Security
Page
27
Software Testing 2010
The best way to ensure security is to involve the security related assessments,
audits and various types of testing right from the first phase of system
development. The level and form of processes used in security testing of any
system varies depending upon the phase, condition and type of system under
testing.
1. What is testing?
In other words, you can consider an example as: suppose you are a
good cook and are expecting some guests at dinner. You start making
dinner; you make few very very very delicious dishes (off-course, those
which you already know how to make). And finally, when you are about
to finish making the dishes, you ask someone (or you yourself) to
check if everything is fine and there is no extra salt/chili/anything,
which if is not in balance, can ruin your evening (This is what called
'TESTING').
This procedure you follow in order to make it sure that you do not
serve your guests something that is not tasty! Otherwise your collar
will go down and you will regret over your failure!
Well, while making food, its ok to have something extra, people might
understand and eat the things you made and may well appreciate your
work. But this isn't the case with Software Project Development. If you
fail to deliver a reliable, good and problem free software solution, you
fail in your project and probably you may loose your client. This can
get even worse!
Page
28
Software Testing 2010
OR
But you must know the fact that, if you make something, you hardly
feel that there can be something wrong with what you have developed.
It's a common trait of human nature, we feel that there is no problem
in our designed system as we have developed it and it is perfectly
functional and fully working. So the hidden bugs or errors or problems
of the system remain hidden and they raise their head when the
system goes into production.
On the other hand, its a fact that, when one person starts checking
something which is made by some other person, there are 99%
chances that checker/observer will find some problem with the system
(even if the problem is with some spelling that by mistake has been
written in wrong way.). Really weird, isn't it? But that’s a truth!
Even though its wrong in terms of human behavior, this thing has been
used for the benefit of software projects (or you may say, any type of
project). When you develop something, you give it to get checked
(TEST) and to find out any problem, which never aroused while
development of the system. Because, after all, if you could minimize
the problems with the system you developed, it’s beneficial for
yourself. Your client will be happy if your system works without any
problem and will generate more revenues for you.
BINGO, it's really great, isn't it? That's why we need testing!
Page
29
Software Testing 2010
A tester is a person who tries to find out all possible errors/bugs in the
system with the help of various inputs to it. A tester plays an important
part in finding out the problems with system and helps in improving its
quality.
If you could find all the bugs and fix them all, your system becomes
more and more reliable.
A tester has to understand the limits, which can make the system
break and work abruptly. The more number of VALID BUGS tester finds
out, the better tester he/she is!
A perfect software product is built when every step is taken with full
consideration that ‘A right product is developed in a right manner’.
‘Software Verification & Validation’ is one such model, which helps the
system designers and test engineers to confirm that a right product is
build right way throughout the development process and improve the
quality of the software product.
Page
30
Software Testing 2010
‘Verification & Validation Model’ makes it sure that, certain rules are
followed at the time of development of a software product and also
makes it sure that the product that is developed fulfills the required
specifications. This reduces the risk associated with any software
project up to certain level by helping in detection and correction of
errors and mistakes, which are unknowingly done during the
development process.
What is Verification?
The standard definition of Verification goes like this: "Are we building
the product RIGHT?" i.e. Verification is a process that makes it sure
that the software product is developed the right way. The software
should confirm to its predefined specifications, as the product
development goes through different stages, an analysis is done to
ensure that all required specifications are met.
During the Verification, the work product (the ready part of the
Software being developed and various documentations) is
reviewed/examined personally by one ore more persons in order to
find and point out the defects in it. This process helps in prevention of
potential bugs, which may cause in failure of the project.
Walkthroughs:
Walkthrough can be considered same as inspection without formal
preparation (of any presentation or documentations). During the
Page
31
Software Testing 2010
Buddy Checks:
This is the simplest type of review activity used to find out bugs in a
work product during the verification. In buddy check, one person goes
through the documents prepared by another person in order to find out
if that person has made mistake(s) i.e. to find out bugs which the
author couldn’t find previously.
What is Validation?
Validation is a process of finding out if the product being built is right?
i.e. whatever the software product is being developed, it should do
what the user expects it to do. The software product should
functionally do what it is supposed to, it should satisfy all the
functional requirements set by the user. Validation is done during or at
the end of the development process in order to determine whether the
product satisfies specified requirements.
All types of testing methods are basically carried out during the
Validation process. Test plan, test suits and test cases are developed,
which are used during the various phases of Validation process. The
phases involved in Validation process are: Code Validation/Testing,
Integration Validation/Integration Testing, Functional
Validation/Functional Testing, and System/User Acceptance
Testing/Validation.
Page
32
Software Testing 2010
Integration Validation/Testing:
Integration testing is carried out in order to find out if different (two or
more) units/modules co-ordinate properly. This test helps in finding out
if there is any defect in the interface between different modules.
Functional Validation/Testing:
This type of testing is carried out in order to find if the system meets
the functional requirements. In this type of testing, the system is
validated for its functional behavior. Functional testing does not deal
with internal coding of the project, in stead, it checks if the system
behaves as per the expectations.
Please Note: The above given Testing phases also include many sub-
phases, which are not included here.
Organisations are turning to automated testing tools more often than ever before to
ensure that their applications are working properly prior to deployment. That is
particularly important today as more and more applications are written for use on
the Web which is the most public of arenas.
Page
33
Software Testing 2010
But for many IT and quality assurance managers, automation is a daunting task. So
how do you get quick wins with automation? This article aims to help you develop a
test strategy for automation that fits your circumstances:
Test automation may provide test execution elements, however what other ways
could computers help testing. For example: test data generation; installations;
file/database comparisons and analysing test results.
2. Decide what your automated test goal is (and re-evaluate your test
mission as it may change)
Efficiency
This can be one Test Automation Goal, where the key aspects may be reducing
testing costs, reducing the time in the testing phase, automating regression tests or
improving test coverage.
Service
Tightening build schedules, preventing destabilization, playing to computer and
human strengths and increasing management confidence in the product.
Multiplying resources
Platform testing, configuration testing, model based tests and Data driven tests
When embarking on the use of scripted test tools the same practices used in
software development apply. This includes code standards and reviews. The Key to
success with scripting is to:
-- Use a data driven approach, so that a simple change to data can spawn a whole
new raft of tests
-- Use keyword approaches to enable not technical users to 'drive' the automation.
Page
34
Software Testing 2010
There are many automated testing tools that can aid testing from excel
spreadsheets to Code analysers. The key here is to find tools that are right for your
environment.
What environment(s) do you wish to test? Web, SAP, Client/Server, .net, Java, UNIX
It is all too easy, however to select an automation tool and be tempted to try to use
it for the total automation of all applications. Some tasks do not lend themselves to
the tool, but can still be automated using other tools such as command files or PC
functionality. There may also be some processes where it is more efficient to test
manually. You should not attempt to automate these.
5. Decide who will perform the automation and use the tools
It is critical to decide who is going to use/and or create the tools. Don't treat test
automation as a part time project. Allocate dedicated individuals time so they can
provide real value. Overall when preparing a test automation strategy, plan to
achieve small successes and grow.
Page
35
Software Testing 2010
The primary objectives of preceding test phases (unit, integration and systems
testing) are those of defect identification and ensuring that each step in the process
of building the software delivers working products (verification).
The objective of Acceptance Testing is to give confidence that the software being
developed, or changed, satisfies the specified requirements, meets customer/user
expectations and is fit for business purpose (validation). Unlike the other test
phases, an objective of acceptance testing is not to actively look for faults. The
expectation should be that the preceding stages of testing have identified and
resolved software faults and that the system is ready for operational use.
Clear entry criteria may need to be put in place to ensure that the expectation of a
system being ‘production ready’ has been met. This is an opportunity to review the
effectiveness, completeness and outcome of previous test phases and to declare
any known problems that remain prior to acceptance test commencing.
As a testing phase with it’s own objectives, acceptance should not seek to duplicate
other test phases. However, it may repeat previously run test scenarios to provide
confidence that preceding test phases have been completed as appropriate. Those
involved in or responsible for acceptance should dictate the objectives and
coverage of the test phase. In doing so they are likely to consider the following:
• Have the acceptance test conditions been demonstrated?
• Do the test conditions model realistic business scenarios?
• Is there appropriate coverage of each business function?
• Is there appropriate coverage of system reports and screens
• Is there appropriate coverage of application menu options?
• Are updated training materials and system documentation subject to test?
It is important to minimise the number of changes taking place to the system during
the acceptance test phase. The cost of rework, to change software or system
operation, at this stage of development is high. Implemented changes may
invalidate testing that has been already conducted and require greater levels of
regression testing to be carried out. Ideally, the software under test should be
considered as ‘frozen’ during the acceptance phase.
Where faults are uncovered during acceptance, ‘work-arounds’ should first be
investigated before attempting to make changes to the software. However, it may
be necessary to agree the correction of high severity/priority problems and to
defer/timetable the correction of others before acceptance can occur. These steps
of investigation, review, agreement and/or deferral may need to be repeated until
the point where the system is considered as acceptable.
The type of project or software under test may allow for parts/modules of the
system that meet their acceptance criteria to be signed off in advance of others
that don’t. For projects of this type acceptance can be an ongoing and evolving
Page
36
Software Testing 2010
In order to minimise the risk of software faults and system changes, acceptance
should not just start at software delivery. It needs to be a function of the earliest
design and development stages. Early involvement of users in the system
requirements and design stages can minimise the risk of gaps and
misinterpretations occurring in end system functionality. The benefits reaped at
later stages can be the avoidance of re-work and the reduced cost of system
delivery.
"I know you believe you understand what you think I said, but I'm not sure you
realize that what you heard is not what I meant."
And so it goes with the software development process. That's why all of the truly
successful software development teams rely upon creating and presenting visual
prototypes, including screen shots or screen mockups, in an effort to improve
communications between all parties.
And let's not put all of the blame on the client for failing to understand what was
said during the requirements gathering process. The opportunity to "get it wrong"
exists between team members, between the team and the actual users, and
between the team and the project's major stakeholders.
While we all agree that using visual prototypes represent the best opportunity for
reducing project risk at the design phase, creating those visual prototypes can
actually involve a level of effort that's nearly equal to the development phase,
depending upon the size of the project and the number of screens involved.
To make matters worse, many top-notch software development teams don't have
system analysts on board. The task of developing the requirements often falls to the
programmer who has the best communication skills and who is capable of making
themselves understood by mere mortals who speak an entirely different language.
Page
37
Software Testing 2010
But even those communication skills are not enough to ensure that what the client
is asking for closely resembles what the development team is planning on building.
And that brings us back to the subject of visual screen prototype tools. These tools
have a lot in common with the fairy tale of Goldilocks and The Three Bears.
Some of them are too hard to use. Some of them are too expensive to use, and
some of them are too inflexible. It's hard to find the one that's "just right."
But all of that may be about to change thanks to a new visual prototype tool called
MockupScreens. And although it was designed BY a software development team
FOR software development teams, you'd think it was designed by Goldilocks herself
because they got it "just right."
MockupScreens flattens the learning curve for creating visual prototypes. ANYONE
can use this tool, and you don't have to know a thing about drawing.
The level of effort required to build rich, robust and 100% actual depictions of the
software is less than it takes to point and click your way through a basic PowerPoint
presentation. But the results are far more spectacular, and much more realistic,
because all of the users and team members see actual screens that look just as
they will when they are actually created by the development team.
If you already use a prototyping tool to deliver your screen, the chances are it falls
into one of these categories:
2. It's complicated to use and there are only one or two members of your
development team that can make it perform properly.
Or, worse, you're using some "draw" program to create your screens manually.
If any of the above scenarioes sound familiar to you, then you really need to check
out MockupScreens. Not only does the one-time license fee cost about the same as
a week's worth of "Venti, no room" at Starbucks for your development team, but it's
drop-dead easy to use as well.
In fact, MockupScreens is the fastest, easiest and least expensive way to get all of
the users and your development team nodding their heads at the same time and
actually being confident that they are all agreeing to the same thing!
Page
38
Software Testing 2010
It's probably the best way to eliminate most all of the project risk that occurs at the
specifications and design phase.
The development teams that put MockupScreens together went out of their way to
model the program after the simple process of manually sketching screens on
paper.
But if you want to kick it up a notch, you can. MockupScreens gives you the
capability to add comments, icons, questions and other elements right into the
drawings. You can hide these elements from the users, and make them visible only
to other development team members if you want to, or you can let it all hang out.
It's your choice.
You can even hold the room spellbound by having MockupScreens generate slide
shows that will take the users through each screen exactly as they will appear when
the system goes live.
Because MockupScreens is so easy to use, you can make changes to the screens in
response to user input, in real time, and continue right on with the meeting.
Features like that are what raises your visibility and enhances your professional
image in the eyes of the client.
But as I mentioned already, probably the best part of using MockupScreens is that
you don't have to be an artist to use it. Heck, you don't even have to be a
programmer. All you do is boot up MockupScreens and it's all gravy from there.
The keyboard shortcuts are intelligently designed and there's context-sensitive help
wherever you need it (even though you probably never will need it).
Real-world tests with brand new MockUpScreens users consistently show that it's a
snap to create as many as 5 screens in about 10 minutes! Try doing that with
whatever it is you're using now.
MockupScreens is going to make you and every member of your design team heros
in the eyes of your clients. And if you don't believe me, download the eval version at
Page
39
Software Testing 2010
The testing stage of software development is definitely the most complex and
arguably the most important. From my experience most organizations break down
and drop the ball during the crucial testing stage. Success requires very effective
test management.
Most internal company organizations lack the training and time to effectively act out
this vital stage in the software development process. Test management is the most
difficult aspect of effective testing because testing requires input from various
channels at differing times. Not only do you need to carefully map out tasks and
responsibilities but you also need to know what the performance expectations are. I
have found test management tools an invaluable resource in accomplishing these
tasks. There are management tools for nearly all aspects of software testing
including capturing requirements, storing test scripts, and defect management.
There are many different test management tools to choose from. The leader in the
industry is by far Mercury Interactive TestDirector. Mercury is the global leader in
business technology optimization (BTO). Mercury Interactive is one of the largest
and fastest growing enterprise software companies in the world. Estimates show
nearly 90% of Fortune 100 companies, and even several government agencies
around the world rely on Mercury Interactive to fulfill their BTO strategies. This is a
company that you can rely on for high quality and consistent results. Their
TestDirector program creates an uncomplicated way to execute and manage the
testing process. The primary advantage of using a test management tool is to help
you manage your testing in a quick and effective manner. These tools facilitate
communication between involved parties and simultaneously cut down on the total
amount of time spent developing and running tests. Common features of test
management tools include the ability to repeat the process for gathering
requirements, scheduling tests, analyzing results, and managing any defects. This
adds confidence that your software is going meet performance expectations. Some
common questions Mercury Interactive recommends that you ask several questions
before using TestDirector: Does your organization lack the software testing tools to
deploy high-quality applications quickly and effectively? Does your organization
have the appropriate level of communication, organization, documentation, and
structure in place for every testing project? Do you have a consistent, repeatable
testing process? Are you leveraging strategic sourcing initiatives for quality
assurance? Answering these questions will help you better evaluate your
organizations need for test management tools.
Page
40
Software Testing 2010
TestDirector has four modules: requirements, test plan, test lab and defects
management which are integrated to enable a smooth flow of information between
various aspects of the total testing process. It is completely web-enabled in order to
efficiently support communication and cooperation among testing teams that are
dispersed throughout a company. Utilizing a test management tool is truly a wise
decision. The efficiency and reliability it brings to an organization exuberates
confidence and control that will then be emulate in employee performance. As
processes of integration smoothly take place employees gain greater trust in
management and the of the company’s ability to be competitive and adapt.
Complete and thorough software testing with the help of TestDirector will keep the
business running smoothly during and after implementation. The potential risk of
after implementation finding bugs and glitches will cost you money to fix and a lot
of productivity. Businesses are run on systems. Good businesses have good
systems. Using test management software will ensure that your systems are built
on a solid foundation.
1. Planning
2. Analysis
3. Design
4. Software Development
5. Implementation
6. Software Testing
7. Deployment
8. Maintenance
Software Testing is an important factor in a product's life cycle, as the product will have greater
life, only when it works correctly and efficiently according to the customer's requirements.
Page
41
Software Testing 2010
Before moving further towards introduction to software testing, we need to know a few concepts
that will simplify the definition of software testing.
• Error: Error or mistake is a human action that produces wrong or incorrect result.
• Defect (Bug, Fault): A flaw in the system or a product that can cause the component to
fail or misfunction.
• Failure: It is the variance between the actual and expected result.
• Risk: Risk is a factor that could result in negativity or a chance of loss or damage.
Thus Software testing is the process of finding defects/bugs in the system, that occurs due to an
error in the application, which could lead to failure of the resultant product and increase in
probability of high risk. In short, software testing have different goals and objectives, which
often include:
1. finding defects;
2. gaining confidence in and providing information about the level of quality;
3. preventing defects.
If you are new to the field of software testing, then the article software testing for beginners will
be of great help.
The primary function of software testing is to detect bugs in order to correct and uncover it. The
scope of software testing includes execution of that code in various environment and also to
examine the aspects of code - does the software do what it is supposed to do and function
according to the specifications? As we move further we come across some questions such as
"When to start testing?" and "When to stop testing?" It is recommended to start testing from the
initial stages of the software development. This not only helps in rectifying tremendous errors
before the last stage, but also reduces the rework of finding the bugs in the initial stages every
now and then. It also saves the cost of the defect required to find it. Software testing is an
ongoing process, which is potentially endless but has to be stopped somewhere, due to the lack
of time and budget. It is required to achieve maximum profit with good quality product, within
the limitations of time and money. The tester has to follow some procedural way through which
he can judge if he covered all the points required for testing or missed out any. To help testers to
carry out these day-to-day activities, a baseline has to be set, which is done in the form of
checklists. Read more on checklists for software tester.
• Defects and Failures: As we discussed earlier, defects are not caused only due to the
coding errors, but most commonly due to the requirement gaps in the non-functional
requirement, such as usability, testability, scalability, maintainability, performance and
security. A failure is caused due to the deviation between an actual and an expected
result. But not all defects result to failures. A defect can turn into a failure due to the
Page
42
Software Testing 2010
change in the environment and or the change in the configuration of the system
requirements.
• Input Combination and Preconditions: Testing all combination of inputs and initial
state (preconditions), is not feasible. This means finding large number of infrequent
defects is difficult.
• Static and Dynamic Analysis: Static testing does not require execution of the code for
finding defects, whereas in dynamic testing, software code is executed to demonstrate the
results of running tests.
• Verification and Validation: Software testing is done considering these two factors.
1. Verification: This verifies whether the product is done according to the
specification?
2. Validation: This checks whether the product meets the customer requirement?
• Software Quality Assurance: Software testing is an important part of the software
quality assurance. Quality assurance is an activity, which proves the suitability of the
product by taking care of the quality of a product and ensuring that the customer
requirements are met.
Software test type is a group of test activities that are aimed at testing a component or system
focused on a specific test objective; a non-functional requirement such as usability, testability or
reliability. Various types of software testing are used with the common objective of finding
defects in that particular component.
Software testing is classified according to two basic types of software testing: Manual Scripted
Testing and Automated Testing.
• Unit Testing
• Integration Testing
• System Testing
• Acceptance Testing
1. Alpha Testing
2. Beta Testing
• Functional Testing
• Performance Testing
Page
43
Software Testing 2010
1. Load Testing
2. Stress Testing
• Smoke Testing
• Sanity Testing
• Regression Testing
• Recovery Testing
• Usability Testing
• Compatibility Testing
• Configuaration Testing
• Exploratory Testing
For further explanation of these concepts, read more on types of software testing.
Automated Testing: Manual testing is a time consuming process. Automation testing involves
automating a manual process. Test automation is a process of writing a computer program in the
form of scripts to do a testing which would otherwise need to be done manually. Some of the
popular automation tools are Winrunner, Quick Test Professional (QTP), LoadRunner, SilkTest,
Rational Robot, etc. Automation tools category also includes maintenance tool such as
TestDirector and many other.
The software testing methodologies or process includes various models that built the process of
working for a particular product. These models are as follows:
• Waterfall Model
• V Model
• Spiral Model
• Rational Unified Process(RUP)
• Agile Model
• Rapid Application Development(RAD)
• Test Plan: A test specification is called a test plan. A test plan is documented so that it
can be used to verify and ensure that a product or system meets its design specification.
• Traceability matrix: This is a table that correlates or design documents to test
documents. This verifies that the test results are correct and is also used to change tests
when the source documents are changed.
• Test Case: Test cases and software testing strategies are used to check the functionality
of individual component that is integrated to give the resultant product. These test cases
are developed with the objective of judging the application for its capabilities or features.
Page
44
Software Testing 2010
• Test Data: When multiple sets of values or data are used to test the same functionality of
a particular feature in the test case, the test values and changeable environmental
components are collected in separate files and stored as test data.
• Test Scripts: The test script is the combination of a test case, test procedure and test data.
• Test Suite: Test suite is a collection of test cases.
Software testing process is carried out in the following sequence, in order to find faults in the
software system:
Test
Sr.No Case Test Case Name Steps/Action Expected Results
Id
User views the page to check
whether it includes UserId and
Password textboxes with Screen dispalys user
Checking User Interface
1. TC1. appropriate labels. Also expects interface requirements
requirements.
that Submit and Cancel buttons according to the user.
are available with appropriate
captions
2. TC2. Textbox for UserId i)User types numbers into the i)Error message is
should: textbox. displayed for numeric
Page
45
Software Testing 2010
Finding of a defect or fault in the earlier parts of the software not only saves time and money, but
is also efficient in terms of security and profitability. As we move forward towards the different
Page
46
Software Testing 2010
levels of the software, it becomes difficult and tedious to go back for finding the problems in the
initial conditions of the components. The cost of finding the defect also increases. Thus it is
recommended to start testing from the initial stages of the life cycle.
There are various techniques involved alongwith the types of software testing. There is a
procedure that is to be followed for finding a bug in the application. This procedure is combined
into the life cycle of the bug in the form of contents of a bug, depending upon the severity and
priority of that bug. This life cycle is named as the bug life cycles, which helps the tester in
answering the question - how to log a bug?
There arises a need of measuring the software, both, when the software is under development
and after the system is ready for use. Though it is difficult to measure such an abstract constraint,
it is essential to do so. The elements that are not able to be measured, needs to be controlled.
There are some important uses of measuring the software:
• Code Coverage
• Cyclomatic complexity
• Cohesion
• Coupling
• Function Point Analysis
• Execution time
• Source lines of code
• Bug per lines of code
Page
47
Software Testing 2010
Software testing is a good career opportunity for those who are interested in the software
industry. Video game testing is an offshoot of software testing. There are many industries
specializing in this field. Believe it or not, you can actually get paid to test video games. You can
read more on how to become a video game tester.
I hope this article has helped you gain a deeper insight into software testing. If you are planning
to choose the software testing industry as your career ground, you might like to go through this
extensive list of software testing interview questions. Before you step out for a job in the testing
field or before you take your first step towards becoming a software tester, you can acquire these
software testing certifications.
Software testing certifications will not only boost up ones knowledge, but also prove to be
beneficial for his academic performance. There are some software testing certification programs
that can support the professional aspirations of software testers and quality assurance specialists.
Software testing is indeed a vast field and accurate knowledge is crucial to ensure the quality of
the software developed. I consider that this article on software testing tutorial must have given
you a clearer idea on various software testing types, methodologies and different software testing
strategies.
Page
48
Software Testing 2010
Software testing is an integral part of the software development life cycle (SDLC). Effectively
and efficiently testing a piece of code is equally important, if not more, than writing it. So what
is software testing? Well, for those of you who are new to software testing and quality assurance,
here's the answer to this question.
Software testing is nothing but subjecting a piece of code to both, controlled as well as
uncontrolled operating conditions, in an attempt to observe the output and examine whether it is
in accordance with certain pre-specified conditions. Different sets of test cases and testing
strategies are prepared, all of which aim at achieving one common goal - removing all the bugs
and errors from the code and making the software error-free and capable enough of providing
accurate and optimum outputs. There are different types of software testing techniques and
methodologies. A software testing methodology is different from a software testing technique.
We will have a look at a few software testing methodologies in the later part of this article.
The above software testing methods can be implemented in two ways - manually or by
automation. Manual software testing is done by human software testers who manually i.e.
physically check, test and report errors or bugs in the product or piece of code. In case of
automated software testing, the same process is performed by a computer by means of an
automated testing software such as WinRunner, LoadRunner, Test Director, etc.
Page
49
Software Testing 2010
• Waterfall model
• V model
• Spiral model
• RUP
• Agile model
• RAD
Waterfall Model
The waterfall model adopts a 'top down' approach regardless of whether it is being used for
software development or testing. The basic steps involved in this software testing methodology
are:
1. Requirement analysis
2. Test case design
3. Test case implementation
4. Testing, debugging and validating the code or product
5. Deployment and maintenance
In this methodology, you move on to the next step only after you have completed the present
step. There is no scope for jumping backward or forward or performing two steps
simultaneously. Also, this model follows a non-iterative approach. The main benefit of this
methodology is its simplistic, systematic and orthodox approach. However, it has many
shortcomings since bugs and errors in the code are not discovered until and unless the testing
stage is reached. This can often lead to wastage of time, money and valuable resources.
V Model
The V model gets its name from the fact that the graphical representation of the different test
process activities involved in this methodology resembles the letter 'V'. The basic steps involved
in this methodology are more or less the same as those in the waterfall model. However, this
model follows both a 'top-down' as well as a 'bottom-up' approach (you can visualize them
forming the letter 'V'). The benefit of this methodology is that in this case, both the development
and testing activities go hand-in-hand. For example, as the development team goes about its
requirement analysis activities, the testing team simultaneously begins with its acceptance testing
activities. By following this approach, time delays are minimized and optimum utilization of
resources is assured.
Spiral Model
As the name implies, the spiral model follows an approach in which there are a number of cycles
(or spirals) of all the sequential steps of the waterfall model. Once the initial cycle is completed,
a thorough analysis and review of the achieved product or output is performed. If it is not as per
the specified requirements or expected standards, a second cycle follows, and so on. This
Page
50
Software Testing 2010
methodology follows an iterative approach and is generally suited for very large projects having
complex and constantly changing requirements.
Agile Model
This methodology follows neither a purely sequential approach nor does it follow a purely
iterative approach. It is a selective mix of both of these approaches in addition to quite a few new
developmental methods. Fast and incremental development is one of the key principles of this
methodology. The focus is on obtaining quick, practical and visible outputs and results, rather
than merely following theoretical processes. Continuous customer interaction and participation is
an integral part of the entire development process.
This was a short overview of some commonly used software testing methodologies. With the
applications of information technology growing with every passing day, the importance of
proper software testing has grown multifold.
Software testing life cycle consists of the various stages of testing through which a software
product goes and describes the various activities pertaining to testing that are carried out on the
product. Here's an explanation of the STLC along with a software testing life cycle flow chart. It
Page
51
Software Testing 2010
won't be wrong to call this article, a software testing life cycle tutorial.
Every organization has to undertakes testing of each of its products. However, the way it is
conducted differs from one organization to another. This refers to the life cycle of the testing
process. It is advisable to carry out the testing process from the initial phases, with regard to the
Software Development Life Cycle or SDLC to avoid any complications.
Software testing has its own life cycle that meets every stage of the SDLC. The software testing
life cycle diagram can help one visualize the various software testing life cycle phases. They are
1. Requirement Stage
2. Test Planning
3. Test Analysis
4. Test Design
5. Test Verification and Construction
6. Test Execution
7. Result Analysis
8. Bug Tracking
9. Reporting and Rework
10. Final Testing and Implementation
11. Post Implementation
Page
52
Software Testing 2010
Requirement Stage
This is the initial stage of the life cycle process in which the developers take part in analyzing the
requirements for designing a product. Testers can also involve themselves as they can think from
the users' point of view which the developers may not. Thus a panel of developers, testers and
users can be formed. Formal meetings of the panel can be held in order to document the
requirements discussed which can be further used as software requirements specifications or
SRS.
Test Planning
Test planning is predetermining a plan well in advance to reduce further risks. Without a good
plan, no work can lead to success be it software-related or routine work. A test plan document
Page
53
Software Testing 2010
plays an important role in achieving a process-oriented approach. Once the requirements of the
project are confirmed, a test plan is documented. The test plan structure is as follows:
Test Analysis
Once the test plan documentation is done, the next stage is to analyze what types of software
testing should be carried out at the various stages of SDLC.
Test Design
Test design is done based on the requirements of the project documented in the SRS. This phase
decides whether manual or automated testing is to be done. In automation testing, different paths
Page
54
Software Testing 2010
for testing are to be identified first and writing of scripts has to be done if required. There
originates a need for an end to end checklist that covers all the features of the project.
Test Execution
Planning and execution of various test cases is done in this phase. Once the unit testing is
completed, the functionality of the tests is done in this phase. At first, top level testing is done to
find out top level failures and bugs are reported immediately to the development team to get the
required workaround. Test reports have to be documented properly and the bugs have to be
reported to the development team.
Result Analysis
Once the bug is fixed by the development team, i.e after the successful execution of the test case,
the testing team has to retest it to compare the expected values with the actual values, and declare
the result as pass/fail.
Bug Tracking
This is one of the important stages as the Defect Profile Document (DPD) has to be updated for
letting the developers know about the defect. Defect Profile Document contains the following
The contents of a bug well explain all the above mentioned things.
Page
55
Software Testing 2010
testing has to be done. Once the Quality Analyst assures that the product is ready, the software is
released for production. Before release, the software has to undergo one more round of top level
testing. Thus testing is an ongoing process.
Post Implementation
Once the tests are evaluated, the recording of errors that occurred during various levels of the
software testing life cycle, is done. Creating plans for improvement and enhancement is an
ongoing process. This helps to prevent similar problems from occuring in the future projects. In
short, planning for improvement of the testing process for future applications is done in this
phase.
This was an overview of the software testing life cycle. Thanks to the information technology
boom and continuous innovation in the software industry, the need for software testing is, and
will always be, growing.
Page
56
Software Testing 2010
Software development comprises of various phases which are collectively called as Software
Development Life Cycle (SDLC). At every step the software evolves into a more advanced
product. Software testing happens in the later stage of SDLC. During this phase, the product is
checked for errors as well as it is verified against the user requirements. Software bug is defined
as any error, flaw, fault, mistake which prevents the software from producing the expected
output. Bug is often the result of human errors either in the source code or the design itself. From
its discovery to its resolution, a bug passes through various stages, commonly called as 'bug life
cycle'.
Given below are the stages of a bug life span. Test reports describe in detail the behavior of bug
at each stage.
New
This is the first stage of bug life cycle in which the tester reports a bug. The presence of the bug
becomes evident when the tester tries to run the newly developed application and it does not
respond in an expected manner. This bug is then send to the testing lead for approval.
Open
When the bug is reported to the testing lead, he examines the bug by retesting the product. If he
finds that the bug is genuine, he approves it and changes its status to 'open'.
Assign
Once the bug has been approved and found genuine by the testing lead, it is then send to the
concerned software development team for its resolution. It can be assigned to the team who
created the software or it may be assigned to some specialized team. After assigning the bug to
the software team, the status of the bug is changed to 'assign'.
Test
Page
57
Software Testing 2010
The team to which the bug has been assigned works on the removal of bug. Once, they are
finished with fixing the bug, it is sent back to the testing team for a retest. However, before
sending the bug back to the testing team, its status is changed to 'test' in the report.
Deferred
If the development team changes the status of the bug to 'deferred', it means that the bug will be
fixed in the next releases of the software. There can be myriad reason why the software team
may not consider fixing the bug urgently. This includes lack of time, low impact of the bug or
negligible potential of the bug to induce major changes in the normal functioning of the software.
Rejected
Although, the testing lead might have approved the bug stating it as a genuine one, the software
development team may not always agree. Ultimately, it is the prerogative of the development
team to decide if the bug is really genuine or not. If they doubt the presence or impact of the bug,
then they may change its status to 'rejected'.
Duplicate
If the development team finds that the same bug has been repeated twice or there are two bugs
which point to the same concept, then the status of one bug is changed to 'duplicate'. In this case,
fixing one bug automatically takes care of the other bug.
Verified
If the software development team sends the fixed bug back for retesting, then the bug undergoes
rigorous testing procedure again. If at the end of the test, it is not found then its status is changed
to 'verified.'
Reopened
If the bug still exists, then its status is changed to 'reopened'. The bug then traverses the entire of
its life cycle once again.
Closed
If no occurrence of bug is reported and if the software functions normally, then the bug is
'closed.' This is the final stage in which the bug has been fixed, tested and approved.
Read more on
Bug life cycle is an important concept in software testing. Understanding bug life
cycle is important from the point of view of delivering an error free software to the
customer.
Page
58
Software Testing 2010
Software testing industry presents a plethora of career opportunities for candidates, who are
interested in pursuing a career in the software industry. If you are the kind of a person, who does
not enjoy software development, yet very keen about making a career in the software field, then
software testing could be the right option for you. Software testing field offers several job
positions in testing, Quality Assurance (QA), Quality Control etc. However, you need to have
your basics in place, so as to improve your chances of acquiring a job in this particular industry.
Before applying for any IT job, it is imperative that you have a sound understanding of the field
you are hoping to venture in. Besides being technically sound, you should also keep yourself
abreast with the latest tools and trends in the software testing industry. Remember, software
testing is a volatile field, hence, the things that you learned in your curriculum may have become
obsolete by the time you are ready for a job. There are several types of software testing and
software testing methodologies, which you must be thorough with, before going for an interview.
Typically, your set of interview questions for software testing would depend upon the particular
area of software testing you are interested in. Hence, we have divided the questions into five
common categories. More on job interview tips.
• What will be the test cases for product testing? Give an example of test plan template.
• What are the advantages of working as a tester for a product based company as opposed
to a service based company?
• Do you know how a product based testing differs from a project based testing? Can you
give a suitable example?
• Do you know what is exactly meant by Test Plan? Name its contents? Can you give a
sample Test Plan for a Login Screen?
• How do you differentiate between testing a product and testing any web-based
application?
• What is the difference between Web based testing and Client server testing?
• How to perform SOAP Testing manually?
• Explain the significance of Waterfall model in developing a product.
Page
59
Software Testing 2010
• What do you do when there isn't enough time for thorough testing?
• What are the normal practices of the QA specialists with perspective of a software?
• Can you tell the difference between high level design and low level design?
• Can you tell us how Quality Assurance differs from Quality Control?
• You must have heard the term Risk. Can you explain the term in a few words? What are
the major components of the risk?
• When do you say your project testing is completed? Name the factors.
• What do you mean by a walk through and inspection?
• What is the procedure for testing search buttons of a web application both manually and
using Qtp8.2?
• Explain Release Acceptance Testing. Explain Forced Error Testing. Explain Data
Integrity Testing. Explain System Integration Testing.
• How does compatibility testing differ while testing in Internet explorer and testing in
Firefox?
• How do you know that all the scenarios for testing are covered?
• Can you explain the Testing Scenario? Also explain scenario based testing? Give an
example to support your answer.
• Consider a yahoo application. What are the test cases you can write?
• Differentiate between test scenario and test case?
• Is it necessary to create new Software requirement document, test planning report, if it is
a 'Migrating Project'?
• Explain the difference between smoke testing and sanity testing?
• What are all the scenarios to be considered while preparing test reports?
• What is an 'end to end' scenario?
• Other than requirement traceability matrix, what are the other factors that we need to
check in order to exit a testing process ?
• What is the procedure for finding out the length of the edit box through WinRunner?
Page
60
Software Testing 2010
• Can you have a defect with high severity and low priority and vice-versa i.e high priority
and low severity? Justify your answer.
• Can you explain the difference between a Bug and a Defect? Explain the phases of bug
life cycle
• What are the different types of Bugs we normally see in any of the projects? Also include
their severity.
• What is the difference between Bug Resolution Meeting and Bug Review Committee?
Who all participate in Bug Resolution Meeting and Bug Review Committee?
• Can you name some recent major computer system failures caused by software bugs?
What do you mean by 'Reproducing a bug'? What do you do, if the bug was not
reproducible?
Software testing is a vast field and there is really no dearth of software testing interview
questions. You can explore the Internet for more software testing interview questions and of
course, the solutions. Hope this article helps you to get the job of your dreams. Good Luck!
Page
61
Software Testing 2010
In our day-to-day life, when we go out, shopping any product such as vegetable, clothes, pens,
etc. we do check it before purchasing them for our satisfaction and to get maximum benefits. For
example, when we intend to buy a pen, we test the pen before actually purchasing it i.e. if its
writing, does it break if it falls, does it work in extreme climatic conditions, etc. So, though its
the software, hardware or any product, testing turns to be mandatory.
• Manual Scripted Testing: This is considered to be one of the oldest type of software
testing methods, in which test cases are designed and reviewed by the team, before
executing it.
• Automated Testing: This software testing type applies automation in the testing, which
can be applied to various parts of a software process such as test case management,
executing test cases, defect management, reporting of the bugs/defects. The bug life cycle
helps the tester in deciding how to log a bug and also guides the developer to decide on
the priority of the bug depending upon the severity of logging it. Software bug testing or
software testing to log a bug, explains the contents of a bug that is to be fixed. This can
be done with the help of various bug tracking tools such as Bugzilla and defect tracking
management tools like the Test Director.
Page
62
Software Testing 2010
Functional Testing: In this type of testing, the software is tested for the functional requirements.
This checks whether the application is behaving according to the specification.
Performance Testing: This type of testing checks whether the system is performing properly,
according to the user's requirements. Performance testing depends upon the Load and Stress
Testing, that is internally or externally applied to the system.
1. Load Testing : In this type of performance testing, the system is raised beyond the limits
in order to check the performance of the system when higher loads are applied.
2. Stress Testing : In this type of performance testing, the system is tested beyond the
normal expectations or operational capacity
Usability Testing: This type of testing is also called as 'Testing for User Friendliness'. This
testing checks the ease of use of an application. Read more on introduction to usability testing.
Regression Testing: Regression testing is one of the most important types of testing, in which it
checks whether a small change in any component of the application does not affect the
unchanged components. Testing is done by re-executing the previous versions of the application.
Smoke Testing: Smoke testing is used to check the testability of the application. It is also called
'Build Verification Testing or Link Testing'. That means, it checks whether the application is
ready for further major testing and working, without dealing with the finer details.
Sanity Testing: Sanity testing checks for the behavior of the system. This type of software
testing is also called as Narrow Regression Testing.
Parallel Testing: Parallel testing is done by comparing results from two different systems like
old vs new or manual vs automated.
Recovery Testing: Recovery testing is very necessary to check how fast the system is able to
recover against any hardware failure, catastrophic problems or any type of system crash.
Installation Testing: This type of software testing identifies the ways in which installation
procedure leads to incorrect results.
Page
63
Software Testing 2010
Configuration Testing: This testing is done to test for compatibility issues. It determines
minimal and optimal configuration of hardware and software, and determines the effect of
adding or modifying resources such as memory, disk drives and CPU.
Compliance Testing: This type of testing checks whether the system was developed in
accordance with standards, procedures and guidelines.
Error-Handling Testing: This software testing type determines the ability of the system to
properly process erroneous transactions.
Manual-Support Testing: This type of software testing is an interface between people and
application system.
Inter-Systems Testing: This type of software testing method is an interface between two or
more application systems.
Exploratory Testing: Exploratory Testing is a type of software testing, which is similar to ad-
hoc testing, and is performed to explore the software features. Read more on exploratory testing.
Volume Testing: This testing is done, when huge amount of data is processed through the
application.
Scenario Testing: This type of software testing provides a more realistic and meaningful
combination of functions, rather than artificial combinations that are obtained through domain or
combinatorial test design.
User Interface Testing: This type of testing is performed to check, how user-friendly the
application is. The user should be able to use the application, without any assistance by the
system personnel.
System Testing: System testing is the testing conducted on a complete, integrated system, to
evaluate the system's compliance with the specified requirements. This type of software testing
validates that the system meets its functional and non-functional requirements and is also
intended to test beyond the bounds defined in the software/hardware requirement specifications.
User Acceptance Testing: Acceptence Testing is performed to verify that the product is
acceptable to the customer and it's fulfilling the specified requirements of that customer. This
testing includes Alpha and Beta testing.
1. Alpha Testing: Alpha testing is performed at the developer's site by the customer in a
closed environment. This testing is done after system testing.
Page
64
Software Testing 2010
2. Beta Testing: This type of software testing is done at the customer's site by the customer
in the open environment. The presence of the developer, while performing these tests, is
not mandatory. This is considered to be the last step in the software development life
cycle as the product is almost ready.
Unit Testing: This type of testing is done at the developer's site to check whether a particular
piece/unit of code is working fine. Unit testing deals with testing the unit as a whole.
Static and Dynamic Analysis: In static analysis, it is required to go through the code in order to
find out any possible defect in the code. Whereas, in dynamic analysis the code is executed and
analyzed for the output.
Statement Coverage: This type of testing assures that the code is executed in such a way that
every statement of the application is executed at least once.
Decision Coverage: This type of testing helps in making decision by executing the application,
at least once to judge whether it results in true or false.
Condition Coverage: In this type of software testing, each and every condition is executed by
making it true and false, in each of the ways at least once.
Path Coverage: Each and every path within the code is executed at least once to get a full path
coverage, which is one of the important parts of the white box testing.
Integration Testing: Integration testing is performed when various modules are integrated with
each other to form a sub-system or a system. This mostly focuses in the design and construction
of the software architecture. Integration testing is further classified into Bottom-Up Integration
and Top-Down Integration testing.
1. Bottom-Up Integration Testing: In this type of integration testing, the lowest level
components are tested first and then alleviate the testing of higher level components
using 'Drivers'.
2. Top-Down Integration Testing: This is totally opposite to bottom-up approach, as it tests
the top level modules are tested and the branch of the module are tested step by step
using 'Stubs' until the related module comes to an end.
Security Testing: Testing that confirms, how well a system protects itself against unauthorized
internal or external, or willful damage of code, means security testing of the system. Security
testing assures that the program is accessed by the authorized personnel only. Read more on brief
introduction to security testing.
Page
65
Software Testing 2010
Mutation Testing: In this type of software testing, the application is tested for the
code that was modified after fixing a particular bug/defect.
Software testing methodologies and different software testing strategies help to get
through this software testing process. These various software testing methods show
you the outputs, using the above mentioned software testing types, and helps you
check if the software satisfies the requirement of the customer. Software testing is
indeed a vast subject and one can make a successful carrier in this field. You could
go through some software testing interview questions, to prepare yourself for some
software testing tutorials.
In short, the activity or process that proves the suitability of a product for the intended purpose
could be described as quality assurance. The quality assurance process takes care of the quality
of the products and ensures that customer requirements pertaining to the products are met. The
certifications used to assess different products have different parameters which should be
understood thoroughly. Total quality management is vital for the survival and profitability of
business nowadays.
Capability Maturity Model Integration (CMMi): The CMMi acts as a guiding force in the
improvement of the processes of an organization. The management of development, acquisition
and maintenance of the services and products of a company is also improved with the help of
CMMi. A set of proven practices are placed in a structure to assess the process area capability
and organizational maturity of a company. The priorities for improvement are established and it
is seen that these priorities are implemented properly with the help of CMMi.
People Capability Maturity Model (P-CMM): The P-CMM model is similar to SW-CMM
Page
66
Software Testing 2010
(Software Capability Maturity Model) in its approach. The objective of P-CMM is to improve
the software development capability of an organization by means of attracting, developing,
motivating, organizing and retaining the manpower or the required talent. The management and
development of the workforce of a company is guided by the P-CMM model. The P-CMM
model makes use of the best current practices used in organizational and human resource
development to achieve its objectives.
e Services Capability Model (eSCM): The eSCM model serves the needs of the BPO/ITES
industries. This model is used to assist the customers in measuring the service provider's
capability. The measurement is needed for establishing and managing the outsourcing
relationships which improve continually.
BS 7799: It is a security standard which originated in the mid-nineties, and till the year 2000 it
evolved into a model known known as BS EN ISO17799. It is difficult to comply with the
requirements/standards of this model since it covers the security issues comprehensively and
contains the control requirements that are significant in number.
The quality control and quality assurance certifications help in developing the trust of
customers in a particular product. The quality assurance certificates awarded by
various agencies also act as a motivational force for industries to maintain the
required standards. The short account of various agencies awarding the
certifications would help the concerned people in the industries.
Page
67
Software Testing 2010
Stress testing has different meaning for different industries where it is used. For a financial
industry/sector, stress testing means a process of testing financial instruments to find out their
robustness and level of accuracy they can maintain under extreme conditions such as sudden or
continuous market crash at a certain level, sudden or extreme change in various parameters, for
example interest rates, repo and reverse repo used in the financial sector, sudden rise or decline
in the price of materials that can affect financial projections etc. For the manufacturing industry,
stress testing may include different parameters and operating process for testing of different
systems. For medical industry, stress testing means a process that can help understand a patient’s
condition, etc.
In other words, stress testing helps find out the level of robustness and consistent or satisfactory
performance even when the limits for normal operation for the system (software/hardware) is
crossed.
Most important use of stress testing is found in testing of software and hardware that are
supposed to be operating in critical or real time situation. Such as a website will always be online
and the server hosting the website must be able to handle the traffic in all possible ways (even if
the traffic increases manifold), a mission critical software or hardware that works in real time
scenario etc. Stress testing in connection with websites or certain software is considered to be an
effective process of determining the limit, at which the system/software/hardware/website shows
robustness, is always available to perform its task, effectively manages the load than the normal
scenario and even shows effective error management under extreme conditions.
1. Almost 90% of the software/systems are developed with an assumption that they will be
operating under normal scenario. And even if it is considered that the limit of normal
operating conditions will be crossed, it is not considerably as high as it really could be.
2. The cost or effect of a very important (critical) software/system/website failure under
extreme conditions in real time can be huge (or may be catastrophic for the organization
or entity owning the software/system).
3. It is always better to be prepared for extreme conditions rather than letting the
system/software/website crash, when the limit of normal operation is crossed.
4. Testing carried out by the developer of the system/software/website may not be sufficient
to help unveil conditions which will lead to crash of the system/software when it is
actually submitted to the operating environment.
Page
68
Software Testing 2010
5. It's not always possible to unveil possible problems or bugs in a system/software, unless
it is subjected to such type of testing.
To help overcome problems like denial of service attacks, in case of web servers for a web site,
security breach related problems due to spamming, hacking and viruses etc., problems arising
out of conditions where software/system/website need to handle requests for
resource allocation for requesting processes at the time when all the required
resources are already allocated to some other process that needs some more
resources to complete its work (which is called as deadlock situation), memory leak,
race condition etc.
This type of testing is mostly done with the help of various stress testing softwares
available in market. These tools are configured to automate a process of increasing
stress (i.e. creation and increasing degree of adverse environment) on a
system/software/website and capturing values of various parameters that help
confirm the robustness, availability and performance of the
system/software/website being tested. Few of the actions involved in stress testing
are bombarding a website with huge number of requests, running of many resource
hungry applications in a computer, making numerous attempts to access ports of a
computer in order to hack it and use it for various purposes such as spamming,
spreading virus etc.
Intensity of the adverse conditions is increased slowly while measuring all the
parameters till the point where the system/software/website crashes. The collected
data (observation and parameter values) are used for further improvement of the
system/software/website.
Acceptance testing (also known as user acceptance testing) is a type of testing carried out in
order to verify if the product is developed as per the standards and specified criteria and meets all
the requirements specified by customer. This type of testing is generally carried out by a
user/customer where the product is developed externally by another party.
Acceptance testing falls under black box testing methodology where the user is not very much
interested in internal working/coding of the system, but evaluates the overall functioning of the
system and compares it with the requirements specified by them. User acceptance testing is
considered to be one of the most important testing by user before the system is finally delivered
or handed over to the end user.
Acceptance testing is also known as validation testing, final testing, QA testing, factory
acceptance testing and application testing etc. And in software engineering, acceptance testing
Page
69
Software Testing 2010
may be carried out at two different levels; one at the system provider level and another at the end
user level (hence called user acceptance testing, field acceptance testing or end-user testing).
Acceptance testing in software engineering generally involves execution of number test cases
which constitute to a particular functionality based on the requirements specified by the user.
During acceptance testing, the system has to pass through or operate in a computing environment
that imitates the actual operating environment existing with user. The user may choose to
perform the testing in an iterative manner or in the form of a set of varying parameters (for
example: missile guidance software can be tested under varying payload, different weather
conditions etc.).
The outcome of the acceptance testing can be termed as success or failure based on the critical
operating conditions the system passes through successfully/unsuccessfully and the user’s final
evaluation of the system.
The test cases and test criterion in acceptance testing are generally created by end user and
cannot be achieved without business scenario criteria input by user. This type of testing and test
case creation involves most experienced people from both sides (developers and users) like
business analysts, specialized testers, developers, end users etc.
1. Test cases are created with the help of business analysts, business customers (end users),
developers, test specialists etc.
2. Test cases suites are run against the input data provided by the user and for the number of
iterations that the customer sets as base/minimum required test runs.
3. The outputs of the test cases run are evaluated against the criterion/requirements specified
by user.
4. Depending upon the outcome if it is as desired by the user or consistent over the number
of test suites run or non conclusive, user may call it successful/unsuccessful or suggest
some more test case runs.
5. Based on the outcome of the test runs, the system may get rejected or accepted by the
user with or without any specific condition.
Alpha Testing & Beta Testing: Alpha testing is a type of acceptance testing carried out at
Page
70
Software Testing 2010
developer’s site by users (internal staff). In this type of testing, the user goes on testing the
system and the outcome is noted and observed by the developer simultaneously.
Beta testing is a type of testing done at user’s site. The users provide their feedback to the
developer for the outcome of testing. This type of testing is also known as field testing. Feedback
from users is used to improve the system/product before it is released to other users/customers.
Compatibility testing is one of the several types of software testing performed on a system that is
built based on certain criteria and which has to perform specific functionality in an already
existing setup/environment. Compatibility of a system/application being developed with, for
example, other systems/applications, OS, Network, decide many things such as use of the
system/application in that environment, demand of the system/application etc. Many a times,
users prefer not to opt for an application/system just because it is not compatible with any other
system/application, network, hardware or OS they are already using. This leads to a situation
where the development efforts taken by developers prove to be in vain.
Compatibility testing can be automated using automation tools or can be performed manually
and is a part of non-functional software testing.
Page
71
Software Testing 2010
has similar configuration as the actual environment in which the system/application is supposed
to fit and start working).
Compatibility testing can help developers understand the criteria that their
system/application needs to attain and fulfill, in order to get accepted by intended
users who are already using some OS, network, software and hardware etc. It also
helps the users to find out which system will better fit in the existing setup they are
using.
The most important use of the compatibility testing is as already mentioned above:
to ensure its performance in a computing environment in which it is supposed to
Page
72
Software Testing 2010
Page
73
Software Testing 2010
Page
74
Software Testing 2010
Page
75
Software Testing 2010
Page
76