VIKRAM

Software Testing: Testing is a process of executing a program with the intent of finding
error.

Software Engineering: Software Engineering is the establishment and use of sound

engineering principles in order to obtain economically software that is more reliable and
works efficiently on real machines.

Software engineering is based on Computer Science, Management Science, and

Economics, Communication Skills and Engineering approach.

What should be done during testing?

Confirming product as
• Product that has been developed according to specifications
• Working perfectly
• Satisfying customer requirements

Why should we do testing?

• Error free superior product
• Quality Assurance to the client
• Competitive advantage
• Cut down costs

How to test?
Testing can be done in the following ways:
• Manually
• Automation (By using tools like WinRunner, LoadRunner, Test Director …)
• Combination of Manual and Automation.

Software Development Phases:

Information Gathering: It encompasses requirements gathering at the strategic business
level.

Planning: To provide a framework that enables the management to make reasonable

estimates of

• Resources
• Cost
• Schedules
• Size

Requirements Analysis: Data, Functional and Behavioral requirements are identified.

• Data Modeling: Defines data objects, attributes, and relationships.

• Functional Modeling: Indicates how data are transformed in the system.
 VIKRAM

• Behavioral Modeling: Depicts the impact of events.

Design: Design is the engineering representation of product that is to be built.

• Data Design: Transforms the information domain model into the data structures
that will be required to implement the software.
• Architectural design: Relationship between major structural elements of the
software. Represents the structure of data and program components that are
required to build a computer-based system.
• Interface design: Creates an effective communication medium between a human
and a computer.
• Component level Design: Transforms structural elements of the software
architecture into a procedural description of software components.

Coding: Translation into source code (Machine readable form)

Testing: Testing is a process of executing a program with the intent of finding error

• Unit Testing: It concentrates on each unit (Module, Component…) of the software

as implemented in source code.
• Integration Testing: Putting the modules together and construction of software
architecture.
• System and Functional Testing: Product is validated with other system elements
are tested as a whole
• User Acceptance Testing: Testing by the user to collect feedback.

Maintenance: Change associated with error correction, adaptation and enhancements.

• Correction: Changes software to correct defects.

• Adaptation: Modification to the software to accommodate changes to its external
environment.
• Enhancement: Extends the software beyond its original functional requirements.
Prevention: Changes software so that they can be more easily corrected, adapted and
enhanced.
 VIKRAM

BRS: Consists of definitions of customer requirements. Also called as CRS/URS

S/wRS: Consists of functional requirements to develop and system requirements(s/w &

H/w) to use.

Review: A verification method to estimate completeness and correctness of documents.

HLDD: Consists of the overall hierarchy of the system in terms of modules.

LLDD: Consists of every sub module in terms of Structural logic (ERD) and Backend
Logic (DFD)

Prototype: A sample model of an application without functionality is called as prototype

(Screens)

WBT: A coding level testing technique to verify completeness and correctness of the
programs. Also called as Glass BT or Clear BT

BBT: It is an .exe level of testing technique to validate functionality of an application

with respect to customer requirements. During this test engineer validate internal
processing depends on external interface.

Verification: whether system is right or wrong?

Validation: whether system is right system or not?

Test Scenario: Is what the test is going to do, we can also use TEST Description instead of it.

Quality:

• Meet customer requirements

• Meet customer expectations (cost to use, speed in process or performance,
security)
• Possible cost
• Time to market

For developing the quality software we need LCD and LCT

LCD: A multiple stages of development stages and the every stage is verified for
completeness.

V model:
 VIKRAM

Build: When coding level testing over. It is a completely integration tested modules.
Then it is called a build. Build is developed after integration testing. (.exe)

Test Management: Testers maintain some documents related to every project. They will
refer these documents for future modifications.

Assessment of Development Plan

Information Gathering Prepare Test Plan
& Analysis Requirements Phase Testing

Design Phase Testing

Design and Coding Program Phase Testing (WBT)

Functional & System Testing

Install Build User Acceptance Testing
Test Environment Process

Port Testing
Maintenance Test Software Changes
Test Efficiency

Port Testing: This is to test the installation process.

Change Request: The request made by the customer to modify the software.

Defect Removal Efficiency:

DRE= a/a+b.
a = Total no of defects found by testers during testing.
b = Total no of defects found by customer during maintenance.

DRE is also called as DD (Defect Deficiency).

BBT, UAT and Test management process where the independent testers or testing team
will be involved.

Refinement form of V-Model: Due to cost and time point of view v-model is not
applicable to small scale and medium scale companies. These types of organizations are
maintaining a refinement form of v-model.
 VIKRAM

BRS/URS/CRS User Acceptance Testing

S/wRS Functional & System Testing

HLDD Integration Testing

LLDD Unit Testing

Code
Fig: Refinement Form of V-Model

Development starts with information gathering. After the requirements gathering

BRS/CRS/URS will be prepared. The Business Analyst does this.

During the requirements analysis all the requirements are analyzed at the end of this
phase S/wRS is prepared. It consists of the functional (customer requirements) + System
Requirements (h/w + S/w) requirements. It is prepared by the system analyst.

During the design phase two types of designs are done. HLDD and LLDD. Tech Leads
will be involved.

During the coding phase programmers develop programs.

During unit testing, they conduct program level testing with the help of WBT techniques.

During the Integration Testing, the testers and programmers or test programmers
integrating the modules to test with respect to HLDD.

During the system and functional testing the actual testers are involved and conduct tests
based on S/wRS.

During the UAT customer site people are also involved, and they perform tests based on
the BRS.

From the above model the small scale and medium scale organizations are also conducts
life cycle testing. But they maintain separate team for functional and system testing.
 VIKRAM

Reviews during Analysis:

Quality Analyst decides on 5 topics after completion of information gathering and
analysis a review meeting conducted to decide following 5 factors.

1. Are they complete?

2. Are they correct?
3. Are they achievable?
4. Are they reasonable? (With respect to cost & time)
5. Are they testable?

Reviews during Design:

After the completion of analysis of customer requirements and their reviews, technical
support people (Tech Leads) concentrate on the logical design of the system. In this every
stage they will develop HLDD and LLDD.

After the completion of above like design documents, they (tech leads) concentrate on
review of the documents for correctness and completeness. In this review they can apply
the below factors.

• Is the design good? (Understandable or easy to refer)

• Are they complete? (All the customer requirements are satisfied or not)
• Are they correct? (The design flow is correct or not)
• Are they following able? (The design logic is correct or not)
• Do they handle error handling? (The design should be able to specify the positive
and negative flow also)

User Information

Login
User Inbox

Invalid User

Unit Testing:
After the completion of design and their reviews programmers are concentrating on
coding. During this stage they conduct program level testing, with the help of the WBT
techniques. This WBT is also known as glass box testing or clear box testing.

WBT is based on the code. The senior programmers will conduct testing on programs
WBT is applied at the module level.
 VIKRAM

There are two types of WBT techniques, such as

1. Execution Testing
 Basis path coverage (correctness of every statement execution.)
 Loops coverage (correctness of loops termination.)
 Program technique coverage (Less no of Memory Cycles and CPU
cycles during execution.)

2. Operations Testing: Whither the software is running under the customer expected
environment platforms (such as OS, compilers, browsers and etc…sys s/w.)

Integration Testing: After the completion of unit testing, development people

concentrate on integration testing, when they complete dependent modules of unit testing.
During this test programmers are verifying integration of modules with respect to HLDD
(which contains hierarchy of modules).

There are two types of approaches to conduct Integration Testing:

• Top-down Approach
• Bottom-up approach.

Stub: It is a called program. It sends back control to main module instead of sub module.
Driver: It is a calling Program. It invokes a sub module instead of main module.

Top-down: This approach starts testing, from the root.

Main Stub

Sub Sub
Module Module
Bottom-Up: This approach starts testing, from lower-level modules. Drivers are used to
connect the sub modules. (Ex login, create driver to accept default uid and pwd)
 VIKRAM

Driver

Sub
Module

Sub
Module
Sandwich: This approach combines the Top-down and Bottom-up approaches of the
integration testing. In this middle level modules are testing using the drivers and stubs.

Driver

Sub
Module
Stub

Sub Sub
Module Module

System Testing: After the completion of Coding and that level tests (U & I) development
team releases a finally integrated all modules set as a build. After receiving a stable build
from development team, separate testing team concentrate on functional and system
testing with the help of BBT.

This testing is classified into 4 divisions.

• Usability Testing (Ease to use or not. Low level Priority in Testing)

• Functional Testing (Functionality is correct or not. Medium Priority in Testing)
• Performance Testing (Speed of Processing. Medium Priority in Testing)
• Security Testing (To break the security of the system. High Priority in Testing)

From the testers point of view functional and usability tests are important.

Usability Testing: User friendliness of the application or build. (WYSIWYG.)

 VIKRAM

Usability testing consists of following subtests also.

I.User Interface Testing

• Ease of Use (understandable to end users to operate)

• Look & Feel (Pleasantness or attractiveness of screens)

• Speed in interface (Less no. Of events to complete a task.)

II.Manual Support Testing: In general, technical writers prepares user manuals after
completion of all possible tests execution and their modifications also. Now a days help
documentation is released along with the main application.

Development Team releases Build

User Interface Testing

Remaining System Testing techniques like

Functionality, Performance and Security Tests System Testing

Manual Support Testing

Help documentation is also called as user manual. But actually user manuals are prepared
after the completion of all other system test techniques and also resolving all the bugs.

Functional testing: During this stage of testing, testing team concentrate on " Meet
Customer Requirements". For performing what functionality, the system is developed
met or not can be tested.

For every project functionality testing is most important. Most of the testing tools, which
are available in the market, are of this type.

The functional testing consists of following subtests

Functionality or Requirements Testing: During this subtest, test engineers validates

3correctness of every functionality in our application build, through below coverage.
If they have less time to do system testing, they will be doing Functionality Testing only.
 VIKRAM

Functionality or Requirements Testing has following coverage:

• Behavioral Coverage (Object Properties Checking).

• Input Domain Coverage (Correctness of Size and Type of every i/p Object).
• Error Handling Coverage (Preventing negative navigation).
• Calculations Coverage (correctness of o/p values).
• Backend Coverage (Data Validation & Data Integrity of database tables).
• Service Levels (Order of functionality or services).
• Successful Functionality (Combination of above all).

All the above coverage is mandatory or must.

Input Domain Testing: During this test, the test engineer validates size and type of every
input object. In this coverage, test engineer prepares boundary values and equivalence
classes for every input object.

Boundary Value analysis:

Boundary values are used for testing the size and range of an object.

Equivalence Class Partitions:

Equivalence classes are used for testing the type of the object.

Recovery Testing: This test is also known as Reliability testing. During this test, test
engineers validate that, whether our application build can recover from abnormal
situations or not.

Abnormal

Backup & Recovery

Procedures

Normal

Compatibility Testing: This test is also known as portable testing. During this test, test
engineer validates continuity of our application execution on customer expected
platforms( like OS, Compilers, browsers, etc..)

During this compatibility two types of problems arises like

 VIKRAM

1. Forward compatibility
2. Backward compatibility

Forward compatibility:
The application which is developed is ready to run, but the project technology or
environment like OS is not supported for running.

Build OS

Backward compatibility:
The application is not ready to run on the technology or environment.

Build OS

Configuration Testing: This test is also known as Hardware Compatibility testing.

During this test, test engineer validates that whether our application build supports
different technology i.e. hardware devices or not?

Inter Systems Testing: This test is also known as End-to-End testing. During this test,
test engineer validates that whither our application build coexistence with other existing
software in the customer site to share the resources (H/w or S/w).

Installation Testing: Testing the applications, installation process in customer specified

environment and conditions.

Build Instal
1. Setup Program
+Required lation
S/w
components Customer Site 2. Easy Interface
to run Like
Environment
3. Occupied Disk Space

The following conditions or tests done in this installation process

• Setup Program: Whither Setup is starting or not?

• Easy Interface: During Installation, whither it is providing easy interface or not?

 VIKRAM

• Occupied Disk Space: How much disk space it is occupying after the installation?

Sanitation Testing: This test is also known as Garbage Testing. During this test, test
engineer finds extra features in your application build with respect to S/w RS.
Maximum testers may not get this type of problems.

Parallel or Comparitive testing: During this test, test engineer compares our
application build with similar type of applications or old versions of same application to
find competitiveness.

This comparative testing can be done in two views:

• Similar type of applications in the market.
• Upgraded version of application with older versions.
Performance Testing: It is an advanced testing technique and expensive to apply. During
this test, testing team concentrate on Speed of Processing.

This performance test classified into below subtests.

1. Load Testing
2. Stress Testing
3. Data Volume Testing
4. Storage Testing

Load Testing:
This test is also known as scalability testing. During this test, test engineer
executes our application under customer expected configuration and load to estimate
performance.

Load: No. of users try to access system at a time.

This test can be done in two ways

1. Manual Testing. 2.By using the tool, Load Runner.

Stress Testing:
During this test, test engineer executes our application build under
customer expected configuration and peak load to estimate performance.

Data Volume Testing:

A tester conducts this test to find maximum size of allowable or
maintainable data, by our application build.

Storage Testing:
Execution of our application under huge amounts of resources to estimate
storage limitations to be handled by our application is called as Storage Testing.
Trashing
Performance =
 VIKRAM

--
+

Resources
Security Testing: It is also an advanced testing technique and complex to apply.
To conduct this tests, highly skilled persons who have security domain knowledge.

This test is divided into three sub tests.

Authorization: Verifies author’s identity to check he is an authorized user or not.

Access Control: Also called as Privileges testing. The rights given to a user to do a
system task.

Encryption / Decryption:
Encryption- To convert actual data into a secret code which may not be understandable
to others.
Decryption- Converting the secret data into actual data.

Source Encryption Decryption Destination

Client Server

Destination Decryption Encryption Source

User Acceptance Testing: After completion of all possible system tests execution, our
organization concentrate on user acceptance test to collect feed back. To conduct user
acceptance tests, they are following two approaches like Alpha Test and Beta Test.

Note: In s/w development projects are two types based on the products like software
application (also called as Project) and Product.

Software Application (Project): Get requirements from the client and develop the
project. This software is for only one company. And has specific customer. For this Alpha
test will be done.
 VIKRAM

Product: Get requirements from the market and develop the project. This software may
have more than one company. And has no specific customer. For this β- Version or Trial
version will be released in the market to do Beta test.

Alpha Testing Beta Testing

For what software applications

For software products.
applicable to specific customer
By customer site like people.
By real customer
In customer site like environment.
In development site
Real environment.
Virtual environment
Collect Feedback.
Collect Feedback.

Testing during Maintenance:

After the completion of UA Testing, our organization
concentrates on Release Team (RT) formation. This team conducts Port Testing in
customer site, to estimate completeness and correctness of our application installation.

During this Port testing Release team validate below factors in customer site:

• Compact Installation
• Overall Functionality
• Input device handling
• Output device handling
• Secondary Storage Handling
• OS Error handling
• Co-existence with other Software

The release team does the above tests. After the completion of above testing, the Release
Team will gives training and application support in customer site for a period.

During utilization of our application by customer site people, they are sending some
Change Request (CR) to our company. When CR is received the following steps are done
Based on the type of CR there are two types,
1. Enhancement
2. Missed Defect
 VIKRAM

Change Request

Missed Defect
Enhancement
Impact Analysis
Impact Analysis CCB
Perform that change
Perform that change
Review old test process capability to
Test that S/w Change improve

Test that S/w Change

Change Control Board: It is the team that will handle customer requests for
enhancement changes.

Testing Stages Vs Roles:

Reviews in Analysis – Business Analyst / Functional Lead.

Reviews in Design – Technical Support / Technical Lead.
Unit Testing – Senior Programmer.
Integration Testing – Developer / Test Engineer.
Functional & System Testing – Test Engineer.
User Acceptance Testing – Customer site people with involvement of testing team.
Port Testing – Release Team.
Testing during Maintenance – Change Control Board

Testing Terminology:-

Monkey / Chimpanzee Testing: The coverage of main activities only in your application
during testing is called as monkey testing.

Exploratory Testing: Level by level of activity coverage of activities in your application

during testing is called exploratory testing.

Sanity Testing: This test is also known as Tester Acceptance Test (TAT). They test for
whither developed team build is stable for complete testing or not?
 VIKRAM

Development Team Released Build

Sanity Test / Tester Acceptance Test

Functional & System Testing

Smoke Testing: An extra shakeup in sanity testing is called as Smoke Testing. Testing
team rejects a build to development team with reasons, before start testing.

Be bugging: Development team release a build with known bugs to testing them.

Bigbang Testing: A single state of testing after completion of all modules development is
called Bigbang testing. It is also known as informal testing.

Incremental Testing: A multiple stages of testing process are called as incremental

testing. This is also known as formal testing.

Manual Vs Automation: A tester conducts a test on application without using any third
party testing tool. This process is called as Manual Testing. A tester conducts a test with
the help of software testing tool. This process is called as Automation.

Need for Automation:

When tools are not available they will do manual testing only. If your company already
has testing tools they may follow automation.

For verifying the need for automation they will consider following two types:

Impact of the test: It indicates test repetition Impact of the test: It indicates test
repetition. Criticality: Load testing, for 1000 users. Criticality indicates complex to apply
that test manually. Impact indicates test repetition.

Retesting: Re execution of our application to conduct same test with multiple test data is
called Retesting.

Regression Testing: The re execution of our test on modified build to ensure bug fix
work and occurrences of side effects is called regression testing.
Any dependent modules may also cause side effects.
 VIKRAM

Impacted Passed Tests

Modifie
d Build
Failed Tests
Build

11 Test Fail
Development
10 Tests Passed
Selection of Automation: Before starting one project level testing by one separate testing
team, corresponding project manager or test manager or quality analyst defines the need
of test automation for that project depends on below factors.

Type of external interface: GUI – Automation. CUI – Manual.

Size of external interface: Size of external interface is Large – Automation.

Size of external interface is small – Manual.

Expected No. of Releases: Several Releases – Automation. Less Release – Manual.

Maturity between expected releases: More Maturity – Manual.

Less Maturity – Automation.

Tester Efficiency: Knowledge of automation on tools to test engineers – Automation.

No Knowledge of automation on tools to test engineers – Manual.

Support from Senior Management: Management accepts – Automation.

Management rejects – Manual.
 VIKRAM

Testing Policy
C.E.O
Company Level
Test Strategy
Test Manager/
QA / PM
Test Methodology

Test Lead Test Plan

Test Cases

Test Procedure

Project Level
Test Lead, Test Test Script
Engineer

Test Log

Defect Report

Test Lead
Test Summary Report
 VIKRAM

Address

Testing Definition: Verification & Validation of S/w

Testing Process: Proper Test Planning before start testing
Testing Standard: 1 Defect per 250 LOC / 1 Defect per 10 FP
Testing Measurements: QAM, TMM, and PCM.

CEO Sign

QAM: Quality Assessment Measurements

TMM: Test Management Measurements
PCM: Process Capability Measurements

Test Strategy:
1. Scope & Objective: About need for testing in your organisation
2. Business Issues: Budget Controlling for testing
3. Test approach: defines the testing approach between development stages and
testing factors.
4. Test environment specifications: Required test documents developed by testing
team during testing.
5. Roles and Responsibilities: Defines names of jobs in testing team with required
responsibilities.
6. Communication & Status Reporting: Required negotiation between two
consecutive roles in testing.
7. Testing measurements and metrics: To estimate work completion in terms of
Quality Assessment, Test management process capability.
8. Test Automation: Possibilities to go test automation with respect to corresponding
project requirements and testing facilities / tools available.
9. Defect Tracking System: Required negotiation between the development and
testing team to fix defects and resolve.
10. Change and Configuration Management: required strategies to handle change
requests of customer site.
11. Risk Analysis and Mitigations: Common problems appear during testing and
possible solutions to recover.
12. Training plan: Need of training for testing to start/conduct/apply.
 VIKRAM

Test Factors:
1. Authorization:
Security Testing
Functionality / Requirements Testing
2. Access Control :
Security Testing
Functionality / Requirements Testing
3. Audit Trail :
Error Handling Testing
Functionality / Requirements Testing
4. Correctness:
All black box Testing Techniques
5. Continuity in Processing:
Execution Testing
Operations Testing
6. Coupling :
Inter Systems Testing
7. Ease of Use:
User Interface Testing
Manual Support Testing
8. Ease of Operate:
Installation testing
9. File Integrity:
Recovery Testing
Functionality / Requirements Testing
10. Reliability:
Recovery Testing
Stress Testing
11. Portable:
Compatibility Testing
Configuration Testing
12. Performance:
Load Testing
Stress Testing
Data Volume Testing
Storage Testing
13. Service Levels:
Stress Testing
Functionality / Requirements Testing
14. Methodology:
Compliance Testing
15. Maintainable:
Compliance Testing
 VIKRAM

Test Methodology: Test strategy defines over all approach. To convert an over all
approach into corresponding project level approach, quality analyst / PM defines test
methodology.

Step 1: Collect test strategy

Step 2: Project type

Project Type Information Gathering & Design Coding System Maintenance

Analysis Testing
Traditional Y Y Y Y Y
Off-the-Shelf X X X Y X
Maintenance X X X X Y

Step 3: Determine application type: Depends on application type and requirements the
QA decrease number of columns in the TRM.
Step 4: Identify risks: Depends on tactical risks, the QA decrease number of factors
(rows) in the TRM.
Step 5: Determine scope of application: Depends on future requirements / enhancements,
QA try to add some of the deleted factors once again. (Number of rows in the TRM)
Step 6: Finalize TRM for current project
Step 7: Prepare Test Plan for work allocation.

Testing Process:

Test Test Test

Test Design Test
Initiation Plannin Executio
Closur
g n
e
Regression
Testing Defect

Test
Report

PET (Process Experts Tools and Technology): It is an advanced testing process

developed by HCL, Chennai. This process is approved by QA forum of India. It is a
refinement form of V-Model.
 VIKRAM

Information Gathering (BRS)

Analysis (S/wRS)

Design (HLDD & LLDD) PM / QA Test Initiation

Coding Test Lead Test Planning

Unit Testing Study S/wRS & Design Docs

+
Integration Testing Test Design
Initial Build

Level – 0 (Sanity / Smoke / TAT)

Test Automation

Test Batches Creation

(Modified
Build) Next
Select a batch and starts
Bug execution (Level - 1 )
Resolving (Regression)
(Level – 2)

Defect Independent
Defect If u got any mismatch then
Fixing suspend that Batch
Report

Otherwise

Test Closure

Final Regression / Pre Acceptance / Release

/ Post Mortum / Level -3 Testing

User Acceptance Test

Sign Off
 VIKRAM

Test Planning: After completion of test initiation, test plan author concentrates on test
plan writing to define “what to test, how to test, when to test and who to test “.

What to test - Development Plan

How to test - S/wRS
When to test - Design Documents
Who to test - Team Formation

Development Plan & S/wRS & Team Formation

Design Documents
Identify tactical Risks
Test Plan
Prepare Test Plan
TRM
Review Test Plan

1. Team Formation
In general test planning process starts with testing team
formation, depends on below factors.

• Availability of Testers
• Test Duration
• Availability of test environment resources
The above three are dependent factors.

Test Duration:

Common market test team duration for various types of projects.

C/S, Web, ERP projects - SAP, VB, JAVA – Small - 3-5 months
System Software - C, C++ - Medium – 7-9 months
Machine Critical - Prolog, LISP - Big - 12-15 months

System Software Projects: Network, Embedded, Compilers …

Machine Critical Software: Robotics, Games, Knowledge base, Satellite, Air Traffic.

2. Identify tactical Risks

After completion of team formation, test plan author concentrates on risks analysis
and mitigations.

1) Lack of knowledge on that domain

2) Lack of budget
 VIKRAM

3) Lack of resources (h/w or tools)

4) Lack of testdata (amount)
5) Delays in deliveries (server down)
6) Lack of development process rigor
7) Lack of communication (Ego problems)

3. Prepare Test Plan

Format:
1) Test Plan id: Unique number or name
2) Introduction: About Project
3) Test items: Modules
4) Features to be tested: Responsible modules to test
5) Feature not to be tested: Which ones and why not?
6) Feature pass/fail criteria: When above feature is pass/fail?
7) Suspension criteria: Abnormal situations during above features testing.
8) Test environment specifications: Required docs to prepare during testing
9) Test environment: Required H/w and S/w
10) Testing tasks: what are the necessary tasks to do before starting testing
11) Approach: List of Testing Techniques to apply
12) Staff and training needs: Names of selected testing Team
13) Responsibilities: Work allocation to above selected members
14) Schedule: Dates and timings
15) Risks and mitigations: Common non technical problems
16) Approvals: Signatures of PM/QA and test plan author

4. Review Test Plan

After completion of test plan writing test plan author concentrate on review of that
document for completeness and correctness. In this review, selected testers also involved
to give feedback. In this reviews meeting, testing team conducts coverage analysis.

 S/wRS based coverage (What to test)

 Risks based coverage (Analyze risks point of view)
 TRM based coverage (Whither this plan tests all tests given in TRM)

Test Design:
After completion of test plan and required training days, every selected
test engineer concentrate on test designing for responsible modules. In this phase test
engineer prepares a list of testcases to conduct defined testing, on responsible modules.
There are three basic methods to prepare testcases to conduct core level testing.
 VIKRAM

 Business Logic based testcase design

 Input Domain based testcase design
 User Interface based testcase design

Business Logic based testcase design: In general test engineers are writing list of
testcases depends on usecases / functional specifications in S/wRS. A usecase in S/wRS
defines how a user can use a specific functionality in your application.

BRS

S/wRS
Usecases +
Functional TestCases
Specifications

HLDD

LLDD

Coding .Exe

To prepare testcases depends on usecases we can follow below approach:

Step 1: Collect responsible modules usecases

Step 2: select a usecase and their dependencies (Dependent & Determinant)
Step 2-1: identify entry condition
Step 2-2: identify input required
Step 2-3: identify exit condition
Step 2-4: identify output / outcome
Step2-5: study normal flow
Step 2-6: study alternative flows and exceptions
Step3: prepare list of testcases depends on above study
Step 4: review testcases for completeness and correctness

TestCase Format:

After completion of testcases selection for responsible modules, test engineer prepare an
IEEE format for every test condition.
 VIKRAM

TestCase Id: Unique number or name

TestCase Name: Name of the test condition
Feature to be tested: Module / Feature / Service
TestSuit Id: Parent batch Id’s, in which this case is participating as a member.
Priority: Importance of that testcase
Po – Basic functionality
P1 – General Functionality (I/p domain, Error handling)
P2 – Cosmetic TestCases
(Ex: p0 – os, p1-difft oss, p2 – look & feel)
Test Environment: Required H/w and S/w to execute the test cases
Test Effort: (Person Per Hour or Person / Hr) Time to execute this test case ( 20 Mins )
Test Duration: Date of execution
Test Setup: Necessary tasks to do before start this case execution
Test Procedure: Step by step procedure to execute this testcase.

Step No. Action I/p Required Expected Result Defect ID Comments

Test Design Test Execution

TestCase Pass/Fail Criteria: When that testcase is Pass, When that testcase is fail.

Input Domain based TestCase Design:

To prepare functionality and error handling testcases, test engineers are using UseCases
or functional specifications in S/wRS. To prepare input domain testcases test engineers
are depending on data model of the project (ERD & LLD)

Step1: Identify input attributes in terms of size, type and constraints.

(Size- range, type – int, float constraint – Primary key)
Step2: Identify critical attributes in that list, which are participating in data retrievals and
Manipulations.
Step3: Identify non critical attributes which are input, output type.
Step4: Prepare BVA & ECP for every attribute.

ECP ( Type ) BVA ( Size / Range )

Input Attribute Valid Invalid Minimum Maximum

Fig: Data Matrix

 VIKRAM

User Interface based testcase design:

To conduct UI testing, test engineer write a list of test cases, depends on our organization
level UI rules and global UI conventions.

For preparing this UI testcases they are not studying S/wRS, LLDD etc…
Functionality testcases source: S/wRS. I/P domain testcases source: LLDD

Testcases: For all projects applicable

Testcase1: Spelling checking
Tesecase2: Graphics checking (alignment, font, style, text, size, micro soft 6 rules)
Testcase3: Meaningful error messages or not. (Error Handling Testing – related message
is coming or not. Here they are testing that message is easy to understand or not)

TestCase4: Accuracy of data displayed (WYSIWYG) (Amount, d o b)

Testcase5: Accuracy of data in the database as a result of user input.

(Tc4 screen level, tc5 at database level)

Form Table

DSN
Bal 66.666 66.7

Testcase6: Accuracy of data in the database as a result of external factors?

DS

Mail Server
Image Image
compression Decompression

Mail + Mail +
.Gif .Gif

Import
Testcase7: Meaningful Help messages or not? (First 6 tc for uit and 7 manual support
testing)
 VIKRAM

Review Testcases: After completion of testcases design with required documentation

[IEEE] for responsible modules, testing team along with test lead concentrate on review
of testcases for completeness and correctness. In this review testing team conducts
coverage analysis

1. Business Requirements based coverage

2. UseCases based coverage
3. Data Model based coverage
4. User Interface based coverage
5. TRM based coverage

Fig: Requirements Validation / Traceability Matrix.

Business Requirements Sources (Use Cases, Data Model…) TestCases

****** ***** *
*
***** *
*
***** *
*
Test Execution:
Development Site Initial Build Testing Site

Level-0 (Sanity /
Smoke / TAT)

Stable Build Test Automation

Defect Report
Defect Fixing
Level-1
(Comprehensive)
8-9
Times

Bug Resolving Level-2 (Regression)

Modified Build

Level-3 (Final
Regression)

Test Execution levels Vs Test Cases:

Level 0 – P0
 VIKRAM

Level 1– P0, P1 and P2 testcases as batches

Level 2– Selected P0, P1 and P2 testcases with respect to modifications
Level 3– Selected P0, P1 and P2 testcases at build.

Test Harness = Test Environment + Test Bed

Build Version Control: Unique numbering system. (FTP or SMTP)

Server
Softbase
Build
FTP

Test
Environment

After defect reporting the testing team may receive

• Modified Build
• Modified Programs

To maintain this original builds and modified builds, development team use version
control softwares.

Server

1 2

Modified Modified
Build Programs
Test
Environment
Embed into
Old Build
 VIKRAM

Level 0 (Sanity / Smoke / TAT):

After receiving initial build from development team, testing team install into test
environment. After completion of dumping / installation testing team ensure that basic
functionality of that build to decide completeness and correctness of test execution.

During this testing, testing team observes below factors on that initial build.

1. Understandable: Functionality is understandable to test engineer.

2. Operable: Build is working without runtime errors in test environment.
3. Observable: Process completion and continuation in build is estimated by tester.
4. Controllable: Able to Start/ Stop processes explicitly.
5. Consistent: Stable navigations
6. Maintainable: No need of reinstallations
7. Simplicity: Short navigations to complete task.
8. Automatable: Interfaces supports automation test script creation.

This level-0 testing is also called as Testability or Octangle Testing (bcz based on 8
factors).

Test Automation: After receiving a stable build from development team, testing team
concentrate on test automation.

Test Automation two types: Complete and Selective.

Test Automation

* (All P0 and
Complete Selective carefully
selected P1
Testcases)
Level-1: (Comprehensive Testing):
After completion of stable build receiving from development team and automation,
testing team starts test execution of their testcases as batches. The test batch is also
known as TestSuit or test set. In every batch, base state of one testcase is end state of
previous testcase.
During this test batches execution, test engineers prepares test log with three types
of entries.

1. Passed
2. Failed
3. Blocked
Passed: All expected values are equal to actual.
 VIKRAM

Failed: Any expected value is variated with actual.

Blocked: Corresponding testcases are failed.

Skip Passed

In Queue In Progress Failed Closed

Partial
Blocked Pass / Fail

Level-2 Regression Testing: Actually this Regression testing is part of Level-1 testing.
During comprehensive test execution, testing team reports mismatches to development
team as defects. After receiving that defect, development team performs modifications in
coding to resolve that accepted defects. When they release modified build, testing team
concentrate on regression testing before conducts remaining comprehensive testing.

Severity: Seriousness of the defect defined by the tester through Severity (Impact and
Criticality) importance to do regression testing. In organizations they will be giving three
types of severity like High, Medium and Low.

High: Without resolving this mismatch tester is not able to continue remaining testing.
(Show stopper).
Medium: Able to continue testing, but resolve must.
Low: May or may not resolve.

Ex: High: Database not connecting.

Medium: Input domain wrong. (Accepting wrong values also)
Low: Spelling mistake.

Xyz are three dependent modules. If u find bug in z, then

Do on z and colleges: High

Full z module: Medium
Partial z module: Low
 VIKRAM

Resolved Bug

Severity

High Medium Less

All P0 All P0 Some P0

All P1 Selected P1 Some P1
Selected P2 Some P2 Some P2

On modified Build to
ensure bug resolving

Possible ways to do Regression Testing:

Case 1: If development team resolved bug and its severity is high, testing team will re
execute all P0, P1 and carefully selected P2 test cases with respect to that modification.

Case 2: If development team resolved bug and its severity is medium, testing team will re
execute all P0, selected P1 [80-90 %] and some of P2 test cases with respect to that
modification.

Case 3: If development team resolved bug and its severity is low, testing team will re
execute some of the P0, P1, P2 test cases with respect to that modification.

Case 4: If development team performs modifications due to project requirement changes,

testing team reexecute all P0 and selected testcases.

Severity: With respect to functionality

Priority: With respect to customer.

Severity: All defects are not with same severity.

Priority: All defects are not with same priority.
 VIKRAM

Severity: Seriousness of the defect.

Priority: Importance of the defect.

Severity: Project functionality point of view important.

Priority: Customer point of view important.

Defect Reporting and Tracking:

During comprehensive test execution, test engineers are reporting mismatches to
development team as defect reports in IEEE format.

1. Defect Id: A unique number or name.

2. Defect Description: Summary of defect.
3. Build Version Id: Parent build version number.
4. Feature: Module / Functionality
5. Testcase name and Description: Failed testcase name with description
6. Reproducible: (Yes / No)
7. If yes, attach test procedure.
8. If No, attach snapshots and strong reasons
9. Severity: High / Medium / Low
10. Priority
11. Status: New / Reopen (after 3 times write new programs)
12. Reported by: Name of the test engineer
13. Reported on: Date of Submission
14. Suggested fix: optional
15. Assign to: Name of PM
16. Fixed by: PM or Team Lead
17. Resolved by: Name of the Developer
18. Resolved on: Date of solving
19. Resolution type:
20. Approved by: Signature of the PM

Defect Age: The time gap between resolved on and reported on.
Defect Submission:
 VIKRAM

QA

Test Manager Project Manager

Test Lead Team Lead

Test Engineer Developers

Transmittal Reports
Fig: Large Scale Organizations.

Defect Submission:

Project Manager

Test Lead Team Lead

Test Engineer Developers

Transmittal Reports

Fig: Small Scale Organizations.

 VIKRAM

Defect Status Cycle:

New

Fixed (Open, Reject, Deferred)

Closed

Reopen

Bug Life Cycle:

Detect Defect

Reproduce Defect

Report Defect

Fix Bug

Resolve Bug

Close Bug

Resolution Type:

There are 12 resolution types such as

1. Duplicate: Rejected due to defect like same as previous reported defect.
2. Enhancement: Rejected due to defect related to future requirement of the
customer.
3. H/w Limitation: Raised due to limitations of hardware (Rejected)
4. S/w Limitation: Rejected due to limitation of s/w technology.
 VIKRAM

5. Functions as design: Rejected due to coding is correct with respect to design

documents.
6. Not Applicable: Rejected due to lack of correctness in defect.
7. No plan to fix it: Postponed part timely (Not accepted and rejected)
8. Need for More Information: Developers want more information to fix. (Not
accepted and rejected)
9. Not Reproducible: Developer want more information due to the problem is not
reproducible. (Not accepted and rejected)
10. User misunderstanding: (Both argues you r thinking wrong) (Extra negotiation
between tester and developer)
11. Fixed: Opened a bug to resolve (Accepted)
12. Fixed Indirectly: Differed to resolve (Accepted)

Types of Bugs:

UI bugs: (Low severity)

Spelling mistake: High Priority
Wrong alignment: Low Priority

Input Domain bugs: (Medium severity)

Object not taking Expected values: High Priority
Object taking Unexpected values: Low Priority

Error Handling bugs: (Medium severity)

Error message is not coming: High Priority
Error message is coming but not understandable: Low Priority

Calculation bugs: (High severity)

Intermediate Results Failure: High Priority
Final outputs are Wrong: Low Priority

Service Levels bugs: (High severity)

Deadlock: High Priority
Improper order of Services: Low Priority

Load condition bugs: (High severity)

Memory leakage under load: High Priority
Doesn't allow customer expected load: Low Priority

Hardware bugs: (High severity)

Printer not connecting: High Priority
Invalid printout: Low Priority

Id control bugs: (Medium severity) Wrong version no, Logo

Version Control bugs: (Medium severity)
 VIKRAM

Source bugs: (Medium severity) Mismatch in help documents

Test Closure:
After completion of all possible testcase execution and their defect reporting and
tracking, test lead conduct test execution closure review along with test engineers.

In this review test lead depends on coverage analysis:

• BRS based coverage

• UseCases based coverage (Modules)
• Data Model based coverage (i/p and op)
• UI based coverage (Rules and Regulations)
• TRM based coverage (PM specified tests are covered or not)

Testing team try to execute the high priority test cases once again to confirm correctness
of master build.

Final Regression Process:

Gather requirements
Effort estimation (Person/Hr)
Plan Regression
Execute Regression
Report Regression

User Acceptance Testing:

After completion of test execution closure review and final regression, our organization
concentrates on UAT to collect feed back from customer / customer site like people.
There are two approaches:
1. Alpha testing
2. Beta testing
SignOff:
After completion of UA and then modifications, test lead creates Test Summary Report
(TSR). It is a part of s/w release note. This TSR consists of

1. Test Strategy / Methodology (what tests)

2. System Test Plan (schedule)
3. Traceability Matrix (mapping requirements and testcases)
4. Automated Test Scripts (TSL + GUI map entries)
5. Final Bug summary Report
Description Found By Status(Closed / Severity Module / Comme
Bug Id Deferred) Functionality Nts