What is Risk Based Testing?

- By Susheel Vellanki, Sr QA Engineer, Symphony Corporation

Before understanding risk based testing, we must understand the importance of risk based testing by answering following
questions.
1) Can we achieve 100% test coverage always? Do we always have enough time to do full regression testing? How much of
regression testing has been automated or can be automated?
2) How much % of compatibility testing (browsers, databases, OSs, application servers, networks, hardware platforms) is typically
covered during regression testing?
3) How can we assure that the most frequently used modules/components are working fine? Do you use tools to analyze code
coverage from manual or automated regression testing?
4)What is the $ value of the time spent in trying to cover 100% regression and compatibility testing vs. risks in releasing the
software to customers early due to internal and external factors in the project?
Let’s explore the evolution of risk based testing and what makes it different from other types of testing by answering the above
questions. Ideally, no project, and especially no software project, is executed without adverse external influences. We plan to have
100% test coverage on the software products that need to be delivered to the end users. But in a practical world this is seldom
possible due to potential problems like delays in requirements and development, non-availability of required test environments on
time, budgetary constraints etc. “Why do we need so much time for testing”, “Why do we need to do so many types of testing”,
“Reduce the testing time”, “I will take the risk, let us release the software” - are the first reactions by business to IT teams and
reduction in testing time is the most common solution adopted by most companies during delays in early phases of software
development lifecycles. However, IT teams especially testing teams must learn how to operate and still delivery high quality
software under such circumstances. Testing teams can start by prioritizing the tests that need to be performed and this is
determined by a set of key people working on the project like the Test Manager, Test Lead with the help of Business Analyst.
Though it is a problem based testing technique, it is applied in the case of looming deadlines and limited resources. When
combined with the traditional ways to test the application, it limits the scope of the testing efforts. This technique is used to
prioritize the development and execution of tests based on the two factors, ‘impact and likelihood of failure’ of the functionality
or the aspect being tested. How and Why Risk Based Testing Formulating a risk based testing approach needs an understanding of
the risks involved before you start testing the product. There are going to be some residual risks that cannot be eliminated. Risk-
Based testing helps to find the right quality (No high severity defects and very less low severity defects) level that can be delivered
within limited schedule and resources. This is done by identifying business and technical requirements for an application and
prioritizing these requirements on the basis of factors ‘likelihood and impact’ of failure. Risk of a requirement can be assessed
based on:
 A) Which requirements and attributes are critical for the success of the application/product and the business value of the same
based on the time to market. Business Analyst or Project Manager in consultation with the customer can come out with this
information.
B) How often is a feature/requirement used?
C) Is there is work around for the user to be able to use the application effectively for this requirement/feature?
D) How visible is a problem in a feature or attribute (for customers, users and people outside)?
E) Can we deliver without this requirement/feature?
Each requirement may result in many test scenarios and hundreds of test cases. Testing team with their deep understanding of
business knowledge and end-user perspective must prioritize the test scenarios and test cases, based on risk analysis performed
on the requirements and identify the tests that need to be executed. Testing teams should analyze the requirement and test
scenarios and score them on following factors based on the business knowledge the team has acquired working on the same
project or business area: -Consequence of failure (CF) (For e.g., business impact of failure)
 -Probability of failure (PF) (For e.g., size and complexity of a requirement, use of open source components)
-Vulnerability of failure (VF) (for e.g., features/requirements with lot of defects from past test execution cycles)
Scores the requirement on following factors: Functionality, Reliability, Usability, Efficiency, Maintainability and Portability.
 
Risk based testing is only effective in teams with good business knowledge and end-user perspective. This will determine the risk
value of the requirement/test scenario. Even when testing teams do not have sufficient time to write detailed test cases, test
scenarios must be documented in detail with key validations to be performed during execution and prioritized. Formalizing this
process and incorporating this strategy in your projects will help the testing team deliver the right business deliverables on time
and with adequate test coverage. This technique makes it easy for the organizations to deliver the right level of quality in the
context of changing business requirements. It provides a critical level of objectivity in determining what things to test by using a
combination of business and technical requirements to prioritize the testing process. The challenging part though is how to
measure success of risk based testing strategy on a project. There are certain ways in which it can be achieved – Return on
Investment (ROI), Delivered defect density, Code coverage analysis and Testing effectiveness.
Why is it Important?
The risk based approach supports the strategy that can expose all the risks and eliminate them. Additionally, it also aims to reduce
the impact of residual risks that were not anticipated or eliminated. A complete understanding of the system and its operational
environment helps to understand the potential risks thereby establishing a well-defined scope of risk identification and assessment
thus enabling the testing team to prioritize the test cases to be executed in the order of the high risk to low risk requirements and
high risk test cases to low risk test cases for each of the requirements.
To Conclude, Risk Based Testing is an approach that requires skill and experience to isolate the most important tests on the basis of
technical and business risks and constraints. Risk based testing is a powerful testing technique that enables the testing teams to
streamline their testing efforts. This knowledge helps in mitigating the risk and minimizing the testing efforts, thus, bringing an
objectivity to test designing and test management activities. It enables testing teams to make informed decisions while defining exit
criteria. It is debatable if the Risk Based Testing is an art or science. It is science because, testing teams follow a technique/structured
process to arrive at the test cases to be executed to ensure acceptable quality of the product or application within the constrained
time lines. It is an art because deriving the test cases to be executed to obtain acceptable quality depends on the knowledge, skill,
experience and judgment of the person prioritizing the tests. For me risk based testing is 50% science and 50% art. What do you think?
Please share your thoughts and experiences on risk based testing and whether it is an art or science or combination of both.