An empirical based mutation

testing of concurrent object

oriented programs
BATCH NO 14PSE19 Under The Guidance Of

1.D.Gowtham - 14003386 Dr.T.Srinivas Prasad

2.K.Pushpanjali - 14003368
3.K.Yashwanth - 14003656
4.T.Poojitha - 14003343
Abstract
Mutation testing is a modern approach which gives more appropriate results. In
comparison to traditional approaches, it gives high quality output. Previously it is not
used mostly because of its high cost factor. This is because mutation testing deals
with white box testing. White Box testing checks every module of the software in
detail. If we use this it takes a lot of time and money. Recent approaches which
came in mutation testing made it easy to implement for any software. Mutation
take a look ating could be a fault based mostly testing technique within which
mutants area unit generated within the program and apply totally different test
cases on the mutants. Some mutants are killed and some are alive. On the bottom
of killed and alive mutants, mutant score is calculated. Based on the mutants which
are alive the test cases can be improvised there by the quality of the source code
is increased. we propose a tool which gives more effective output of testing. We
propose a tool which takes the outputs of various static tools available and
combines it with the outputs of dynamic tools available. Our proposed tool includes
outputs of available tools like Jester, Mujava, PMD to effectively detect the
vulnerabilities and produce high quality software as output.
Mutation Testing
Mutation Testing is a type of software testing where we mutate (change) certain
statements in the source code and check if the test cases are able to find the errors.
The changes in mutant program are kept extremely small, so it does not affect the
overall objective of the program.
The goal of Mutation Testing is to assess the quality of the test cases which should be
robust enough to fail mutant code. This method is also called as Fault based testing
strategy as it involves creating fault in the program
Mutation was originally proposed in 1971 but lost fervor due to high costs involved.
Now, again it has picked steam and is widely used for languages such as java and
XML.
Objective

To access the quality of the tests by performing them on mutated code.

To use this assessments to help construct more adequate tests.

To thereby product a suite of valid tests which can be used on real programs.
Mutation process
Step by step process to execute
mutation testing
Step 1: Faults are introduced into the source code of the program by creating many
versions called mutants. Each mutant should contain a single fault, and the goal is to
cause the mutant version to fail which demonstrates the effectiveness of the test
cases.
Step 2: Test cases are applied to the original program and also to the mutant program.
A test case should be adequate, and it is tweaked to detect faults in a program.
Step 3: Compare the results of original and mutant program.
Step 4: If the original program and mutant programs generate the same output, then
that the mutant is killed by the test case. Hence the test case is good enough to detect
the change between the original and the mutant program.
Step 5: If the original program and mutant program generate different output, Mutant is
kept alive. In such cases, more effective test cases need to be created that kill all
mutants.
Example
Original Program Mutant Program
Class Minimum{ Class Minimum{
public static void main(String args[]) public static void main(String args[])
{ {
int a; int a;
int b; int b;
if(a < b) if(a > b)
System.out.print(a is the minumum); System.out.print(a is the minumum);
else if(b < a) else if(b < a)
System.out.print(b is the minimum); System.out.print(b is the minimum);
else else
System.out.print(Both are equal); System.out.print(Both are equal);
} }
} }
Types of Mutation Testing
Mutation testing could be fundamentally categorized into 3 types

Statement Mutation - developer cut and pastes a part of code of which

the outcome may be removal of some line.

Value Mutation- values of primary parameters are modified.

Decision Mutation- control statements are to be changed.
Mutation Score
Adequacy of the test data is calculated using the mutation score.

No. of killed mutants

Mutation Score = ------------------------------------- * 100
No. of non-equivalent mutants

If the mutation score is 100% then the test data is said to be adequate.

Adequate test data is used to find the non-recoverable faults.

Advantages of Mutation Testing
It is a powerful approach to attain high coverage of the source program.
This testing is capable comprehensively testing the mutant program.
Mutation testing brings a good level of error detection to the software developer.
This method uncovers ambiguities in the source code, and has the capacity to
detect all the faults in the program.
Customers are benefited from this testing by getting most reliable and stable system.
Disadvantages of Mutant testing

Mutation testing is extremely costly and time consuming since there are many
mutant programs that need to be generated.
Since its time consuming, it's fair to say that this testing cannot be done without an
automation tool.
Each mutation will have the same number of test cases than that of the original
program. So, a large number of mutant programs may need to be tested against
the original test suite.
As this method involves source code changes, it is not at all applicable for black
box testing.
Mutation operators
 Literature Review

According to J.H.Andrews[3] mutants,when using carefully selected mutation operators

and after removing equivalent mutants, can provide a good indication of the fault
detection ability of a test suite. Furthermore, it shows the danger of using faults selected
by humans as, in the systems studied in this article, it leads to underestimating the fault
detection ability of test suites. Also it shows that the mutants we generate are not easier
to detect than real faults, it also suggest that they are not harder to detect than real
faults to any practical degree.
 According to Joanna Strug[4] a mutation testing-based method for generating negative
test cases that are able to support an assessment of a system ability to handle a wide range
of unexpected situations. Future work concerning application of mutation testing to test
cases should also include development of tools supporting the generation and execution of
mutants and experimental evaluation of the approach. Availability of such tools would
significantly increase the possibility of adapting the approach in practice.
 Acording to Pawan Kumar Chaurasia[2], Test cases are effective by generating the
relationship between test case and mutation score. On the base of test case mutant are
defined as weak and strong mutants. Weak mutation testing is to generate mutant
framework which is applied on the mutant operator and its technique. It is very expensive
testing, mutants are used for security policies to find the weak positions in security
features. Efficiency of the program can also be increased by calculating the mutant
operators. Mutants effectiveness can also be categorized into highly effective, effective
and low effective.
 According to Lei Mia &Kerong Bena[7], by consistency checking method we can check
the consistency level of a particular program.By defining a set of mutation operators based
on the mutation location, and examining each operator to avoid generating
unreasonable mutant, which can reduce the number of mutants. Then, the required
condition of generating test case for the mutant is analyzed. more complex experiments
are used to validate the usefulness of our method. On this basis, better application strategy
is studied to improve the efficiency of specification mutation testing.
Our work
We have used some tools for mutation testing
MUJAVA
JESTER
Java (muJava) is a mutation system for Java programs. It automatically
generates mutants for both traditional mutation testing and class-level mutation
testing. Java can test individual classes and packages of multiple classes. Tests
are supplied by the users as sequences of method calls to the classes under test
encapsulated in methods in separate classes JUNIT classes.
Jester is different than code coverage tools, because it can find code that is
executed by the running of tests but not actually tested. Jester's approach is
called mutation testing or automated error seeding. However, Jester is not
meant as a replacement for code coverage tools, merely as a complementary
approach.
Concurrent object oriented programming
Operator Category Concurrency Mutation Operators for Java
Modify Parameters of MXT Modify Method-X Time (wait(), sleep(), join(), and await() method calls)
Concurrency Methods MSP - Modify Synchronized Block Parameter
ESP - Exchange Synchronized Block Parameters
MSF - Modify Semaphore Fairness
MXC - Modify Permit Count in Semaphore and Modify Thread Count in Latches and Barriers
MBR - Modify Barrier Runnable Parameter
Modify the Occurrence of RTXC Remove Thread Method-X Call (wait(), join(), sleep(), yield(), notify(), notifyAll() Methods)
Concurrency Method Calls RCXC Remove Concurrency Mechanism Method-X Call (methods in Locks, Semaphores, Latches, Barriers, etc.)
RNA - Replace notifyAll() with notify()
RJS - Replace join() with sleep()
ELPA - Exchange Lock/Permit Acquisition
EAN - Exchange Atomic Call with Non-Atomic

Modify Keywords ASTK Add static Keyword to Method

RSTK Remove static Keyword from Method
RSK - Remove synchronized Keyword from Method
RSB - Remove synchronized Block
RVK - Remove volatile Keyword
RFU - Remove finally Around Unlock
Switch Concurrency RXO - Replace One Concurrency Mechanism-X with Another (Locks, Semaphores, etc.)
Objects EELO - Exchange Explicit Lock Objects
Modify Critical Region SHCR - Shift Critical Region
SKCR - Shrink Critical Region
EXCR Expand Critical Region
SPCR - Split Critical Region
Our proposed Static tool for OOPs
 In this we use certain static tools for java such as JTEST,PMD java to check the bugs
that are present in the particular project and produce the output in the form of
excel sheet .we follow the following approach
We will combine the output of all the tools using a merging algorithm and then we
export the output data to the database this results in efficiency of the particular
project.
Conclusion
Mutation testing is a form of white box testing which requires the change in the
source code. Mutation testing systematically evaluates the quality of existing
test cases by calculating the mutation score. However, mutation testing suffers
from equivalent mutants in which the testing has to be carried out manually, as
well as a high computational cost associated with a large pool of generated
mutants for the original program. Mutation testing can be applied to all the
design phases, coding. Apart from testing phase it can also be applied to other
phases of the project. Mutation testing can be implemented parallel to the
testing phase to achieve high quality test cases and quality software meeting all
the standards. Using mutation testing teams get higher performance related to
quality test cases.
 References
1. Durelli, V.H.S., Offutt, A.J., Delamaro, M.E.: Toward Harnessing High-level
Language Virtual Machines for Further Speeding up Weak Mutation Testing. IEEE
Fifth Internation- al Conference on Software Testing, Verification and Validation
(2012).
2. Boonyakulsrirung, P., Suwannasare, T.: A Weak Mutation Testing Framework for
WS- BPEL. Eighth International Joint Conf.on Computer Science and Soft.Engineering
(2011).
3. Y. Jia and M. Harman, An Analysis and Survey of the Development of Mutation
Testing, CREST Center, Kings College, London, Tech. Rep. TR-09-06,2009
4. M. Polo, M. Plattini and I Garcala-Rodriguez Decreasing the Cost of Mutation
Testing with Second Order Mutants Software Testing Verification and Reliability, vol.
19, no. 2, pp. 111-131, June 2008.
5. J. Andrews, L. Briand, and Y. Labiche, Is mutation an appropriate tool for testing
experiments? in Proc. Intl. Conference on Software Engineering (ICSE). ACM, 2005,
pp. 402411.
6. [2]Y. Jia and M. Harman, An analysis and survey of the development of mutation
testing, IEEE Transactions on Software Engineering (TSE), vol. 37, no. 5, pp. 649678,
2010
7. Philippe Chevalley and Pascale Thevenod-Fosse. A Mutation Analysis Tool for Java
Programs. Journal on Software Tools for Technology Transfer (STTT), September 2001.
8. Yu-Seung Ma, Yong-Rae Kwon and Jeff Offutt. Inter-Class Mutation Operators for
Java. In 13th International Symposium on Software Reliability Engineering (ISSRE 02)
pages 352363, Annapolis Maryland, November 2002.