Beruflich Dokumente
Kultur Dokumente
7.
8.
9.
10.
Line coverage
Path coverage
Data coverage
Types of coverage output
1. Call points
Configuring the coverage tool
Generating aggregate reports for separate test runs
1. Generating a single aggregate report for a grouping of test runs
2. Generating an aggregate report from a set of data files
Checking minimum coverage
Activating coverage in a Maven project
Turning coverage output off
Standalone mode
Code coverage consists of a set of software metrics that can tell you how much of the production code is
covered by a given test suite. It's purely quantitative, and does not say anything about the quality of either
the production code or the test code. That said, the examination of code coverage reports will
sometimes lead to the discovery of unreachable code which can be eliminated. But more importantly,
such reports can be used as a guide for the discovery of missing tests. This is not only useful when
creating tests for existing production code, but also when writing tests first, such as in the practice of
TDD (Test Driven Development).
JMockit Coverage provides three different and complementary code coverage metrics: line coverage,
path coverage, and data coverage. An example coverage report showing all metrics can be found here.
Line coverage
The line coverage metric tells us how much of the executable code in a source file has been exercised by tests. Each executable line of code can be
uncovered, covered, or partially covered. In the first case, none of the executable code in it was executed at all. In the second, all of the code was fully
executed at least once. In the third case, only part of the executable code in the line was executed. This can happen, for example, with lines of code containing
multiple logical conditions in a complex boolean expression. JMockit Coverage identifies all three cases, computing the coverage percentage for each
executable line of code accordingly: 0% for an uncovered line, 100% for a covered line, or some value in between for a partially covered line.
A branching point exists wherever the program makes a decision between two possible execution paths to follow. Any line of code containing a logical
condition will be divided in at least two executable segments, each belonging to a separate branch. An executable line of source code with no branching
points contains a single segment. Lines with one or more branching points contain two or more executable segments, separated by consecutive branching
points in the line.
Lets say that NS >= 1is the number of executable segments on a given line. If NEis the number of segments in that line which were executed at least once
during a test run (ie, they are covered segments), then we can calculate the coverage percentage for the line as 100 * NE / NS.
Similarly, the line coverage percentage for a whole source file is calculated from the total number of executable segments and the total number of covered
segments, considering all executable lines of code in the file. The percentage for a package, in turn, is calculated from the total and covered numbers of
segments in the whole set of source files belonging to the package. Finally, the total code coverage percentage is computed by the same formula on the totals
for all packages.
Path coverage
A completely different metric is path coverage, which is computed for method and constructor bodies, not for lines or segments of code. It tells us how many of
the possible execution paths through a method or constructor, from entry to exit, have been executed at least once during the test run.
Note that each method or constructor has a single point of entry, but can have multiple exits. An exit occurs when a returnor throwstatement is executed.
These are normal exits, of course. A method/constructor execution can also terminate abruptly, by propagating an exception (or error) thrown as a result of a
method call, an attempt to access a nullreference, or some other action which caused an unintended program failure.
Each possible path can be either fully executed (covered) or not (uncovered). Paths that execute only partially (ie, they were terminated abruptly) are simply
considered as uncovered.
The path coverage percentage for a method or constructor body is computed in a way similar to the line coverage computation. If NPis the number of possible
paths through the implementation body and NPEis the number of paths executed from entry to exit, then the metric is computed as 100 * NPE / NP. Also in
the same way as the line coverage metric, we extend this formula to the whole source file, the whole package, and the whole set of packages touched by the
test run.
Data coverage
jmockit.googlecode.com/svn/trunk/www/tutorial/CodeCoverage.html
1/5
9/26/13
Measures how many of the instance and static non-final fields were fully exercised by the test run. To be fully exercised, a field must have the last value
assigned to it read by at least one test. The percentages are calculated as 100 * NFE / NF, where NFis the number of non-final fields and NFEthe number
of fully exercised fields.
2/5
9/26/13
jarfiles. For example, "orderMngr\.domain\..+" selects all classes in the orderMngr.domainpackage as well as in any sub-packages.
The example above uses "myBaseDir" as the base directory where a separate JVM instance will run. Two output directories containing "coverage.ser"
data files are specified, as command line arguments. Other configuration parameters can be specified through the "jmockit-coverage-xyz" system
properties. This separate JVM instance will read each of the "coverage.ser" data files, merge the coverage data in memory, and then generate the
aggregate HTML report before exiting.
jmockit.googlecode.com/svn/trunk/www/tutorial/CodeCoverage.html
3/5
9/26/13
In Maven 2/3, the surefireplugin is the one usually responsible for actually running tests. To configure the coverage tool, specify values for the appropriate
"jmockit-coverage-xyz" system properties. For example, the output directory for generated files can be specified through the jmockit-coverageoutputDirproperty.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration><argLine>
-Djmockit-coverage-outputDir=target/my-coverage-report
<!-- other "jmockit-coverage" properties, if needed -->
</argLine></configuration>
</plugin>
If the tests are executed on a Java SE 5 JVM, the "-javaagent" JVM parameter should be used as shown below. When running on Java 6 or newer, this
parameter is not necessary.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration><argLine>
-javaagent:"${settings.localRepository}"/com/googlecode/jmockit/jmockit/${jmockit.version}/jmockit-${jmockit.version}.jar
<!-- "jmockit-coverage" properties, if any are needed -->
</argLine></configuration>
</plugin>
Finally, if the tests don't actually use the JMockit mocking APIs, it's still possible to use the coverage tool. In this case, the only dependency needed is the one
on "jmockit-coverage". Additionally, it's necessary to configure the surefireplugin as follows:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration><argLine>
-javaagent:"${settings.localRepository}"/com/googlecode/jmockit/jmockit-coverage/
${jmockit.coverage.version}/jmockit-coverage-${jmockit.coverage.version}.jar
<!-- "jmockit-coverage" properties, if any are needed -->
</argLine></configuration>
</plugin>
Standalone mode
jmockit.googlecode.com/svn/trunk/www/tutorial/CodeCoverage.html
4/5
9/26/13
In previous sections we described the most typical way to use the coverage tool, by enabling it during a JUnit/TestNG test run to measure test coverage. The
tool can also be used in a more general context, though: the standalone mode, where it can attach to any Java 5+ process to measure line and path coverage
metrics on specified classes, regardless of which code is making calls into said classes.
To activate standalone mode, the target JVM instance must be started with the "-javaagent:<proper/path/>jmockit-coverage.jar" command line
argument. That's it; none of the JMockit toolkit jars need to be present in the classpath of the target process. Initial configuration settings for the coverage tool
can be specified through the "jmockit-coverage-xyz" system properties previously described, but this is entirely optional; the configuration properties can
be modified later through a dedicated UI.
Once the target process is running with the JMockit Coverage Java agent, the user should connect to it with a JMX client which can access arbitrary
"MBeans". Usually, the standard JConsole tool available in a Java 5+ JDK will be used. The JMockit Coverage MBean provides several configuration
properties (the same ones which can be set with "-D" on the command line), and one operation through which the desired output can be generated. The user
interface provided by JConsole is shown below, where the process that is running with the coverage tool is a Tomcat 7 server instance.
The configuration properties (shown as "Attributes" of the "CoverageControl" MBean above) are as before, except in the case of "SrcDirs" (which
corresponds to jmockit-coverage-srcDirs). If this property is not specified, no attempt is made to find source files for the classes considered for
coverage.
jmockit.googlecode.com/svn/trunk/www/tutorial/CodeCoverage.html
5/5