5 things you didn't know about ...

Java
performance monitoring, Part 2
Java process monitoring with the JDK's built-in profilers

Skill Level: Introductory

Ted Neward (ted@tedneward.com)

Principal
Neward & Associates

13 Jul 2010

If it's news to you that the JDK ships with the full-featured profiler JConsole, you'll be
even more surprised to learn about the five stand-alone profiling utilities introduced in
this article. Find out how lightweight (and in some cases experimental) Java™
process monitoring and analysis tools can help you hone in on performance
bottlenecks like thread starvation, deadlocks, and object leaks.

About this series

So you think you know about Java programming? The fact is, most
developers scratch the surface of the Java platform, learning just
enough to get the job done. In this series, Ted Neward digs beneath
the core functionality of the Java platform to uncover little-known
facts that could help you solve even the stickiest programming
challenges.

Full-featured, built-in profilers like JConsole and VisualVM sometimes cost more
than they're worth in performance overhead — particularly in systems running on
production hardware. So, in this second article focusing on Java performance
monitoring, I'll introduce five command-line profiling tools that enable developers to
focus on just one aspect of a running Java process.

The JDK includes many command-line utilities that can be used to monitor and
manage Java application performance. Although most of them are labeled
"experimental" and therefore technically unsupported, they're still useful. Some

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 1 of 8
developerWorks® ibm.com/developerWorks

might even be seed material for special-purpose tools that could be built using
JVMTI or JDI (see Resources).

1. jps (sun.tools.jps)
Many command-line tools require that you identify the Java process that you want to
monitor. This isn't so different from similar tools that monitor native-operating-system
processes and also require a process identifier to work.

The "VMID" identifier is not always the same as the native operating system process
identifier ("pid"), which is why we need the JDK's jps utility.

Using jps within a Java process

As with most of the tools shipped with the JDK, and all of the tools
mentioned in this article, the jps executable is actually a thin
wrapper around the Java class or set of classes that carry out the
vast majority of the work. Under Windows®, the tools are .exe files
that use the JNI Invocation API to call directly to the class in
question; under UNIX®, most of the tools are symbolic links to a
shell script that kicks off a generic Java launcher with the right class
name specified.

If you want to use jps's (or any other tool's) functionality from within
a Java process — such as an Ant script — it's relatively easy to just
call main() on the class that is the "main" class for each tool. For
easy reference, that class name appears in parentheses after each
tool's name.

jps — whose name reflects the ps utility found on most UNIX systems — tells us
the JVMID of a running Java application. As its name implies, jps returns the
VMIDs for all discoverable Java processes running on a given machine. If jps
doesn't discover a process, it doesn't mean that the Java process can't be attached
or spelunked. It just means that it isn't advertising itself as available.

If the Java process can be found, jps will list the command-line used to launch it.
This way of differentiating between Java processes is important because, as far as
the operating system is concerned, all Java programs are "java." For most
purposes, the VMID is the important number to note.

Getting started with profilers

The easiest way to get started with profiling utilities is to use a demo program like
the SwingSet2 demo found at demo/jfc/SwingSet2. Doing this avoids potential
hangups with processes running as background/daemon processes. Once you're
familiar with the tool and its overhead, you can try it on your actual programs.

After you load up your demo app, run jps and note the returned vmid. For best

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 2 of 8
ibm.com/developerWorks developerWorks®

effect, launch the Java program with the -Dcom.sun.management.jmxremote

property set. If you don't use this setting, some data gathered by some of the tools
below will be unavailable.

2. jstat (sun.tools.jstat)
The jstat utility can be used to gather a variety of different statistics. jstat
statistics are sorted into "options" that are specified at the command-line as the first
parameter. As of JDK 1.6, you can view the list of options available by running
jstat with the command -options. Some options are shown in Listing 1:

Listing 1. jstat options

-class
-compiler
-gc
-gccapacity
-gccause
-gcnew
-gcnewcapacity
-gcold
-gcoldcapacity
-gcpermcapacity
-gcutil
-printcompilation

The utility's JDK documentation (see Resources) will tell you what each of the
options in Listing 1 returns, but the majority of them are used to gather performance
information about the garbage collector, or just parts of it. The -class option
reveals loaded and unloaded classes (making it a great utility for detecting
ClassLoader leaks within the app server or your code), and both -compiler and
-printcompilation display information about the Hotspot JIT compiler.

By default, jstat will display the information at the moment you check it. If you
want snapshots taken at regular intervals, specify the intervals in milliseconds
following the -options command. jstat will continuously display snapshots of the
monitored process's information. If you want jstat to take a specific number of
snapshots before terminating, specify that number after the interval/period value.

If 5756 were the VMID for a running SwingSet2 process started a few minutes ago,
then the following command would tell jstat to produce a gc snapshot dump every
250 milliseconds for 10 iterations, then quit:

jstat -gc 5756 250 10

Note that Sun (now Oracle) reserved the right to change the output of the various
options, or even the options themselves, without warning. That's the downside of

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 3 of 8
developerWorks® ibm.com/developerWorks

using unsupported utilities. See the Javadocs for complete details about each of the
columns in the jstat output.

3. jstack (sun.tools.jstack)
Knowing what's happening inside of a Java process vis-a-vis the executing threads
is a common diagnostic challenge. For example, when an application has suddenly
stopped processing, it's obvious that some kind of starvation has been reached, but
it isn't obvious just by looking at the code where the starvation has occurred, or why.

jstack is a utility that returns a complete dump of the various threads running in an
app, which you can then use to pinpoint the problem.

Running jstack with the VMID of the desired process will generate a stack dump.
In this regard, jstack works the same as pressing Ctrl-Break within the console
window in which a Java program is running, or calling
Thread.getAllStackTraces() or Thread.dumpStack() on each of the
Thread objects in the VM. A jstack call also dumps information about non-Java
threads running within the VM, which aren't always available as Thread objects.

jstack's -l parameter offers up a slightly longer dump that includes more detailed
information about the locks held by each of the Java threads, and can thus be
invaluable in finding (and squashing!) deadlock or scalability bugs.

4. jmap (sun.tools.jmap)
Sometimes, the problem you're dealing with is an object leak, such as an
ArrayList (which could hold thousands of objects) that just isn't getting released
when it should. Another problem that is more general would be an expanding heap
that never seems to shrink, despite active garbage collection.

When you're trying to locate an object leak, it's incredibly helpful to take a picture of
the heap at a given moment in time, and then look through what's there. jmap
provides the first part of that functionality by taking a snapshot picture of the heap.
You can then analyze the heap data using the jhat utility described in the next
section.

Using jmap is straightforward, like all the other utilities described here. Simply point
jmap at the VMID of the Java process you want to snapshot, and give it some
parameters to describe the resulting file produced. The options you'll pass to jmap
consist of the name of the file to dump to and whether to use a text file or binary file.
A binary file is the most useful option, but only when combined with some kind of
indexing tool — manually slogging through several megabytes of text filled with
hexadecimal values is not the best way to spend your day.

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 4 of 8
ibm.com/developerWorks developerWorks®

For a more casual glance at the Java heap, jmap also supports a -histo option.
-histo produces a text histogram of the objects currently strongly referenced in the
heap, sorted by the total number of bytes consumed by that particular type. It also
gives the total number of instances of that particular type, which allows for some
primitive calculations and guesses about relative cost per instance.

Unfortunately, jmap doesn't have a period-and-max-count option like jstat does,

but it's relatively easy to put calls to jmap (or to jmap.main()) into a loop in either
a shell script or other class to take snapshots periodically. (In fact, this would be a
good extension to add to jmap, either as a source patch to the OpenJDK itself or as
an extension to another utility.)

5. jhat (com.sun.tools.hat.Main)
Once you've dumped the heap into a binary file, you can use jhat to analyze the
binary heap dump file. jhat creates an HTTP/HTML server that can be surfed in a
browser, giving an object-by-object view of the heap, frozen in time. While it could be
amusing to walk through the heap, object reference by object reference, you're
probably better served by doing some kind of automated analysis of the whole mess.
Fortunately, jhat supports an OQL syntax for doing that analysis.

For example, running an OQL query for all Strings with more than 100 characters
would look like this:

select s from java.lang.String s where s.count >= 100

The results are displayed as links to the objects, which then display the complete
contents of that object, the field references to other objects as additional links that
can be dereferenced. OQL queries can also invoke methods on the objects
themselves, use regular expressions as part of the query, and use built-in query
tools. One query tool, the referrers() function, displays all the referrers that refer
to an object of a given type. Here's the query to find all the referrers to File objects:

select referrers(f) from java.io.File f

You can find the full syntax of OQL and its features under the "OQL Help" page
inside the jhat browser environment. Combining jhat with OQL is a powerful way
to do a targeted investigation of a misbehaving heap.

In conclusion
The JDK's profiling extensions can be very useful when you need to get a closer

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 5 of 8
developerWorks® ibm.com/developerWorks

look at what's going on inside a Java process. All of the tools introduced in this
article can be used by themselves from the command line. They also can be
powerfully combined with JConsole or VisualVM. Whereas JConsole and VisualVM
provide an overarching view of the Java virtual machine, specifically focused tools
like jstat and jmap let you fine-tune your investigation.

Coming up in the 5 things series: scripting.

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 6 of 8
ibm.com/developerWorks developerWorks®

Resources
Learn
• 5 things you didn't know about ... (Ted Neward, developerWorks, 2010): Find
out how much you don't know about the Java platform, in this series dedicated
to turning Java technology trivia into useful programming tips.
• JDK Tools and Utilities: Learn about the experimental monitoring and
troubleshooting tools discussed in this article: jps, jstat, jstack, jmap, and
jhat.
• "Top 10 Java Performance Troubleshooting Tools" (Sajan Kumar, Javalobby,
July 2008): JConsole is at the top of this list, followed by a number of tools that
could be new to readers of this article.
• "Acquiring JVM Runtime Information" (Dustin Marx, Dustin's Software
Development Cogitations and Speculations, June 2009): More ways to combine
the JDK's built-in monitoring and management tools such as jps, jinfo and
JConsole.
• "Java theory and practice: Urban performance legends" (Brian Goetz,
developerWorks, April 2003): Brian investigates three well-known "facts" about
Java performance.
• "Java theory and practice: Plugging memory leaks with weak references" (Brian
Goetz, developerWorks, November 2005): How to detect and resolve a
common cause of unintentional object retention.
• JVMTI (JVM Tool Interface): Learn more about the the Java platform-native
programming interface that supports profiling, debugging, monitoring, thread
analysis, and coverage analysis tools. JDI (Java Debug Interface) provides
information for debuggers that need to remotely access the running state of VM.
• The developerWorks Java technology zone: Hundreds of articles about every
aspect of Java programming.
Discuss
• Get involved in the My developerWorks community.

About the author

Ted Neward
Ted Neward is the principal of Neward & Associates, where he
consults, mentors, teaches, and presents on Java, .NET, XML
Services, and other platforms. He resides near Seattle, Washington.

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 7 of 8
developerWorks® ibm.com/developerWorks

Trademarks
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the
United States, other countries, or both.

5 things you didn't know about ... Java performance monitoring, Part 2 Trademarks
© Copyright IBM Corporation 2010. All rights reserved. Page 8 of 8