Beruflich Dokumente
Kultur Dokumente
Performance tuning is not a silver bullet. Simply put, good system performance
depends on: good design, good implementation, defined performance objectives,
and performance tuning.
Since JBoss Performance tuning involves also tuning the environment on which
jBoss is run, the first tutorial will start discussing about JVM settings and OS
settings on which JBoss can produce best results. Then we'll see some specific JBoss
config settings.
The complete address space reserved for object memory can be divided into
the young and tenured generations.
The young generation consists of eden and two survivor spaces. Most objects are
initially allocated in eden. One survivor space is empty at any time, and serves as
the destination of any live objects in eden and the other survivor space during the
next copying collection. Objects are copied between survivor spaces in this way
until they are old enough to be tenured(copied to the tenured generation).
A third generation closely related to the tenured generation is the permanent
generation which holds data needed by the virtual machine to describe objects that
do not have an equivalence at the Java language level. For example objects
describing classes and methods are stored in the permanent generation
Use the the command line option -verbose:gc causes information about the heap
and garbage collection to be printed at each collection. For example, here is output
from a large server application:
It's demonstrated
that an application that spends 10% of its time in garbage collection can
lose 75% of its throughput when scaled out to 32 processors
(http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html)
JBoss tuning tip 2: Set -Xms and -Xmx to the same value
By default, the virtual machine grows or shrinks the heap at each collection to try to
keep the proportion of free space to live objects at each collection within a specific
range.
Setting -Xms and -Xmx to the same value. This increase predictability by removing
the most important sizing decision from the virtual machine.
is if you have an 8 processor box then you'll garbage collect your data with 8
threads. In order to turn on the parallel collector use the flag -XX:+UseParallelGC.
You can also specify how many threads you want to dedicate to garbage collection
using the flag -XX:ParallelGCThreads=8.
JBoss tuning tip 8: Don't choose an heap larger then 70% of your
OS memory
Choose a maximum heap size not more then 70% of the memory to avoid excessive
page faults and thrashing.
JBoss tuning tip 9: Tune the Heap ratio
This is one of most important tuning factor: the heap ratio. The heap ratio
specifies how the amount of the total heap will be partitioned between the
young and the tenured space. What happens if you have lots of long lived data
(cached data, collections ) ? maybe you're in this situation:
The problem here is that the long lived data overflows the tenured generation.
When a collection is needed the tenured generation is basically full of live data.
Much of the young generation is also filled with long lived data. The result was thata
minor collection could not be done successfully (there wasn't enough room in
the tenured generation for the anticipated promotions out of the young generation)
so a major collection was done.
The major collection worked fine, but the results again was that the tenured
generation was full of long lived data and there was long lived data in the young
generation. There was also free space in the young generation for more allocations,
but the next collection was again destined to be a major collection.
also space to support minor collections. This particular application used lots of short
lived objects so after the fix mostly minor collections were done.
NewRatio is a flag that specifies the amount of the total heap that will be
partitioned into the young generation. It's the tenured-generation-size / younggeneration-size. For example, setting -XX:NewRatio=3 means that the ratio between
the young and tenured generation is 1:3
If you want a more precise control over the young generation : NewSize is the
initial size of the young generation,MaxNewSize will specify the maximum size of
the young generation
What is the recommeded heap ratios ? Set the tenured generation to be
approximately two times the size of the young generation. With a 2GB of
RAM the recommended sizes are 1200MB for the heap and 400MB for the
young generation.
This recommendation is only a starting point, you have to tune from there and to do
that you have to gather and analyze the garbage collection statistics.
JBoss tuning tip 10: Monitor the free memory with monitors and snapshots
See this tips:
How to monitor jboss graphically ?
How to monitor JBoss with snapshots?
Each operating system sets default tuning parameters differently. For Windows
platforms, the default settings are usually sufficient. However, the UNIX and Linux
operating systems usually need to be tuned appropriately
Solaris tuning parameters:
Check the following TCP parameters with your sysadmin:
/dev/tcp tcp_time_wait_interval
/dev/tcp tcp_conn_req_max_q
/dev/tcp tcp_conn_req_max_q0
/dev/tcp tcp_ip_abort_interval
/dev/tcp tcp_keepalive_interval
/dev/tcp tcp_rexmit_interval_initial
/dev/tcp tcp_rexmit_interval_max
/dev/tcp tcp_rexmit_interval_min
/dev/tcp tcp_smallest_anon_port
/dev/tcp tcp_xmit_hiwat
/dev/tcp tcp_recv_hiwat
/dev/ce instance
/dev/ce rx_intr_time
Tip: Use the netstat -s -P tcp command to view all available TCP parameters.
Set TCP-related tuning parameters using the ndd command
Example: ndd -set /dev/tcp tcp_conn_req_max_q 16384
Tune /etc/system filesystem
Each socket connection to JBoss consumes a file descriptor. To optimize socket
performance, you may need to configure your operating system to have the
appropriate number of file descriptors.
See solaris documentation about this parameters:
set rlim_fd_cur
set rlim_fd_max
set tcp:tcp_conn_hash_size (Solaris 8 and 9)
set ip:ipcl_conn_hash_size (Solaris 10)
set shmsys:shminfo_shmmax Note: This should only be set for machines that have
at least 4 GB RAM or higher.
set autoup
set tune_t_fsflushr
Linux tuning parameters:
cat /proc/sys/fs/file-max
set fs.file-max=102642 into /etc/sysctl.conf
Raise ulimit with /etc/limits.conf (or ulimit -n for current session)
Increase default socket send/receive buffer
sysctl -w net.core.rmem_default=262144
(default socket receive buffer)
sysctl -w net.core.wmem_default=262144
(default socket send buffer)
sysctl -w net.core.rmem_max=262144
(max socket receive buffer)
sysctl -w net.core.wmem_max=262144
(max socket send buffer size)
Optimize MTU. The TCP maximum transfer unit is 1512 on the Internet. If you are
sending larger packets it's a good idea to increase MTU size in order to reduce
packet fragmentation (especially if you have a slow network)
vi /etc/sysconfig/network-scripts/ifcfg-xxx (eth0 for instance)
add "MTU=9000" (for gigabit ethernet)
restart the interface (ifdown eth0;ifup eth0)
Use Big Memory Pages
Default page size is 4KB (usually too small!)
Check page size with:
$ cat /proc/meminfo
If you see "HugePage_Total," "HugePages_Free" and "Hugepagesize", you can apply
this optimization
Here's how to do it (2GB Heap Size Example)
JBoss tuning tip 12: Lots of Requests ? check JBoss thread pool
JBoss thread pool is defined into conf/jboss-service.xml
1
2
3
4
5
6
7
8
9
<mbean code="org.jboss.util.threadpool.BasicThreadPool"
name="jboss.system:service=ThreadPool">
<attribute name="Name">JBoss System Threads</attribute>
<attribute name="ThreadGroupName">System Threads</attribute>
<attribute name="KeepAliveTime">60000</attribute>
<attribute name="MaximumPoolSize">10</attribute>
<attribute name="MaximumQueueSize">1000</attribute>
<attribute name="BlockingMode">run</attribute>
</mbean>
For most applications this defaults will just work well, however if you are running an
application with issues lots of requests to jboss (such as EJB invocations) then
monitor your thread pool. Open the Web Console and look for the
MBeanjboss.system:service=ThreadPool.
Start a monitor on the QueueSize parameter. Have you got a QueueSize which
reaches MaximumPoolSize ? then probably you need to set a higher
MaximumPoolSize pool size attribute
Watchout! Speak at first with your sysadmin and ensure that the CPU capacity
support the increase in threads.
Watchout! if your threads make use of JDBC connections you'll probably need to
increase also the JDBC connection pool accordingly. Also verify that your HTTP
connector is enabled to handle that amount of requests
1
2
3
4
5
The underlying HTTP connector of JBoss needs to be fine tuned for production
settings. The important parameters are:
maxThreads - This indicates the maximum number of threads to be allocated for
handling client HTTP requests. This figure corresponds to the concurrent users that
are going to access the application. Depending on the machine configuration, there
is a physical limit beyond which you will need to do clustering.
acceptCount - This is the number of request threads that are put in request queue
when all available threads are used. When this exceeds, client machines get a
request timeout response.
compression - If you set this attribute to force, the content will be compressed
by JBoss and will be send to browser. Browser will extract it and display the page on
screen. Enabling compression can substantially reduce bandwidth requirements of
your application.
So how do you know if it's necessary to raise your maxThreads
number ? again open the web console and look for the MBean
jboss.web:name=http-127.0.0.1-8080,type=ThreadPool. The key attribute
is currentThreadsBusy. If it's about 70-80% of the the maxThreads you should
consider raising the number of maxThreads.
Watch out! if you increase the maxThreads count you need to raise your JBoss
Thread pool accordingly.
1
2
3
4
5
6
7
8
<init-param>
<param-name>development</param-name>
<param-value>false</param-value>
</init-param>
<init-param>
<param-name>checkInterval</param-name>
<param-value>300</param-value>
</init-param>
References:
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html
http://people.redhat.com/alikins/system_tuning.html
http://community.jboss.org/wiki/JBossASTuningSliming
Please rate
Vote 5
Rate
Details
This Performance tuning tutorial is updated to the release 4.x of the application
server. If you want to learn all about JBoss 4.x-5.x-6.x Performance tuning, Optimal
data Persistence, Clustering tuning, Web application tuning and much more you
should not miss the JBoss Performance Tuning book.
1
2
<mbean code="org.jboss.invocation.pooled.server.PooledInvoker"
name="jboss:service=invoker,type=pooled">
<attribute name="NumAcceptThreads">1</attribute>
<attribute name="MaxPoolSize">300</attribute>
<attribute name="ClientMaxPoolSize">300</attribute>
<attribute name="SocketTimeout">60000</attribute>
<attribute name="ServerBindAddress">${jboss.bind.address}</attribute>
<attribute name="ServerBindPort">4445</attribute>
<attribute name="ClientConnectAddress">${jboss.bind.address}</attribute>
10
<attribute name="ClientConnectPort">0</attribute>
11
<attribute name="ClientRetryCount">1</attribute>
12
13
<attribute name="EnableTcpNoDelay">false</attribute>
</mbean>
There are two key attributes for the PooledInvoker in regards to how many threads
are used in processing requests. The first is the NumAcceptThreads attribute.
The value for this attribute will determine how many threads are created to listen
for incoming requests. These threads will be the ones that call the accept() method
of the server socket (which is a blocking call and will wait there till data is received
on the network interface for the server socket).
The MaxPoolSize is the other key factor: it's the size of the pool containing the
ServerThreads .
How can MaxPoolSize become a bottleneck ? if the accept thread can not get a
worker thread from the pool and the pool size has reached the MaxPoolSize value, it
will wait for one to become available (instead of creating a new one).
Tip 16: Have you got readonly Entity Beans ? tell it to JBoss
JBoss offers a way to handle this situation by defining either an entire EJB as being
"read-only" or simply as a subset of its methods. When accessing a read-only
method (or EJB), while JBoss still prevents concurrent access to the same bean
instance, the bean will not be enrolled in the transaction and will not be locked
during the whole transaction lifetime. Consequently, other transactions can directly
use it for their own work.
1
2
<enterprise-beans>
<entity>
<ejb-name>MyEntity</ejb-name>
<method-attributes>
<method>
<method-name>get*</method-name>
<read-only>true</read-only>
</method>
<method-attributes>
10
11
</entity>
</enterprise-beans>
Tip 18: Configure the EJB container to use cache, when possible.
If the EJB container has exclusive access to the persistent store, it doesnt need to
synchronize the in-memory bean state from the persistent store at the beginning of
each transaction.
So you could activate the so-called Commit-A option that caches entity bean state
between transactions. In order to activate this option :
1
2
<jboss>
<enterprise-beans>
3
4
<container-configurations>
<container-configuration extends=
6
7
8
9
10
11
12
<entity>
<ejb-name>MyEntity</ejb-name>
13
<configuration-name
14
15
<method-attributes>
16
<method>
17
<method-name>get*</method-name>
18
19
20
21
22
<read-only>true</read-only>
</method>
<method-attributes>
</entity>
</jboss>
23
Commit option A can boost your Entity Bean but what happens when running in a
cluster ? in a cluster configuration more than one JBoss node will access the same
database. Furthermore, they will not only read data, but may also update the db
store.Consequently, we now have as many points of write access to the database as
we have JBoss instances in the cluster.
For these scenarios, JBoss incorporates a handy tool: the cache invalidation
framework. It provides automatic invalidation of cache entries in a single node or
across a cluster of JBoss instances. As soon as an entity bean is modified on a node,
an invalidation message is automatically sent to all related containers in the cluster
and the related entry is removed from the cache. The next time the data is required
by a node, it will not be found in cache, and will be reloaded from the database.
In order to activate it, add to your Entity Bean the cache-invalidation tag.
<entity>
<ejb-name>MyEntity</ejb-name>
<configuration-name>
5
6
7
</configuration-name>
<method-attributes>
<method>
<method-name>get*</method-name>
<read-only>true</read-only>
10
</method>
11
<method-attributes>
12
<cache-invalidation>True</cache-invalidation>
13
</entity>
<container-configuration>
<call-logging>false</call-logging>
<invoker-proxy-binding-name>entity-pooled-invoker</invoker-proxy-binding-name>
<sync-on-commit-only>true</sync-on-commit-only>
....
</container-configuration>
JBoss ships with lots of services, however you'll seldom need to use them all. The
service is usually deployed as *-service.xml under the deploy directory. Sometimes
it's also deployed as .sar/.rar archive.
In order to remove the service, remove the file in the "Server/deploy" column. If
needed remove also the relative libs stated under "Server/lib"
Servizio
Server/deploy
Server/lib
Mail service
mail-service.xml
mail-plugin.jar,
mail.jar,activation.jar
cache-invalidation-service.xml
client-deployer-service.xml
hibernate-deployer-service.xml
jboss-hibernate.jar, hibernate2.jar,
cglib-full-2.0.1.jar, odmg-3.0.jar
HSQL DB
hsqldb-ds.xml
hsqldb-plugin.jar, hsqldb.jar
jms folder
jbossmq.jar
http-invoker.sar
HTTP)
XA Datasources
jboss-xa-jdbc.rar
JMX Console
jmx-console.war
Web Console
management/web-console.war
JSR-77
management/console-mgr.sar
monitoring-service.xml
jboss-monitoring.jar
Schedule Manager
schedule-manager-service.xml
scheduler-plugin.jar, schedulerplugin-example.jar
scheduler-service.xml
If you are removing a core JBoss service like JMS or EAR Deployer then you need to
remove it also from the jboss-service.xml :
1
2
3
4
<mbean code="org.jboss.management.j2ee.LocalJBossServerDomain"
name="jboss.management.local:j2eeType=J2EEDomain,name=Manager">
<attribute name="MainDeployer">jboss.system:service=MainDeployer</attribute>
<attribute name="SARDeployer">jboss.system:service=ServiceDeployer</attribute>
<attribute name="EARDeployer">jboss.j2ee:service=EARDeployer</attribute>
<attribute name="EJBDeployer">jboss.ejb:service=EJBDeployer</attribute>
<attribute name="RARDeployer">jboss.jca:service=RARDeployer</attribute>
<attribute name="CMDeployer">jboss.jca:service=ConnectionFactoryDeployer</attribute>
<attribute name="WARDeployer">jboss.web:service=WebServer</attribute>
10
<attribute name="CARDeployer">jboss.j2ee:service=ClientDeployer</attribute>
11
<attribute name="MailService">jboss:service=Mail</attribute>
12
<attribute name="JMSService">jboss.mq:service=DestinationManager</attribute>
13
<attribute name="JNDIService">jboss:service=Naming</attribute>
14
15
16
17
<attribute name="JTAService">jboss:service=TransactionManager</attribute>
<attribute name="UserTransactionService">jboss:service=ClientUserTransaction</attribut
<attribute name="RMI_IIOPService">jboss:service=CorbaORB</attribute>
</mbean>
Comment the following line in log4j.xml in order to remove logs on the Console:
1
<p class="ol-foreground"><root>
2
3
4
-->
<appender-ref ref="FILE"></appender-ref>
</root>
5
6
</p>
Consider raising the log level to the highest level possible in production. Here only
error logs are written:
1
<p class="ol-foreground"><root>
<appender-ref ref="FILE"></appender-ref>
5
6
</root>
</p><p class="ol-foreground"></p>