Java Messaging Service

Agenda

JMS basics
JMS Objects Messaging Models Message Browsing Message Headers Message Persistence Delivery Modes Message Selectors Temporary Destinations MDB Reinitializing on server failure

JMS with Transactions

6/13/2011

WebLogic JMS

Agenda

Advanced JMS
FrontEnd Vs BackEnd Distributed Destinations Flow Control JMS Bridge JMS Migration Foreign JMS Server

Common JMS problems JMS OOM JMS lost Message JMS Message Redelivery JMS with JDBC store hung under load Pending Messages Clients unable to recover from a failed JMS Server Poison Messages

6/13/2011

WebLogic JMS

Agenda

Tuning JMS Performance

Configuration Tuning Application Design

JMS Debug

6/13/2011

WebLogic JMS

A common JMS application

Application A wishes to send messages to Application B

A is the Producer - Sender Publisher B is the Consumer Receiver Subscriber

6/13/2011

WebLogic JMS

JMS Basics

Messaging models:
Point to point (Queue) Publish/subscribe (Topic)

JMS Administered objects

Connection Factory Connection Session Destination Producer/Publisher Receiver/Subscriber

6/13/2011

WebLogic JMS

What is a Connection?

Definition: represents an open connection to the messaging system. Created by ConnectionFactory Creates Sessions Due to the authentication and communication setup done when a Connection is created, a Connection is a relatively heavyweight JMS object.

6/13/2011

WebLogic JMS

What is a Connection?

Must be started to consume messages. When a connection is created, it is in stopped mode. It is typical to leave the connection in stopped mode until setup is complete to minimize confusions. Note: Does not need to be started to produce messages! ClientID can be set on the connection in two ways: Connection.setClientID() Using JMSConnectionFactory
How can I force a ConnectionFactory to issue just one connection at a time?

6/13/2011

WebLogic JMS

ConnectionFactory

WebLogic automatically provides three connection factories: weblogic.jms.ConnectionFactorya non-XA capable factory. weblogic.jms.XAConnectionFactoryan XA-capable factory weblogic.jms.MessageDrivenBeanConnectionFactoryan XAcapable factory for message driven EJBs. MessageMaximum: Maximum number of messages that may exist for an asynchronous session and that have not yet been passed to the message listener Default is 10 Not applicable for Topics Acts like a sliding window

6/13/2011

WebLogic JMS

ConnectionFactory

Connection Factory Load-Balancing and Fail-Over

Works Just Like Stateless EJB Clustering at the ConnectionFactory.createConnection() Method Level

6/13/2011

WebLogic JMS

10

What is a Session?

Definition: object that manages the active exchange of messages between producers and consumers Creates producers, consumers, and messages. Must create separate Session if a client wishes to simultaneously produce and consume messages (singlethreaded) Adds value to Connection: Defines order of messages consumed and messages produced (done serially) Guarantees delivery by storing message until ACK by consumer queue

6/13/2011

WebLogic JMS

11

What is a Session?

Session types
Non-transacted
QueueSession session = queueConnection.createQueueSession (false, Session.AUTO_ACKNOWLEDGE);

Transacted
QueueSession session = queueConnection.createQueueSession (true, Session.AUTO_ACKNOWLEDGE);

recover() : Stops message delivery in this session, and restarts message delivery with the oldest unacknowledged message.

6/13/2011

WebLogic JMS

12

PTP Queues

Many producers can serialize messages to multiple receivers in a queue. Messages are delivered to a
single client.
Message 3 Producer Message
Receiver1

Distribution Queue
Receiver2

9 Producer Message

4 Message 1

Queue Manager (MOM Server)

Receiver3

For multiple receivers, first come, first serve

6/13/2011 WebLogic JMS

Message

13

Pub/Sub Topics

Publishing and subscribing to a topic decouples producers from consumers.

Subscriber1

1
Publisher

2
Message

Drama Movies Topic

Publisher Message

3 1

Subscriber2

Topic Manager (MOM Server)

6/13/2011

WebLogic JMS

3 2 1
Subscriber3

14

Message Browsing

Only for queue, not for topic (javax.jms.QueueBrowser)

QueueBrowser qbrowser = qsession.createBrowser(queue); Enumeration e = qbrowser.getEnumeration(); Message m = null; while (e.hasMoreElements()) { m = (Message) e.nextElement(); System.out.println("Message ID " + m.getJMSMessageID() +" delivered " + new Date(m.getJMSTimestamp())+" to " + m.getJMSDestination()); }

The application can view the messages in the queue, but the messages are not considered read and are not removed from the queue.

6/13/2011

WebLogic JMS

15

Message Browsing

Not possible to view the pending messages in 8.x. Requested by many customers and its possible in diablo. The JMS API does not require the content of an enumeration to be a static snapshot of queue content. Depends on the JMS provider. WLS produces a snapshot of the messages in the queue at the time the queue browser is created.

6/13/2011

WebLogic JMS

16

Message Headers

JMSCorrelationID JMSDeliveryMode JMSDeliveryTime JMSDestination JMSExpiration JMSMessageID JMSPriority JMSRedelivered JMSReplyTo JMSTimestamp JMSType

6/13/2011

WebLogic JMS

17

JMS Basics: QOS

Quality of service is a popular term with many nuances For this discussion, consider these terms:
Unreliable: The message might get from point A to point B, and it might not At-least-once: No message may be lost, but some messages may be duplicated At-most-once: It is better to lose a message than to deliver it twice Exactly-once: No message may be lost, and no message may be delivered more than once

6/13/2011

WebLogic JMS

18

JMS Basics: Persistence

A persistent message is guaranteed to be delivered at least onceit is not considered sent until it has been safely written in the file or database. WebLogic JMS writes persistent messages to a persistent backing store (file or JDBC database) assigned to each JMS server during configuration. Non-persistent messages are not stored. They are guaranteed to be delivered at least once unless there is a system failure, in which case messages may be lost. If a connection is closed or recovered, all non-persistent messages that have not yet been acknowledged will be redelivered. Once a non-persistent message is acknowledged, it will not be redelivered.

6/13/2011

WebLogic JMS

19

JMS Basics: Persistence

Question: Whats the minimum I need to configure in

weblogic to make a message persistent?

Answer:
Default Delivery Mode=Persistent on the connectionFactory Persistent Store configured on JMSServer

Question: How can I read the messages in the persistent

store?

6/13/2011

WebLogic JMS

20

Persistence: JMS Store

Question: What stops the JMS File stores to grow in size? Answer: A file store never shrinks. When messages get deleted from the file, the files keep the size they had at peak load, but it has free space for new messages. There is no file store compression pre-9.0. If the size is a problem and the customer doesnt want a server bounce, Customer can bounce JMSServer by untargeting the JMSServer, deleting the store, then retargeting JMS. Question: One single JDBC pool for JDBC JMS Store is a big bottleneck for my store performance. What should I do? Answer: Wait for diablo! It has multi pool support.
6/13/2011 WebLogic JMS 21

Persistence: Paging Store

JMS message paging saves memory for both persistent and non-persistent messages , as even persistent messages cache their data in memory. A paged-out message does not free all of the memory that it consumes. The message header and message properties remain in memory for use with searching, sorting, and filtering. Bytes/messages high and low thresholds can be configured at various levels in Admin console to start and stop paging.

6/13/2011

WebLogic JMS

22

Persistence: Paging Store

Question: I have paging enabled and I dont see my persistent messages being paged out to paging store? But I do see non persistent messages going to paging store. Answer: Persistent messages are never paged to the paging store since we already have a persistent copy of the message (in the file or JDBC store). Paged persistent messages continue to be written to the regular backing store (file or database) but the message bodies are removed from memory. Paged non-persistent messages are written to the JMS server's message paging store.

6/13/2011

WebLogic JMS

23

Exercise 1

Configure JMS FileStore and Paging Store and execute the following: Case1: Run a simple test that persist the messages in the store. Case2: Run a test to page persistent and non persistent messages using various threshold parameters.

6/13/2011

WebLogic JMS

24

JMS Basics: Delivery Modes

APIs to get messages

Synchronous (the receive call)
Message msg = qreceiver.receive();

Asynchronous (the MessageListener interface)

public class QueueReceive implements MessageListener{ public void onMessage(Message msg)(){ . } }

6/13/2011

WebLogic JMS

25

JMS Basics: Delivery Modes

Acknowledgement modes when non-transacted

AUTO_ACKNOWLEDGE DUPS_OK_ACKNOWLEDGE CLIENT_ACKNOWLEDGE NO_ACKNOWLEDGE MULTICAST_NO_ACKNOWLEDGE

Note: NO_ACKNOWLEDGE and MULTICAST_NO_ACKNOWLEDGE are just weblogic specific acknowledge modes.

6/13/2011

WebLogic JMS

26

Sending a JMS Message

public class TopicSend { public final static String JNDI_FACTORY="weblogic.jndi.WLInitialContextFactory"; public final static String JMS_FACTORY=javax.jms.TopicConnectionFactory"; public final static String TOPIC="jms.topic.exampleTopic"; public static void main(String[] args) throws Exception { Hashtable env = new Hashtable(); env.put(Context.INITIAL_CONTEXT_FACTORY, JNDI_FACTORY); env.put(Context.PROVIDER_URL, args[0]); InitialContext ctx = new InitialContext(env); Topic topic = (Topic) ctx.lookup(TOPIC); TopicConnectionFactory connectionFactory = (TopicConnectionFactory) ctx.lookup(JMS_FACTORY); TopicConnection connection = connectionFactory.createTopicConnection(); TopicSession session = connection.createTopicSession(false, Session.AUTO_ACKNOWLEDGE); TopicPublisher publisher = session.createPublisher(topic); TextMessage message = session.createTextMessage(); message.setText(hello world); publisher.publish(message); publisher.close(); session.close(); connection.close(); } }

Get a JNDI Initial Context Get a JMS Topic and ConnectionFactory from JNDI Create the connection, session and publisher Create & publish the message Close up shop

6/13/2011

WebLogic JMS

27

Receiving a JMS Message synchronously

public class TopicReceive { public final static String JNDI_FACTORY="weblogic.jndi.WLInitialContextFactory"; public final static String JMS_FACTORY=javax.jms.TopicConnectionFactory"; public final static String TOPIC="jms.topic.exampleTopic"; public static void main(String[] args) throws Exception { Hashtable env = new Hashtable(); env.put(Context.INITIAL_CONTEXT_FACTORY, JNDI_FACTORY); env.put(Context.PROVIDER_URL, args[0]); InitialContext ctx = new InitialContext(env); Topic topic = (Topic) ctx.lookup(topicName); TopicConnectionFactory connectionFactory = (TopicConnectionFactory) ctx.lookup(JMS_FACTORY); TopicConnection connection = connectionFactory.createTopicConnection(); TopicSession session = connection.createTopicSession(false, Session.AUTO_ACKNOWLEDGE); TopicSubscriber subscriber = session.createSubscriber(topic); connection.start(); Message m = subscriber.receive(); subscriber.close(); session.close(); connection.close(); } }

Get a JNDI Initial Context Get a JMS Topic and ConnectionFactory from JNDI Create & Start the connection Get a JMS message Close up shop

6/13/2011

WebLogic JMS

28

Receiving JMS messages asynchronously

Implement MessageListener

Set message listener to receive JMS messages

public class TopicReceive implements MessageListener { ... void setup(Context ctx, String topicName) { topic = (Topic) ctx.lookup(topicName); connectionFactory = (TopicConnectionFactory) ctx.lookup(JMS_FACTORY); connection = connectionFactory.createTopicConnection(); session = connection.createTopicSession(false, Session.AUTO_ACKNOWLEDGE); subscriber = session.createSubscriber(topic); subscriber.setMessageListener(this); connection.start(); }

MessageListener interface for receiving JMS messages asynchronously

public void onMessage(Message message) { int x = message.getIntProperty(x); int y = message.getIntProperty(y); . } }

6/13/2011

WebLogic JMS

29

Message Selectors

Message selectors can be used to filter unwanted messages, and subsequently improve performance by minimizing their impact on network traffic. The sending application sets message header or property fields to classify a message.
msg.setStringProperty(testMe", rais);

The receiving applications specify a simple query string to filter the messages that they want to receive.
session.createReceiver( testQueue,testMe in (rais));

A message selector is a boolean expression. eg:

salary > 64000 and dept in ('eng','qa')
6/13/2011 WebLogic JMS 30

Message Selectors

If really needed, use selectors with Topics, not with Queues. Topic selectors only incur a charge once per message per subscriber. If the message doesn't match a subscriber's selection criteria, it is simply ignored and not placed in that subscriber's "subscription". There is never any duplicate work. When using queues, if there are a lot of messages that don't match what you are looking for, you are going to compare your selector against those messages over and over again each time you go back to look for one of yours. Decision is made on the server side, so less network traffic.

6/13/2011

WebLogic JMS

31

Exercise 2

Creating consumers using Message Selectors: Case1: Create and run a simple queue receiver using a selector Case2: Create and run a simple topic subscriber using a selector Notice the difference in behavior

6/13/2011

WebLogic JMS

32

MDB

MDB is an EJB that acts as a JMS message listener MDB: Function as async stateless Service Consumes messages from Queue/topic Are managed by EJB container Only Required and NotSupported values supported for <trans-

attribute>(default is NotSupported)
maxConcurrentMDBs = Min(max-beans-free-pool, default-thread-pool-size/2+1). For each MDB, container create one connection to the JMS provider Multiple Sessions and consumers (for queue only) depending on the size of maxConcurrentMDBs.
WebLogic JMS 33

6/13/2011

MDB

Question : Which security principal does an MDB use to connect to JMS? Answer. As of WLS 6.1 SP2, an MDB uses the same principle to connect to JMS as it does to process messages. This is either the principle that is mapped to the run-as role specified for the bean, or 'guest' if no run-as role is provided. Prior to WLS 6.1 SP2, this behavior was not well defined.

6/13/2011

WebLogic JMS

34

Reinitializing on server failure: ExceptionListeners

If a Java client crashes then the JMS server will clean up

all the outstanding server-side resource from the crashed client JVM, such as: JMS connection(s) from the crashed client JVM JMS temporary destination(s) created under the above JMS connection(s) JMS session(s) created under the above JMS connection(s) JMS client(s) created under the above JMS session(s) (connection consumer and regular consumer) JMS browser(s) created under the above session(s) JMS producer(s) created under the above session(s)
Well, it all depends if the server ever received PeerGone. What if it never received peergone due to failed network?

6/13/2011

WebLogic JMS

35

Reinitializing on server failure: ExceptionListeners

If WebLogic Server crashes and it is the frontend to the JMS server, then:
A JMS client will lose all the server-side resources listed above. The client's javax.jms.ExceptionListener.onException(...) will be called (if javax.jms.JMSConnection.setExceptionListener is set) with a LostServerException, which extends JMSException.

6/13/2011

WebLogic JMS

36

Reinitializing on server failure: ExceptionListeners

6/13/2011

WebLogic JMS

37

Reinitializing on server failure: ExceptionListeners

Its best to set ExceptionListeners on both connection and session. If a JMS provider detects a problem with a connection/session, it will inform the connections ExceptionListener, if one has been registered.
public class QueueReceive implements MessageListener, ExceptionListener public void onException(JMSException exception) {..} public void init( .. qcon.setExceptionListener(this); ((weblogic.jms.extensions.WLSession)session).setExceptionListener(classna me); qcon.start(); } }

ExceptionListener is meant for only for asynchronous listeners.

6/13/2011 WebLogic JMS 38

JMS with Transactions

Transacted Session vs. User Transactions

Transacted sessions ignore JTA user transactions A transacted session therefore can not participate in an EJB or MDB container managed transaction, or a user transaction explicitly started by an application. A WebLogic MDB can be user transaction aware, but not transacted. In order to make JMS session infected with User Transaction, make sure ConnectionFactory being used is XA enabled or has users

transactions enabled configured to be true.

6/13/2011

WebLogic JMS

39

JMS with Transactions

Same JMS client may produce AND consume messages in the same Tx .However, you may NOT consume the same message you produced with the same Tx Three ways to demarcate the transactions with JMS Create transacted session Use the java TX API (JTA) user TX with JMS Session Use Message Driven Beans
Container managed Tx Bean managed TX

6/13/2011

WebLogic JMS

40

Exercise 4

Demonstrate the behavior of synchronous consumer with both transacted session as well as UserTransaction Case1: Both Tx session and UserTransaction are used.Tx Session rollsback before UserTransaction commits. Case2: Both Tx session and UserTransaction are used.Tx Session commits before UserTransaction rollsback.

6/13/2011

WebLogic JMS

41

MDB in Transactions

When running with NotSupported tx attribute, only AUTO_ACK and DUPS_OK_ACK modes are supported.

For MDB to run in a distributed Tx: Connection factory should be XA enabled Should use container managed transaction with Required Tx-attribute. If onMessage() of MDB throws a system exception (even if its running with NotSupported tx attribute), we discard that MDB instance (as per the ejb specs)

6/13/2011

WebLogic JMS

42

JMS as an XA Resource

Distributed Tx is a TX that spans multiple resource managers Coordination of multiple resources is done by a TM 2PC is used to coordinate the TX Make sure to use XA connection factory Sample scenarios
MDB sending message to other Queue in other JMS Server MDB making a DB call MDB calling a CMP in turn making a DB call Standalone java client pulling a message from one queue and pushing to other queue in a global Transaction.

JMS Server is an XA resource. For example - Two queues in the same JMS server constitute just one single resource
WebLogic JMS 43

6/13/2011

JMS as an XA Resource

Question :
Is it a one-phase tx if my WebLogic JMS JDBC store is on the same database for which I am doing other database work?

Answer:
No. WebLogic JMS is its own resource manager. That is JMS itself implements XAResource and handles the transactions without depending on the database (even when the messages are stored in the database). That means whenever you are using JMS and a database (even if it is the same database as the JMS messages are stored) then it is 2PC.

6/13/2011

WebLogic JMS

44

JMS as an XA Resource

Question :
In a distributed tx, MDB consumes persistent messages from a JDBC JMSStore and makes some DB updates through ejbs using a TXDatasource (using a non xa pool). JDBC JMS Store uses a non xa pool which is fine. Do I need to use xa-pool for the TxDatasource used by ejbs (since there can be at the most 1 non xa resource in dist tx) ??

Answer:
JMS *is* the XAResource - the store behind it must use a non-xa driver. The interactions with the store are not part of the distributed transaction as a separate resource. Only the JMSServer in front of it. If all of the EJBs are using the same TxDataSource, then I don't think underneath it has to be XA. At a minimum, you would have to enable two phase commit on the TxDatasource, and that would be the one non-XA resource in the transaction.
6/13/2011 WebLogic JMS 45

Advanced JMS Topics

FrontEnd Vs BackEnd Distributed Destination Flow Control Message Bridge JMS Migration Foreign JMS Server
6/13/2011 WebLogic JMS 46

FrontEnd Vs BackEnd

6/13/2011

WebLogic JMS

47

Distributed Destinations

Completely transparent to senders and receivers Members of the distributed destination can be located anywhere, but must all be served by JMS servers in a single server cluster Load Balancing for:
Producers Consumers Each Producer.send() call

Failover: When a member becomes unavailable due a server failure, (new) traffic is then redirected toward other available destination members in the set.
6/13/2011 WebLogic JMS 48

Distributed Queues

When a message is sent to a distributed queue, it is sent to exactly one of the physical queues in the set of members for the distributed queue Queue member (but which has no consumers ) can forward messages to other queue members by configuring the Forward Delay attribute (in seconds). when producers are sending messages, queues with zero consumers are not considered for message production, unless all instances of the given queue have zero consumers. QueueSenders each time a message is produced using the sender a decision is made as to which queue member will receive the message. QueueReceivers a single physical queue member is chosen for the receiver at creation time
6/13/2011 WebLogic JMS 49

Server Affinity

When the Server Affinity option is enabled for distributed destinations, then before a WebLogic Server instance attempts to load balance consumers or producers across all the members of a distributed destination in a domain, it will first attempt to load balance across any local members that are running on the same WebLogic Server instance.

6/13/2011

WebLogic JMS

50

Distributed Queue Routing

6/13/2011

WebLogic JMS

51

Distributed Queue Routing

6/13/2011

WebLogic JMS

52

Exercise 5:

Objective: To demonstarte that Consumers listening on a distributed queue are pinned to a particular member.
Start a receiver listening on DistQueue. Check on admin console, which member it got pinned to. Start the sender as:
Java QueueSend t3://localhost:port [queueName]

Send a message on either member1 or member2 directly. Messages would be received from just one member. It shows that the consumers are pinned to a particular member at startup.

6/13/2011

WebLogic JMS

53

Exercise 6

Objective: Demonstrate Server Affinity Attribute for ConnectionFactory

Start consumer1 on Dist Queue Start consumer2 on Dist queue Check the admin console if both the members of dist queue have one consumer Start the producer. Check all the MBeans Send multiple messages. All the messages would be received by just one consumer. Go to admin console and turn off the server affinity of connection factory. Restart the consumers and the producer and send the messages again.
6/13/2011 WebLogic JMS 54

Distributed Topics

Durable subscribers cannot be created for distributed topics. However, you can still create a durable subscription on distributed topic member. The topic members can be located anywhere but must all be served either by a single WebLogic Server or any number of servers in a cluster. When a message is sent to a distributed topic, it is sent to all of the topic members in the distributed topic set . Publishing a message to a specific distributed topic member automatically forwards it to all the other distributed topic members, just as publishing a message to a distributed topic

6/13/2011

WebLogic JMS

55

Distributed Topics Routing

6/13/2011

WebLogic JMS

56

DistributedTopic: Publishers

If one or more of the distributed topic members is not reachable, and the message being sent is non-persistent, then the message is sent only to the available topic members. If one or more of the distributed topic members is not reachable, and the message being sent is persistent, then the message is stored and forwarded to the other topic members when they become reachable.

6/13/2011

WebLogic JMS

57

DistributedTopic: Subscribers

Same as is the case with publishers

Any persistent messages published to one or more unreachable distributed topic members are eventually received by topic subscribers of those topic members once they become reachable. Any non-persistent messages published to those unreachable distributed topic members will not be received by that topic subscriber.
Ultimately, a topic subscriber is pinned to a physical topic member. Why are the docs so confusing ??

6/13/2011

WebLogic JMS

58

Failure Recovery of Consumers

If the consumer is synchronous, then the exception is returned to the user directly. If the topic subscriber is asynchronous, then the exception is delivered inside of a ConsumerClosedException that is delivered to the ExceptionListener defined for the consumer session, if any.
The above statements hold true for both Queue Receivers and Topic Subscribers. The above statements hold true for both distributed and non distributed destinations because ultimately a distributed destination consumeris pinned to a single physical member of distributed destination.

Note:

6/13/2011

WebLogic JMS

59

Distributed Destinations

Question: How important is it to keep the system clocks

synchronized among server instances hosting distributed destination members and their connection factories?

Answer:
It is very important when using distributed topics with nondurable subscribers and may lead to non delivery of messages. Both the connection factory and the distributed topic members will use their local system clock to see if a consumer is created after a message is published. Messages published to topics before consumers are created are not visible to consumers.

6/13/2011

WebLogic JMS

60

Flow Control

Threshold Parameters
Bytes/Messages Threshold High Bytes/Messages Threshold Low:Flow control is still in effect for producers that are below their message flow maximum. Producers can move their rate upward until they reach their flow maximum, at which point they are no longer flow controlled

Flow Control Parameters on ConnectionFactory

Flow Maximum:If a producer is not currently limiting its flow when a
threshold condition is reached, the initial flow limit for that producer is set to Flow Maximum. If a producer is already limiting its flow when a threshold condition is reached (the flow limit is less than Flow Maximum), then the producer will continue at its current flow limit until the next time the flow is evaluated. Flow Minimum Flow Interval

6/13/2011

WebLogic JMS

61

Flow Control

Flow Steps
The number of steps used when a producer is adjusting its flow from the Flow Minimum amount of messages to the Flow Maximum amount, or vice versa. The downward movement (the decay) is geometric (For example, 100, 50, 25, 12.5). The movement upward is linear. The difference is simply divided by the number of steps.

6/13/2011

WebLogic JMS

62

JMS Bridge

Messaging Bridge :
It allows you to configure a forwarding mechanism between any two messaging products, thereby providing interoperability between separate implementations of BEA WebLogic JMS or between WebLogic JMS and another messaging product . Source and target bridge destinations can be either queues or topics. You can specify a quality of service (QOS), message filters, transaction semantics, and connection retry policies Console allows to temporarily suspend bridge traffic whenever necessary, tune the execute thread pool size, and monitor the status of all configured bridges.

6/13/2011

WebLogic JMS

63

JMS Bridge

If you configure a bridge instance to forward non-persistent messages, configure and use a connection factory that has the Default Delivery Mode set to Non-Persistent. Make sure to enable trust between two domains (why?) WLS provides 3 adaptors by default: jms-xa-adp.rar: QOS is Exactly-once. This envelops a received message and sends it within a user transaction (XA/JTA). The source and target JMS connection factories must be configured to use the XAConnectionFactory. jms-notran-adp.rar: no transaction semantics. Used when the required QOS is Atmost-once or Duplicate-okay. jms-notran-adp51.rar
6/13/2011 WebLogic JMS 64

JMS Bridge

Increase bridge thread pool to match number of bridge instances if there are more than five synchronous bridges targeted to the same WebLogic Server.
The way to configure it is to add one entry in the Server entry in config.xml. <Server Name="foo"> .............. <ExecuteQueue Name="MessagingBridge" ThreadCount="XXX"> </Server> The name of the execute queue is hard-coded.

Quality of service degradation If enabled, bridge automatically degrades to next achievable QOS Selector The message selector for the bridge allows you to filter the messages that are sent across the bridge.

6/13/2011

WebLogic JMS

65

JMS Bridge

2 1

6/13/2011

WebLogic JMS

66

JMS Bridge

Question: When should I use a messaging bridge? Answer: When you need store-and-forward functionality/high availability at the cost slightly lower throughput. I want to listen on a weblogic destination !!

Question: When should I avoid using a messaging bridge? Answer: Receiving from a remote destinationuse an message driven EJB or implement a client consumer directly.MDBs automatically retry connecting to their source destination if the source destination is inaccessible, so there is no need to insert a messaging bridge in the message path to provide higher availability. it may have a performance impact.
WebLogic JMS 67

6/13/2011

JMS Bridge

Send messages to a local destinationsend directly to the local destination. Environment with low tolerance for message latency.

6/13/2011

WebLogic JMS

68

JMS Bridge

Question: Whats the recommended way to use JMS Bridge

with distributed destinations?

Answer:
If the source is distributed destination, the best practice is to configure one bridge for each member of a distributed destinations using the member's JNDIName. If the target is a distributed destination, the best practice is to send to the distributed destination using the distributed destination's JNDIName and disable server affinity. This allows the distributed destination to load balance incoming messages.

6/13/2011

WebLogic JMS

69

Exercise 7

Objective:
Setup, configure and test a JMS Bridge between two domains using a queue as a source destination and a queue as a target destination.
Create two domains Make sure all the names are unique in the two domains. Create a queue TestQ in domain1 and a queue TestBridgeQueue in domain2. Setup the trust between two domains Deploy the jms-xa-adp.rar file (JNDI: eis.jms.WLSConnectionFactoryJNDIXA) Make sure the ConnectionFactories being used are XA Enabled. Make sure the Guest Disabled on the domain is unchecked. Create the bridge with Exactly-Once QOS Test the bridge

6/13/2011

WebLogic JMS

70

JMS Bridge - Troubleshooting

Most of the problems are related to:

Bridge's destination is not configured correctly, viz,
Connection URL, Destination JNDI Name, Connection Factory JNDI Name ,User Name/Password.

Actual destination is not running and cannot communicate with the messaging bridge. Most of the adaptor related errors, like
Failed to find the source/target adapter Failed to look up the source/target adapter are either because the adapter is not deployed properly or the JNDI name in the JMSBridgeDestination instance is incorrect .

6/13/2011

WebLogic JMS

71

JMS Bridge - Debugging

In almost all cases, it throws a Resource Exception with a message. The actual cause of the problem (such as JMSException or NamingException) is lost. To specify debug parameters either start WebLogic with -Dweblogic.debug.DebugMessagingBridgeRuntime=true -Dweblogic.debug.DebugMessagingBridgeStartup=true Or you can specify in the config.xml
<ServerDebug DebugMessagingBridgeRuntime=true" DebugMessagingBridgeStartup=true Name="myserver/>

6/13/2011

WebLogic JMS

72

JMS Migration

Clients access a migratable service in a cluster using a migration-aware RMI stub. The RMI stub keeps track of which server currently hosts the pinned service, and it directs client requests accordingly. Currently, migration is supported only for JMS servers and the JTA Transaction Recovery Service . Currently, WebLogic Server does not support automatic migration. Its manual.

6/13/2011

WebLogic JMS

73

JMS Migration

JMS can be migrated independent of the JTA . However, since the JTA provides the transaction control, JMS is usually migrated along with JTA. This ensures that the transaction integrity is maintained before and after the migration. Generally JTA migration should be followed by JMS migration. But? All constrained candidate servers of a migratable target must belong to the same cluster What is a migratable target?

6/13/2011

WebLogic JMS

74

JMS Migration

6/13/2011

WebLogic JMS

75

JMS Migration

JMS Stores have to be configured such that both the source and destination servers have access to them. If the application uses file-based persistence (JMSFileStore), it is recommended to have either a SAN (Storage Area Network) or a Dual ported SCSI disk. Migration can be done through Admin Console or the commandline interface utility.
java weblogic.Admin -url AdminHost:7001 -username adminuser -password adminpassword \MIGRATE -migratabletarget myserver2(migratable) -destination myserver3

6/13/2011

WebLogic JMS

76

JMS Migration

Migration not Always Required Persistent Queuing of Time-Sensitive Messages Persistent Pub/Sub with Durable Subscribers Pending XA Transactions Alternatives to JMS Server Migration Restart Failed WL Server on the Same Machine Migrate WL Server to Another Machine

6/13/2011

WebLogic JMS

77

JMS Migration

Question: What happens to active subscribers/consumers who

have open sessions with the JMS server at the point of starting the migration? Are they sessions automatically switched to the new server?

Answer: Any JMS clients that were connected to the failed WebLogic
Server must have a reconnect logic in place.

Question: What about MDBs? I cannot implement any

exceptionListeners on an MDB

Answer: Dont worry its automatic! When a JMS Server migrates,

MDBs on its destinations migrate automatically but make sure that MDBs are deployed homogeneously to the cluster.

6/13/2011

WebLogic JMS

78

Exercise 8

Objective: Create a cluster of two managed servers and migrate a

JMS server from first to the second managed server.

6/13/2011

Create a cluster of two managed servers. Create a Connection Factory targetted to the cluster Create a JMS server targetted to managed1 with a queue TestQ Create a sender and producer for TestQ using cluster url Both the sender and consumer should have Connection ExceptionListener as well as session ExceptionListener Send a dummy message and test it gets received fine Kill the server hosting the JMS Server and check the TYPE OF EXCEPTIONS getting thrown sender and receiver sides. Migrate the JMS Server to managed2 and check if the sender and receiver recover automatically Send another message after migration is successful.
WebLogic JMS 79

Foreign JMS Server

Foreign JMS Server sort of symbolic link between: A JNDI object in another JNDI directory, like a JMS connection factory or destination object A JNDI name in the JNDI name space for your WLS cluster represents a JNDI provider that is outside the WLS

6/13/2011

WebLogic JMS

80

Common JMS problems

JMS OOM JMS lost Message JMS Message Redelivery JMS with JDBC/File store hung under load Pending Messages Clients unable to recover from a failed JMS Server Poison Messages
6/13/2011 WebLogic JMS 81

JMS OOM

Several factors that may lead to OOM are:

Senders outpacing receivers: In this case two corrective approaches may be followed. Throttling the sender as well as tuning the consumers. Message paging: Even in case of persistent messages, headers still remain in memory Zombie consumers in case of Topics

6/13/2011

WebLogic JMS

82

JMS OOM

Throttling addresses the issue of senders outpacing receivers and senders starving the receivers out of their fair share of CPU and other server resources. Quotas,viz, BytesMaximum and MessagesMaximum settings.
When a producer send exceeds configured quotas, WebLogic JMS throws the exception javax.jms.ResourceAllocationException, and the message is not sent. In this case, an application programs typical remedial action is to retry the send after a delay. Flow control (7.0 onwards) to slow down overactive producers. This flow control takes effect if a configurable messages or bytes threshold is exceeded, at which point producers are forced to slow down by delaying the time when their calls to produce a message return. Flow control is configurable via a producers connection factory.

6/13/2011

WebLogic JMS

83

JMS OOM

Bytes/Messages Threshold High

The upper threshold value that may triggers the following events : Log Messages - A message is logged on the server indicating a high threshold condition. Bytes/Message Paging - If paging is enabled (and a paging store has been configured), then paging is started. Flow Control - If flow control is enabled, the JMS server becomes armed and instructs producers to begin decreasing their message flow.

6/13/2011

WebLogic JMS

84

JMS OOM

Bytes/Messages Threshold Low

The lower threshold value that may triggers the following events : Log Messages - A message is logged on the server indicating that the threshold condition has cleared. Bytes/Message Paging - If paging is enabled, paging is stopped (if paging is occurring). Flow Control - If flow control is enabled, the JMS server becomes disarmed and instructs producers to begin increasing their message flow.

6/13/2011

WebLogic JMS

85

JMS OOM

Note: Make sure to close the connection/session/consumers. What if the client dies abruptly ? Core dump, seg fault of JMS client; Bugs in JMS Consumer client ( any race condition, thread safety issues that may kill the client to close the connection pre-maturely); Network problems that end up closing the JMS connection while live session and producers/consumers not close. What does the spec say about it ? Does RJVM Idletimeout and DGC help in any way ?

6/13/2011

WebLogic JMS

86

JMS Message Redelivery

Why does a JMS message get redelivered?

RuntimeException has been thrown from receivers/MDBs onMessage method Call to ejbcontext.setRollbackOnly() in MDBs onMessage() method. (CMT only) MDB participating in a Transaction failed for some reason. For ex: if the MDB calls another EJB, which makes a JDBC call and the database operation fails. In this case the transaction in which MDB is participated will be rolled back and the message will be redelivered.

6/13/2011

WebLogic JMS

87

JMS Message Redelivery

MDB is transactional but takes a long time to process, the transaction monitor will timeout the transaction and rolls it back - consequently, JMS will redeliver the message. User has made a call to Session.recover() from a standalone receiver Session.acknowledgement () is not called from a standalone receiver with client acknowledgement. But how many times ? What does the spec say about it ?

6/13/2011

WebLogic JMS

88

JMS with JDBC store hung under load

Consider a scenario where a JMS server with JDBC store gets hung under load. Thread dump shows many threads waiting at StoreRequest.waitForIOComplete() call. For eg:

"ExecuteThread: '14' for queue: 'default'" daemon prio=5 tid=0x46d13e90 nid=0x5ac waiting on monitor [0x47fcf000..0x47fcfdb8] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:415) at weblogic.jms.store.StoreRequest.waitForIOComplete(StoreRequest.java:393) at weblogic.jms.backend.BEXATranEntryReceive.finishCommit(BEXATranEntryReceive.java:152) at weblogic.jms.backend.BEXAResource.commit(BEXAResource.java:1090) at weblogic.transaction.internal.ServerResourceInfo.commit(ServerResourceInfo.java:1350) at weblogic.transaction.internal.ServerResourceInfo.commit(ServerResourceInfo.java:527) at weblogic.transaction.internal.ServerSCInfo.startCommit(ServerSCInfo.java:273) at weblogic.transaction.internal.ServerTransactionImpl.localCommit(ServerTransactionImpl.java:1512) at weblogic.transaction.internal.ServerTransactionImpl.globalRetryCommit(ServerTransactionImpl.java) at weblogic.transaction.internal.ServerTransactionImpl.globalCommit(ServerTransactionImpl.java:2035) at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:246) at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:212) at weblogic.ejb20.internal.MDListener.execute(MDListener.java:406) at weblogic.ejb20.internal.MDListener.transactionalOnMessage(MDListener.java:311) at weblogic.ejb20.internal.MDListener.onMessage(MDListener.java:286) at weblogic.jms.client.JMSSession.onMessage(JMSSession.java:2350) at weblogic.jms.client.JMSSession.execute(JMSSession.java:2266) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:234) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:210)

6/13/2011

WebLogic JMS

89

JMS with JDBC store hung under load

One of the threads looks like:

"ExecuteThread: '1' for queue: 'JMSStore<null>.ioThreadPool'" daemon prio=5 tid=0x4745c400 nid=0xcf8 runnable [0x48f8f000..0x48f8fdb8] at java.net.SocketInputStream.socketRead(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:85) at oracle.net.ns.Packet.receive(Unknown Source) at oracle.net.ns.DataPacket.receive(Unknown Source) at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source) at oracle.net.ns.NetInputStream.read(Unknown Source) at oracle.net.ns.NetInputStream.read(Unknown Source) at oracle.net.ns.NetInputStream.read(Unknown Source) at oracle.jdbc.ttc7.MAREngine.unmarshalUB1(MAREngine.java:931) at oracle.jdbc.ttc7.MAREngine.unmarshalSB1(MAREngine.java:893) at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:375) at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1894) at oracle.jdbc.ttc7.TTC7Protocol.executeFetch(TTC7Protocol.java:956) at oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:2138) at oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:2015) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:2877) at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:608) at weblogic.jdbc.pool.Statement.executeUpdate(Statement.java:318) at weblogic.jms.store.JDBCIOStream.write(JDBCIOStream.java:525) at weblogic.jms.store.StoreRequest.doTheIO(StoreRequest.java:255) at weblogic.jms.store.JMSStore.execute(JMSStore.java:326) at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:234) at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:210)

JMS is marked unavailable as an XA Resource

6/13/2011 WebLogic JMS 90

JMS with JDBC store hung under load

How does it work ?

JMS layer submits an I/O request to the store JDBC store queues it up I/O thread performs submitted requests in order of submission I/O thread sends a signal to the submitted I/O request when it completes

Tuning Parameters
MaxXACallMillis (default 120 seconds) MaxResourceUnavailableMillis (default 30 minutes)

6/13/2011

WebLogic JMS

91

Pending Message

Pending means the message could have been:

sent in a transaction but not committed. received and not acknowledged. received and not committed. subject to a redelivery delay (as of WebLogic JMS 6.1 or later). subject to a delivery time (as of WebLogic JMS 6.1 or later). A rolled back message remains pending until the transaction actually rolls back. Rolling it back multiple times does not cause double counting, nor does an exception that set a transaction as rollbackOnly followed by an actual rollback.
Basically a pending message is an in-flight message

6/13/2011

WebLogic JMS

92

Pending Message

MessageMaximum attribute for the connectionFactory (Will talk more

about it during performance talk)

In many cases, its WAD. For example, customer knows that a particular consumer is bound to take a long time in processing a single message. Infected in a Distributed Transaction Persistent messages with JDBC jms store under heavy load.

6/13/2011

WebLogic JMS

93

Clients unable to recover from a failed JMS Server

More important is to understand the expected behavior as per the jms specs. For eg:
Application code is supposed to recover a synchronous consumer Asynchronous consumer would recover only if an exception listener has been registered with the connection object.

In case of JMS server migration, exception listener should be registered with weblogic proprietary session object. All care should be taken to cleanup the JMS objects gracefully.
However no choice for clients getting killed/disconnected abruptly. JMS Specs section 4.3.5: Since a provider typically allocates significant resources outside the JVM on behalf of a connection, clients should close these resources when they are not needed. Relying on garbage collection to eventually reclaim these resources may not be timely enough.

6/13/2011

WebLogic JMS

94

Poison Messages

Poison messages are messages that a receiver must reject. Typically, a message is rejected due to a problem with the message itself, but a message can also be rejected due to a temporary resource outage. If they occur on an asynchronous consumer, the poison message and the subsequent messages following it in the pipeline may need to be flushed, re-inserted into their originating destination, and pipelined again. Leads to endless loop

6/13/2011

WebLogic JMS

95

Poison Messages

Redelivery Delays : It tells JMS to defer redelivering the

message for some amount of time. The message stays out of circulation until the application thinks it is ready to try again, without preventing other messages behind the delayed message from being delivered.

Redelivery Limits: If a message is rolled back or recovered

more times than this limit, then message is removed from the queue and forwarded to its destinations error destination. If the error destination is not configured, and a message exceeds its redelivery limit, that message is simply deleted.

Error Destination
6/13/2011 WebLogic JMS 96

Poison Messages

Asynchronous JMS applications that have access to their Connection may pause message delivery by calling javax.jms.Connection.stop(), and can later call javax.jms.Connection.start() to continue processing messages.
What about MDB ?

6/13/2011

WebLogic JMS

97

JMS debug

JMS:
DebugJMSBackEnd="true" DebugJMSBoot="true" DebugJMSCommon="true" DebugJMSConfig="true" DebugJMSFrontEnd="true" DebugJMSLocking="true" DebugJMSStore="true" DebugJMSXA="true DebugMsgpath=true

MDB:
-Dweblogic.ejb.jms.connect.verbose=true -Dweblogic.ejb.jms.connect.debug=true -Dweblogic.ejb20.jms.poll.debug=true
6/13/2011 WebLogic JMS 98

Important points

WebLogic JMS does not drop (silently delete) messages, except where the designer has explicitly asked for a lower QOS (such as using the multicast option for pub/sub,or message expiration). JMS asynchronous consumers that share a JMS session must not run currently (session should be single threaded- jms specs) A JMS Server can be deployed on only one server. A queue exists only on the JMS Server it was configured for. All the queues on one JMS Server share the same persistent store. No two JMS Servers can share the same persistent store. There can be more than one persistent JDBC store in a database using prefixes. There can be more than one persistent file store in a directory. Templates are needed to create temporary destinations on any JMSServer

6/13/2011

WebLogic JMS

99

Tuning JMS Performance

Benchmark Tips

Throughput:
Measuring sends-per-second is also often misleading, as the rate at which messages enter the system is usually not of interest. Also, sends almost always use fewer resources than receives, and are consequently less likely to be a bottleneck.

Scalability.
Can the server handle dozens, hundreds, or even thousands of connections? How is throughput affected as the number of connections increase? Ideally, it should increase as well.

6/13/2011

WebLogic JMS

101

Benchmark Tips

Fairness:
Ensure that homogenous applications tend to all get served at about the same rate, and that all take turns getting served.

Dropped messages:
WebLogic JMS does not drop (silently delete) messages, except where the designer has explicitly asked for a lower QOS (such as using the multicast option for pub/sub,or message expiration). This is not true for all vendors

6/13/2011

WebLogic JMS

102

Benchmark Tips

Concurrency:
JMS asynchronous consumers that share a JMS session must not run currently (session should be single threaded- jms specs)

Persistence:
JDBC or a file store? If it is a file store, are synchronous writes used for both sends and receives? -Dweblogic.JMSFileStore.store-name.SynchronousWritesEnabled=false

6/13/2011

WebLogic JMS

103

Performance Tuning

Weblogics JMS Performance Tuning can be divided into two broad categories: Configuration Tuning Application Design

6/13/2011

WebLogic JMS

104

Configuration Tuning

Tuning Thread Pools and EJB Pools Client-Side Thread Pools

-Dweblogic.ThreadPoolSize=n (def = 5) -Dweblogic.JMSThreadPoolSize=n

Server-Side Thread Pools eg: <ServerName="server1 NativeIOEnabled="true JMSThreadPoolsSize=18 ServerVersion="7.0.0.0"> <ExecuteQueueName="default ThreadCount=18"/> </Server>

6/13/2011

WebLogic JMS

105

Configuration Tuning

Server-Side Application Pools max-beans-in-free-pool, dispatch policies

6/13/2011

WebLogic JMS

106

Configuration Tuning

Configuring JVM Memory JDBC Stores vs. File Stores Both have the same transaction semantics and guarantees. File stores generate no network traffic; JDBC stores will generate network traffic if the database is on a different machine from the JMS server. File stores are much better suited to paging non-persistent messages. JDBC stores may make it easier to handle failure recovery

6/13/2011

WebLogic JMS

107

Configuration Tuning

Persistent Store Tuning

File Store Tuning
Disabling Synchronous Writes Enhanced Hardware: consider using high-performance hardware for improved reliability, consider using hardware disk mirroring.

JDBC Store Tuning : Follow that particular DBMS tuning guide

6/13/2011

WebLogic JMS

108

Configuration Tuning

Partitioning Your Application

Spreading destinations across multiple stores:

Multiple JMS servers may still optionally be targeted at the same WebLogic server instance, but their stores should typically be on separate databases/disks.
Why do you think it will help ? Clustering JMS
Distributed destinations JMS Server migration

6/13/2011

WebLogic JMS

109

Configuration Tuning

Connection Load Balancing: This round-robin load balancing occurs each time the connection factory is looked up, and each time create connection is called. For example, if a connection factory, CF1, is configured with two targets in a cluster, server1 and server2, and remote JMS clients (e.g. non-server-side clients) obtain their JMS connection using CF1, then the first client routes JMS requests through server1, the second through server2, the third through server1, and so on Connection concentration : WebLogic Server clients always use a single, shared socket between a client and a particular server, no matter how many sessions and consumers exist between them.

6/13/2011

WebLogic JMS

110

Configuration Tuning

Co-locating XA RMs in a distributed tx reduces network overhead eg: Consider an EJB invoking another EJB that sends a JMS to a destination sitting on another server.
reduces # of involved resources. For eg:

if a transaction typically involves two destinations, it helps to make sure that both destinations reside on the same instance of WebLogic Server. In this case, there is a further advantage in configuring both destinations to reside on the same JMS server, which reduces the number of involved resources to one, and thus allows the transaction to optimize and become one-phase.
6/13/2011 WebLogic JMS 111

Configuration Tuning

Tuning the Asynchronous Pipeline Size (MessageMaximum attribute on ConnecionFactory)

Especially if the JMS application defers acknowledges/commits. 2*(ack or commit interval)+1
What are pros and cons of increasing this parameter ?

Message Paging
If enabled, messages are paged once the number of bytes and/or number of messages exceeds a configurable threshold. Persistent messages are never paged to the paging store. If paging is enabled, the message bodies are removed from memory since we already have a persistent copy of the message

6/13/2011

WebLogic JMS

112

Application Design

Topic Vs Queue Asynchronous vs. Synchronous Consumers

A synchronous consumer consumes a thread for the duration of its receive call. As a result, a thread can remain idle for long periods, especially if the call specifies a blocking timeout.

Persistent vs. Non-Persistent:

In WebLogic 7.0, if there are no subscribers, or there are only nondurable subscribers for a topic, the messages will be downgraded to non-persistent. Durable subscribers require a persistent store to be configured on their JMS server, even if they receive only non-persistent messages. WebLogic JMS automatically forces all messages in a temporary destination to non-persistent.

6/13/2011

WebLogic JMS

113

Application Design

Transacted Session vs. User Transactions Transacted sessions ignore JTA user transactions A transacted session therefore can not participate in an EJB or MDB container managed transaction, or a user transaction explicitly started by an application. A WebLogic MDB can be user transaction aware, but not transacted.

Producer Pooling
Just as consumers can be pooled using an MDB pool or a ServerSessionPool, producers can also be pooled. There is no J2EE standard for producer pools, and WebLogic Server currently does not provide one, but an application programmer can write one.
Note: We do provide a sample code on dev2dev

Caching of lookups
6/13/2011 WebLogic JMS 114

Q&A

6/13/2011

WebLogic JMS

115

TM

How business becomes e-business www.bea.com

6/13/2011 WebLogic JMS 116