TIBCO Administrator Fault tolerance

TIBCO Administrator offers fault tolerance capabilities by allowing a client application to be served by multiple servers. Fault tolerance is implemented through the use of the TIBCO Rendezvous distributed queue protocol (RVDQ). Each logical server has a distributed queue based on the server name. The servers in a distributed queue group share the same server name. The TIBCO Administrator client just sees one logical server. Pre-requisites TIBCO Runtime Agent TIBCO Administrator TIBCO HAWK Agent TIBCO Rendezvous Communications daemon To give client applications access to the same data by way of multiple servers, one primary and one or more secondary servers must be installed in an administration domain on multiple machines. Checkpoint activity in TIBCO Business Works process Steps to install TIBCO Administrator: Refer to TIBCO Administration Installation Guide included in the documentation of TIBCO Administrator Primary and Secondary Servers A TIBCO administrative domain can have only one Primary Server and any number of Secondary Servers. Whenever TIBCO administrator is installed it creates an administrative domain with the machine, on which the software is installed, acting as the Primary Server. Secondary Servers can be added to an existing administrative domain by using the Domain Utility. TIBCO Domain Utility is installed as part of the TIBCO Runtime Agent installation. The utility is launched directly after installing the TIBCO Administrator administration server and is used to create the initial administration domain. You can start the utility at any time to manage domain machines, domain configurations, administration server settings, upgrade administration domains and register a TIBCO Enterprise Message Service server. TIBCO Domain Utility is documented in the TIBCO Runtime Agent Domain Utility Users Guide. The guide is part of the TIBCO Runtime Agent documentation set and is available in the <install-path>\tibco\tra\ <version>\doc directory.

Adding a Secondary Server to a Domain 1. Start the Domain Utility and click the Next button on the main screen. 2. Under Category, click Domain Configuration, then click Add a secondary server. 3. Click next and Click the Discover button and select a domain. If a domain does not appear, select Show Advanced and increase the Discover Timeout value. The value sets the amount of time the Domain Utility has to connect to the master server. If no connection is made in the specified time, the discovery operation wills time out. Increase this number on slow systems and click the Discover button again. 4. Click the Next button. The screen that appears displays the ports used by the Web Server. 5. Click the Next button Provide the administration credentials that were defined when the domain was created. 6. Click the Next button. That appears displays a summary of the values you have provided. 7. Click the Next button to add the secondary server to the domain. After joining the domain, the services that support the secondary server are listed. You must start each service to enable the secondary server. 8. Click Finish to end the session. Refer to TIBCO Runtime Agent Domain Utility Users Guide for a description of Secondary Server Fields Setting Fault Tolerant Options for a Process Fault Tolerance options for a process can be set through TIBCO Administrator in the Primary Server only. The FT (Fault Tolerance) Group Settings are displayed only when the particular Business Works process has been deployed on multiple machines in an administrative domain. The Business Works Process should also contain a Checkpoint activity so that it can be run in Fault Tolerant Mode. To Set Fault Tolerant Options 1. In TIBCO Administrator, click Application Management. 2. Select an application and expand it. 3. In the Configuration Builder pane, click process name. A process is named with a .par suffix. 4. Click the General tab. 5. To Deploy a process to multiple machines a. In the Target Machines pane, click Add to Additional Machines to add a process to another machine b. A dialog box will appear displaying all the machines in the domain. Select a machine and click OK. c. Under the FT Weight column select one process instance as primary and the remaining instances as secondary process. Members of a fault-tolerant group can be configured as peers or as master and secondary engines. If all engines are peers, when the machine containing the currently active process engine fails, another peer process engine resumes processing for the first engine, and continues processing until its machine fails. If the engines are configured as master and secondary, the secondary engine resumes processing when the master fails. The secondary engine continues processing until the master recovers. Once the master recovers, thesecondary engine shuts down and the master takes over processing again. The Fault Tolerance tab of the Process Engine deployment resource allows you to specify the member weight of each member of a fault-tolerant group. The member with the highest weight is the

master. You can select "Peer" in the first field on the tab to configure all engines as peers (that is, they all have the same weight). You can select Primary/Secondary to configure the engines as master and secondary. You can also select Custom to specify your own values for the weight of each member of the group. d. Click Save. 6. The Fault Tolerance Options are now displayed. a. Select Run Fault Tolerant. b. Set the Heartbeat Interval: The master engine of a fault-tolerant group broadcasts heartbeat messages to inform the other group members that it is still active. The heartbeat interval determines the time (in milliseconds) between heartbeat messages. In the event if one process engine fails, another engine detects the stop in the masters heartbeat and resumes operation in place of the other engine. All process starters are restarted on the secondary, and services are restarted to the state of their last checkpoint c. Set the Activation Interval: Secondary process engines track heartbeat messages sent from the master engine. This field specifies the amount of time to expire since the last heartbeat from the master before the secondary restarts the process starters and process engines. The Heartbeat Interval should be smaller than the Preparation Interval, which should be smaller than the Activation interval. It is recommended that Activation Interval be slightly over 2 heartbeats. d. Set the Activation Delay: When a master engine resumes operation, the secondary engine shuts down and returns to standby mode. For some situations, it may be necessary to ensure that the secondary engine has completely shut down before the master engine resumes operation. This field is used to specify a delay before the master engine restarts. When the time since the last heartbeat from an active member exceeds this value, the ranking inactive member will receive a "hint" so that it can prepare for activation. e. Click Save. Deploy the Process: 1. In TIBCO Administrator, click Application Management. 2. Click Application > Configuration, where Application is the application created when you loaded the enterprise archive file. 3. Click Deploy and then OK to deploy the application. Running the Process in Fault Tolerant Mode 1. Under Application Management select the deployed application. 2. Click on Service Instances. It will display the number of process instances which can be started. It also displays the IP address of the machine on which the process instance is deployed. 3. One or more process engines in the group can be started. If more than one engine has started, only one is displayed as running (the process residing on the Primary Server) and all other engines (Secondary Server) are displayed as Standing By (initially, as Starting Up). If you shutdown a Primary process engine, the appropriate secondary engine starts automatically. Once the master recovers, the secondary engine shuts down and the master takes over processing again. Restrictions/Constraints: 1. On a particular machine both primary server and secondary server for a single domain cannot be installed. Primary server and secondary server for a particular domain should be installed on different machines.

2. Multiple secondary servers for the same domain cannot be installed on a single machine. 3. Fault tolerance relies on the administrator server. Therefore, the administrator server must be up and running for fault tolerance to work properly. 4. Fault Tolerant options are available only for TIBCO Business Works Process and not for TIBCO Adapter Services 5. When a master process engine fails, its process starters are restarted on the secondary engine. This may not be possible with all process starters. For example, the HTTP Receiver process starter listens for HTTP requests on a specified port on the machine where the process engine resides. If a secondary engine resumes operation for a master engine, the new machine is now listening for HTTP requests on the specified port. HTTP requests always specify the machine name, so incoming HTTP requests will not automatically be redirected to the new machine.