webMethods Integration Server and webMethods Developer Documentation Supplement Version 6.

1 Service Pack 2

Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Validating Published Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Handling Retry Failure for Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Managing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Server Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Introduction
webMethods Integration Server 6.1 SP2 and webMethods Developer 6.1 SP2 provide new features designed to improve the performance and resource utilization of publish-andsubscribe solutions. webMethods Integration Server 6.1 SP2 also provides the ability to manage triggers from Integration Server Administrator. This supplement describes these new features. For more information about the fixes and enhancements included in webMethods Integration Server 6.1 SP2 and webMethods Developer 6.1 SP2, see the Enhancements and Fixes documents located on webMethods Advantage.

Validating Published Documents

Note: The information in this section supplements the contents of the Publish-Subscribe Developers Guide In a publish-and-subscribe solution, Integration Server validates a published document against the associated publishable document type. Validation occurs immediately after the publishing service executes. If Integration Server determines that the published document is invalid (that is, the published document does not conform to the associated publishable document type), the publishing service returns a service exception that indicates the validation error. Integration Server does not publish the document to webMethods Broker or, in the case of local publishing, to the dispatcher. While document validation ensures that document subscribers receive valid documents only, it can be an expensive operation in terms of resources and performance. In some situations, you might not want to validate the published document. For example, you might want to disable document validation when publishing documents that were already validated. Suppose that a back-end resource, such as SAP, created and validated the document and then sent it to Integration Server. If Integration Server in turn,

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

publishes the document to Broker, you might not need to validate the document when publishing it because it was already validated in SAP. Integration Server offers a global setting that you can use to require validation, prevent validation, or indicate that validation should be configured per document type. Developer provides a publishable document type property that you can use to indicate whether instances of a specific publishable document type should be validated. The following sections explain these settings. Note: When deciding whether to disable document validation, be sure to weigh the advantages of a possible increase in performance against the risks of publishing, routing, and processing invalid documents.

Globally Configuring Validation for Published Documents

Integration Server provides a global setting that you can use to control validation of published documents. You can configure this setting to indicate that: Integration Server always validates published documents. Integration Server never validates published documents. Integration Server validates only those published documents that are an instance of a publishable document type for which validation is enabled. This is the default. The following procedure explains how to globally enable or disable validation for published documents. To globally enable or disable validation for published document 1 2 3 4 5 In Integration Server Administrator, go to the Settings > Extended page and click Show and Hide Keys. Select the check box next to watt.server.publish.validateOnIS. Click Save Changes. Integration Server Administrator displays the selected property in the Extended Settings box. Click Edit Extended Settings. In the Extended Settings box, set the watt.server.publish.validateOnIS to one of the following values: Specify...
always

To.... Perform document validation for all published documents. This includes instances of publishable document types for which the Validate on publish property is set false.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Specify...
never

To.... Disable document validation for all publishable document types. This includes instances of publishable document types for which the Validate on publish property is set true. Perform document validation only for instances of publishable document types for which document validation is enabled. That is, Integration Server validates published documents that are instances of publishable document types for which the Validate on publish property is set true. This is the default behavior.

perDoc

6 7

Click Save Changes. Restart Integration Server.

Specifying Validation for a Publishable Document Type

You can indicate on a per document type basis whether or not Integration Server validates instances of that publishable document type. To specify validation for instances of a publishable document type 1 2 In the Navigation panel in Developer, open the publishable document type for which you want to specify validation. In the Properties panel, under Publication, set the Validate on publish property to one of the following: Select... True To... Perform validation for published instances of this publishable document type. This is the default. False Disable validation for published instances of this publishable document type.

On the File menu, click Save.

Notes: Integration Server ignores the value of the Validate on publish property if the watt. server.publish.validateOnIS property is set to always or never.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

webMethods Broker can also be configured to validate the contents of a published document. When it receives the document from an Integration Server, Broker checks the validation level of the Broker document type associated with the published document. If the validation level is set to Full or Open, Broker validates the document contents. If the validation level is set to None, Broker does not validate the document contents. By default, Broker assigns Broker document types created from a publishable document type on an Integration Server a validation level of None. For more information about configuring document validation on the Broker, see the webMethods Broker Administrators Guide.

Handling Retry Failure for Triggers

Note: The information in this section supplements the contents of the Publish-Subscribe Developers Guide. When building a trigger, you can indicate that a trigger service should retry automatically if it fails because of a run-time exception. A run-time exception (specifically, an ISRuntimeException) occurs when the trigger service catches and wraps a transient error and re-throws it as a run-time exception. A transient error is an error that arises from a temporary condition that might be resolved or corrected quickly, such as the unavailability of a resource due to network issues or failure to connect to a database. For each trigger, you can specify the maximum number of attempts Integration Server should make to retry the service. You can also specify how Integration Server handles retry failure for the trigger. That is, you can indicate what action Integration Server takes when it reaches the maximum number of retry attempts and the trigger service still fails because of a run-time exception. You can select one of the following options: Throw service exception. When Integration Server exhausts the maximum number of retry attempts, Integration Server treats the last trigger service failure as a service error. Integration Server rejects the document, and then, if the document is guaranteed, returns an acknowledgement. If the trigger service is configured to generate audit data and save a copy of the input pipeline to the audit log, you can process the document manually using webMethods Monitor to re-invoke the trigger service at a later time. This is the default behavior. Suspend and retry later. When Integration Server reaches the maximum number of retry attempts, Integration Server suspends the trigger and then retries the trigger service at a later time. The following section provides more information about the Suspend and retry later option. For more information about configuring a trigger to retry, see the Publish-Subscribe Developers Guide.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Overview of Suspend and Retry Later

Sometimes, the reason a trigger service continually fails due to a run-time exception is because the transient error that causes the run-time exception cannot be remedied before Integration Server makes all the retry attempts. For example, a transient error condition caused by a database failure might not be remedied until the database is restarted. The time required to restart the database might be greater than the time needed to make all the retry attempts for the trigger service. Alternatively, a trigger might meet the retry limit if a back-end resource used by the trigger service has been taken off line for some weekly maintenance. The default retry failure behavior (Throw service exception) provides a way for you to resubmit the document when the condition that caused the transient error has been remedied. However, this is a manual process that requires webMethods Monitor and requires that service auditing be configured a particular way. Additionally, because the document has already been acknowledged, the processing order will not match the publishing order. The processing order might be especially important for serial triggers that are set up to evaluate a set of documents in a particular order (usually, the processing order matches the publishing order). The Suspend and retry later option provides a way to resubmit the document programmatically and to preserve the publishing order. The following table provides more information about how the Suspend and retry later option works. Step
1 2

Description Integration Server makes the final retry attempt and the trigger service fails because of an ISRuntimeException. Integration Server suspends document processing and document retrieval for the trigger temporarily. The trigger is suspended on this Integration Server only. If the Integration Server is part of a cluster, other servers in the cluster can retrieve and process documents for the trigger. Note: The change to the trigger state is temporary. Document retrieval and document processing will resume for the trigger if Integration Server restarts, the trigger is enabled or disabled, or the package containing the trigger reloads. You can also resume document retrieval and document processing manually using Integration Server Administrator or by invoking the pub.trigger:resumeRetrieval and pub.trigger:resumeProcessing public services.

Integration Server rolls back the document to the trigger document store. This indicates that the required resources are not ready to process the document and makes the document available for processing at a later time. For serial triggers, it also ensures that the document maintains its position at the top of trigger queue.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Step
4

Description Integration Server schedules and executes a resource monitoring service. A resource monitoring service is a service that you create to determine whether the resources associated with a trigger service are available. A resource monitoring service returns a single output parameter named isAvailable. If the resource monitoring service indicates that the resources are available (that is, the value of isAvailable is true), Integration Server resumes document retrieval and document processing for the trigger. If the resource monitoring service indicates that the resources are not available (that is, the value of isAvailable is false), Integration Server waits a short time interval (by default, 60 seconds) and then re-executes the resource monitoring service. Integration Server continues executing the resource monitoring service periodically until the service indicates the resources are available. Tip! You can change the frequency at which the resource monitoring service executes by modifying the value of the watt.server.trigger.monitoringInterval property.

After Integration Server resumes the trigger, Integration Server passes the document to the trigger. The trigger and trigger service process the document just as they would any document in the trigger queue. Note: At this point, the retry count is set to 0 (zero).

Configuring Retry Failure for a Trigger

When configuring retry failure for a trigger, keep the following points in mind: Integration Server retries a trigger service only if the trigger service catches a transient error and then throws a run-time exception. Retry failure handling can be specified only for triggers configured to make a limited number of retry attempts. If the Deliver until property is set to Successful, retry failure handling cannot be specified. If you want Integration Server to suspend the trigger and retry it later, you must provide a resource monitoring service that Integration Server can execute to determine when to resume the trigger. For more information about building resource monitoring service, see Building a Resource Monitoring Service on page 7. Documents can be resubmitted to the trigger service regardless of what retry failure handling option you select. Use the following procedure to specify how Integration Server handles a retry failure for a trigger.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

To configure retry failure for a trigger 1 2 In the Navigation panel in Developer, open the trigger for which you want to configure retry failure handling. In the Properties panel, under Retries on Error, set the Retry failure behavior property to one of the following: Select... Throw service exception To... Indicate that Integration Server throws a service exception when the last allowed retry attempt ends because of a run-time exception. This is the default. Suspend and retry later Indicate that Integration Server suspends the trigger when the last allowed retry attempt ends because of a run-time exception. Integration Server retries the trigger service at a later time.

If you selected Suspend and retry later, then in the Resource monitoring service property specify the service that Integration Server should execute to determine the availability of resources associated with the trigger service. Multiple triggers can use the same resource monitoring service. On the File menu, click Save.

Building a Resource Monitoring Service

A resource monitoring service is a service that you create to check the availability of resources used by a trigger service. When Integration Server suspends a trigger because all retry attempts have failed, Integration Server executes the resource monitoring service to determine if all the resources are available. If you configure a trigger to suspend and retry later when a retry failure occurs, you must create a resource monitoring service. If a resource monitoring service is not specified, the Integration Server never checks the status of the resources associated with the trigger service, never resumes the trigger, and never attempts to retry the trigger service for the document. The trigger remains suspended until the Integration Server restarts, the trigger is enabled or disabled, the package containing the trigger reloads, or you resume the trigger manually. A resource monitoring service must do the following: Use the pub.trigger:resourceMonitoringSpec as the service signature.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Check the availability of the resources used by all the trigger services associated with a trigger. Each condition in a trigger can be associated with a different trigger service. However, you can only specify one resource monitoring service per trigger. Return a value of true or false for the isAvailable output parameter. The author of the resource monitoring service determines what criteria makes a resource available. Catch and handle any exceptions that might occur. If the resource monitoring service ends because of an exception, Integration Server logs the exception and continues as if the resource monitoring service returned a value of false for the isAvailable output parameter. The same resource monitoring service can be used for multiple triggers. When the service indicates that resources are available, Integration Server resumes document retrieval and document processing for all the triggers that use the resource monitoring service.

pub.trigger:resourceMonitoringSpec
WmPublic. Specification for the signature of a resource monitoring service. Input Parameters None. Output Parameters isAvailable String Indicates whether the resources needed by the trigger service are available. The value of this field determines whether Integration Server resumes document processing and document retrieval for the trigger or reexecutes the resource monitoring service. Integration Server continues to execute a resource monitoring service until the value of isAvailable is true. The isAvailable field must have one of the following values: Value
true

Description Indicates that the resources associated with the trigger service are available. Integration Server resumes document retrieval and document processing for the trigger. Indicates that the resources associated with the trigger service are not available. Integration Server will not resume the trigger.

false

Usage Notes The pub.trigger:resourceMonitoringSpec must be used as the service signature for any service used as a resource monitoring service. A resource monitoring service determines whether the resources associated with a trigger service are available for document processing. Integration Server executes a resource monitoring service after retry failure occurs for the trigger. For more information about building a resource monitoring service, see Building a Resource Monitoring Service on page 7.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Managing Triggers
Note: The information in this section replaces the contents of Appendix D: Managing Document Retrieval and Document Processing in the webMethods Integration Server Administrators Guide. Within the publish-and-subscribe model, document retrieval is the process in which the Integration Server uses a server thread to fetch more documents from the Broker. Document retrieval requires a server thread with which to request and retrieve documents from the Broker. Document retrieval also requires memory because the Integration Server keeps temporary copies of the documents it is retrieving in memory. The Integration Server releases the temporary copies from memory after successfully processing the document. The Integration Server provides controls that you can use to manage the server resources used for document retrieval. Specifically, you can use the controls to: Limit the number of server threads used for document retrieval. Manage the rate of document retrieval and the amount of memory used during document retrieval by adjusting trigger queue capacity. Suspend or resume document retrieval for one or more triggers. These controls can be used during development, capacity planning, or at run time. The following sections provide more information about these controls.

Increasing or Decreasing Threads for Document Retrieval

During production and capacity planning, you can increase or decrease the number of threads used to retrieve documents from the Broker. By default, the Integration Server can use up to 100% of the server thread pool to retrieve documents. Each trigger uses a separate server thread to retrieve documents from the Broker. For example, if the maximum size of the server thread pool is 80 threads, and the server can use 100% of the server thread pool to retrieve documents, then up to 80 triggers can request more documents at one time. Note: You can only specify threads for document retrieval if your integration solution includes a Broker. You can limit the maximum number of threads used for document retrieval by specifying the percentage of the server thread pool that can be used to retrieve documents. The Integration Server uses the specified percentage to calculate the number of server threads that can be used to retrieve documents from the Broker.

ISDEV-DS-61SP2-20060623

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

For example, suppose that the maximum size of the server thread pool is 80 threads. If you specify a maximum document retrieval threads percentage of 10%, then the Integration Server can use only 8 threads to retrieve documents at one time. Because the Integration Server uses a separate thread to retrieve documents for each trigger, this means that the Integration Server can retrieve documents for only 8 triggers at one time. Reducing the percentage of the server thread pool used for document retrieval can slow the rate of document retrieval because fewer triggers can retrieve documents simultaneously. It also ensures the availability of server threads for other tasks, such as answering HTTP requests or processing documents. Increasing the percentage of the server thread pool available for document retrieval can increase the arrival rate of documents because it allows more triggers to retrieve documents from the Broker at one time. For more information about setting the number of server threads for document retrieval, see the webMethods Integration Server Administrators Guide.

When to Increase or Decrease Threads for Document Retrieval

Your knowledge of your integration solution is the best tool for determining when to increase and decrease thread usage for document retrieval. For example, if you know that the Integration Server regularly receives a high number of HTTP requests during a certain time period, you might want to decrease thread usage for document retrieval right before the HTTP requests usually begin, then increase document retrieval thread usage after the frequency of HTTP requests slows down. Alternatively, if you know that the Integration Server receives a high volume of incoming documents at the same time each day, you might want to increase the number of threads available for document retrieval during that time period. You can also determine when to increase or decrease threads for document retrieval by monitoring the number of available server threads. To assist with this, you can establish a warning threshold that instructs the Integration Server to alert you when the percentage of available threads drops below a specified level. Specifically, the Integration Server creates a journal log entry stating Available Thread Warning Threshold Exceeded. When you receive this message in the journal log, you can decrease threads for document retrieval to make server threads available to perform other functions. For more information about setting an available threads warning threshold, see the webMethods Integration Server Administrators Guide. Another method of determining when to alter the number of server threads allotted for document retrieval is to monitor the current number of threads retrieving documents from the Broker. The Integration Server Administrator displays this value in the Current Threads field under Document Retrieval on the Settings > Resources > Trigger Management page.

ISDEV-DS-61SP2-20060623

10

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Note: You can also control the resources used for document retrieval by adjusting trigger queue capacity and suspending or resuming document retrieval for triggers. For more information about adjusting trigger queue capacity, see Decreasing the Capacity of Trigger Document Stores on page 11. For more information about suspending (or resuming) document processing, see Suspending and Resuming Document Retrieval on page 13.

Decreasing the Capacity of Trigger Document Stores

You can affect the amount of memory used for document retrieval by adjusting the capacity and refill level of all the trigger queues. The capacity determines the maximum number of documents that the trigger document store can contain. The refill level specifies the number of documents that remain in the trigger queue before the Integration Server requests more documents from the Broker. You can use the Queue Capacity Throttle provided in the Integration Server Administrator to decrease the capacity and refill levels of all the trigger queues on the Integration Server. The Queue Capacity Throttle reduces the capacity and refill levels for all the trigger queues by the same percentage. For example, if you set the Queue Capacity Throttle to 50% of maximum, a trigger queue with a capacity of 10 and a refill level of 4 will have an adjusted capacity of 5 and an adjusted refill level of 2. By decreasing the capacity and refill levels, you can: Reduce the amount of memory needed to retrieve documents from the Broker. Reduced capacity and refill levels mean that the Integration Server retrieves fewer documents for a trigger at one time. Because the Integration Server retrieves fewer documents, the Integration Server uses less memory when retrieving documents. Reduce the memory needed to store the documents while they await processing. Note: Decreasing the capacity might increase the frequency with which the Integration Server retrieves documents, because the Integration Server might empty the trigger document store to the adjusted refill level more quickly.

To decrease the capacity and refill level of trigger queues 1 2 3 Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management, and then click Edit Global Trigger Controls.

ISDEV-DS-61SP2-20060623

11

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Under Document Retrieval, in the Queue Capacity Throttle field, select the percentage of configured capacity at which you want all trigger queues to operate. The Integration Server automatically adjusts the refill levels by the same percentage. Click Save Changes.

Notes: The Queue Capacity Throttle setting is maintained across server restarts and package reloads. If the percentage by which you reduce capacity does not resolve to a whole number, the Integration Server rounds up or rounds down to the nearest whole number. However, if rounding down would reduce the value to 0, the Integration Server rounds up to 1. For example, if you set the Queue Capacity Throttle to 10% of maximum, a trigger queue with a capacity of 15 and refill level of 4 will have an adjusted capacity of 2 and an adjusted refill level of 1 (Integration Server rounds the calculated adjusted capacity of 1.5 up to 2 and rounds the calculated adjusted refill level of 0.4 up to 1). When you reduce the Queue Capacity Throttle and save your changes, the Integration Server does not immediately reduce the number of documents in a trigger queue. Instead, the Integration Server continues to process documents in the trigger queue until it reaches the adjusted refill level. Then, the Integration Server retrieves enough documents to fill the trigger queue to the adjusted capacity. For example, if you set Queue Capacity Throttle to 50%, a trigger queue with a capacity of 8 and a refill level of 2 will have an adjusted capacity of 4 and an adjusted refill level of 1. The Integration Server processes documents in the trigger queue until it reaches the adjusted refill level of only 1 document. Then, the Integration Server retrieves up to 3 documents to increase the number of documents in the queue to 4 (the adjusted capacity). If you reduce the capacity to a low percentage for an extended period of time, the document might expire on the Broker. For each publishable document type, you can specify a Time to live property. This property specifies how long a document can remain on the Broker before the Broker discards it. For more information about publishable document types, see Publish-Subscribe Developers Guide. If you use the Queue Capacity Throttle as part of your capacity planning process and you determine that the configured values for trigger capacity and refill level need to change, you can use webMethods Developer to set the new capacity and refill level values for each trigger.

ISDEV-DS-61SP2-20060623

12

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Suspending and Resuming Document Retrieval

You can reduce the amount of server resources that document retrieval consumes by suspending document retrieval for one or more triggers. Using the Integration Server Administrator, you can: Suspend or resume document retrieval for all triggers. Suspend or resume document retrieval for specific triggers. The following sections provide more information about these options.

Suspending and Resuming Document Retrieval for all Triggers

When you suspend document retrieval for all the triggers on an Integration Server, the Integration Server stops retrieving documents from the Broker. Server resources, such as threads and memory, that would have been used for document retrieval are available for other tasks. Suspending document retrieval globally (for all triggers) is a quick way of freeing up server resources. This can be especially helpful in a situation in which the Integration Server is functioning under heavy load and additional resources are needed immediately. Suspending or resuming document retrieval can be a temporary or permanent change. (The Integration Server considers a document retrieval change to be permanent if you selected the Apply Change Permanently check box when you made the change.) If the change is temporary, the Integration Server reverts to the permanent document retrieval state when the Integration Server restarts or you reload a package. When you reload a package, the Integration Server reverts only the triggers contained in that package to the permanent document retrieval state. For example, suppose that you temporarily suspend document retrieval for all triggers. If you reload the package OrderProcessing, the Integration Server resumes document retrieval for the triggers in the OrderProcessing package only. Tip! On the Settings > Resources > Trigger Management page under Document Retrieval, the Integration Server Administrator indicates a temporary document retrieval change by displaying an asterisk (*) next to the trigger status in the Active column. You can gradually resume document retrieval by setting the Queue Capacity Throttle to a low percentage, such as 10%, and then resuming document retrieval for all triggers. The Integration Server resumes document retrieval at the adjusted capacity for all triggers. You can also gradually resume document retrieval by selectively resuming individual triggers. For example, you might want to resume document retrieval for those triggers that are part of a critical or high priority process.

ISDEV-DS-61SP2-20060623

13

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

To suspend or resume document retrieval for all triggers

.

1 2 3 4 5

Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management. Under Individual Trigger Controls, in the Active column located under Document Retrieval, click edit all. In the Retrieval State list, do the following: Select... Active Suspended To... Resume document retrieval for all of the triggers on the Integration Server. Suspend document retrieval for all of the triggers on the Integration Server.

If you want the state change to be permanent and maintained after the Integration Server restarts or after a package reload, select the Apply Change Permanently check box. If you do not select this check box, the Integration Server considers the change to be temporary. Click Save Changes.

Notes: Integration Server does not suspend (or resume) document retrieval if the trigger is locked or disabled. Integration Server does not suspend (or resume) document retrieval for triggers that have been excluded from trigger management changes using the watt.server.trigger.managementUI.excludeList. For more information about this property, see Server Configuration Parameters on page 24. Suspending document retrieval affects document retrieval from the Broker only. Triggers will continue to receive locally published documents. Additionally, triggers will continue to receive documents delivered to the default client. When you suspend document retrieval, the Integration Server will not dispatch any server threads to retrieve documents from the Broker. Any server threads currently retrieving documents for the trigger will execute to completion. When you suspend document retrieval, documents to which this trigger subscribes will collect in the triggers client queue on the Broker. Documents remain in the trigger's client queue until document retrieval resumes for the trigger or the documents expire.

ISDEV-DS-61SP2-20060623

14

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

When you resume document retrieval, Integration Server resumes document retrieval for all triggers at the percentage specified by the Queue Capacity Throttle. If you do not resume document retrieval before the server restarts, the trigger package reloads, or the trigger properties are modified, the Broker discards any volatile documents in that trigger's client queue.

Suspending and Resuming Document Retrieval for a Specific Trigger

Sometimes, instead of suspending or resuming document retrieval for all triggers, you might want to suspend or resume document retrieval for specific triggers. Following are some situations in which you might want to suspend or resume document retrieval for specific triggers. When a back-end system needs maintenance or is becoming unresponsive, you might want to suspend document retrieval for triggers that interact with the back-end system. By suspending document retrieval, documents that would normally accumulate on the Integration Server awaiting processing remain on the Broker. This keeps memory and other server resources available for other activities. When the back-end system becomes available, you can resume document retrieval for the associated triggers. After suspending document retrieval for all triggers, you might resume document retrieval for specific triggers. If the server is functioning under an unusually heavy load, you might first suspend retrieval for all triggers and then gradually resume retrieval, starting with the triggers involved in key processes. The following procedure explains how to suspend or resume document retrieval for an individual trigger. To suspend or resume document retrieval for a trigger 1 2 3 4 5 Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management. Under Individual Trigger Controls, locate the row containing the trigger for which you want to suspend document retrieval. In the Active column located under Document Retrieval, click Yes or No. (The Active column displays Yes if document retrieval is active for the trigger; No if document retrieval is suspended. An asterisk (*) appears next to the status if the document retrieval state is temporary.)

ISDEV-DS-61SP2-20060623

15

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

In the Retrieval State list, do the following: Select... Active Suspended To... Resume document retrieval for this trigger. Suspend document retrieval for this trigger.

If you want the state change to be permanent and maintained after the Integration Server restarts, select the Apply Change Permanently check box. If you do not select this check box, the Integration Server considers the change to be temporary. Click Save Changes.

Notes: Integration Server will not suspend or resume document retrieval for the specified trigger if the trigger is locked by a user. When you resume document retrieval, Integration Server resumes retrieval for the trigger at the percentage specified by the Queue Capacity Throttle. In a flow service, you can suspend or resume document retrieval for individual triggers by invoking the pub.trigger:suspendRetrieval service or the pub.trigger:resumeRetrieval service, respectively. You can filter the list of displayed triggers using the
watt.server.trigger.managementUI.excludeList property. For more information about this property, see Server Configuration Parameters on page 24.

Managing Document Processing

Within the publish-and-subscribe model, document processing is the process of evaluating documents against trigger conditions and executing the appropriate trigger services to act on those documents. Document processing requires a server thread with which to evaluate the document and execute the trigger service. It also requires memory in which to keep a copy of the document during document evaluation and trigger service execution. The Integration Server provides various controls that you can use to manage the rate of and resource demands for document processing. Specifically, you can: Limit the number of server threads used for document processing. Manage the number of server threads that can be used to process documents for a trigger concurrently. Suspend or resume document processing for one or more triggers.

ISDEV-DS-61SP2-20060623

16

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

These controls can be used as part of your capacity planning or used during production. The following sections provide more information about these controls.

Increasing or Decreasing Threads for Document Processing

During production and capacity planning, you can increase or decrease the number of threads that can be used to process documents simultaneously. The number of threads available for document processing helps determine the rate at which the Integration Server processes documents that it receives. By default, the Integration Server can use up to 100% of the server thread pool to process documents (execute triggers). Each time the Integration Server processes a document, the server uses a server thread. For example, if the maximum size of the server thread pool is 80 threads, and the server can use 100% of the server thread pool to execute triggers, then up to 80 triggers can execute at the same time. That is, the Integration Server can process up to 80 documents simultaneously. You can control the number of server threads available for document processing (trigger execution) by specifying the maximum percentage of the server thread pool that can be used to execute documents. The Integration Server uses the percentage to calculate the number of server threads that can be used for trigger execution (document processing). For example, suppose that the maximum size of the server thread pool is 80 threads. If you set 10% as the maximum percentage of document processing threads, then the Integration Server can use up to 8 threads to execute triggers at one time. By reducing the number of server threads used for processing documents, you can make server threads and memory available to perform other tasks, such as executing HTTP requests and retrieving documents. Alternatively, you can increase the number of threads for document processing to allow the server to process more documents simultaneously. This can also allow the server to drain the trigger queues more quickly and request additional documents more frequently. Document processing for serial and concurrent triggers combined cannot exceed the value determined by the maximum document processing threads percentage. If you reduce the percentage of document processing server threads, and concurrent triggers continue to consume the maximum execution threads possible (according to their configured settings), serial triggers must wait longer for server threads to become available. This is especially likely if the Integration Server contains concurrent triggers that execute longrunning services. For more information about setting the number of server threads for document processing, see the webMethods Integration Server Administrators Guide. Tip! If you decrease the percentage of threads that can be used for document processing, consider decreasing the Execution Threads Throttle to prevent concurrent triggers from monopolizing available server threads.

ISDEV-DS-61SP2-20060623

17

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

When to Increase or Decrease Threads for Processing Documents

Your knowledge of your integration solution is the best tool for determining when to adjust thread usage for document processing (trigger execution). For example, suppose that a batch process that occurs at the same time each day results in a spike in document publishing. You might want to increase threads for document processing right before the batch process starts to make server threads available to process documents. Alternatively, if you observe memory constraints or other resource issues, you can decrease the number of threads for document processing. Document processing consumes memory because the Integration Server keeps the document in memory while the server thread evaluates the document and executes the trigger service. You can also determine when to modify the number of threads allowed for document processing by monitoring thread usage. You can do this by viewing the thread usage information displayed on the Server > Statistics page. However, you can also establish a warning threshold that instructs the Integration Server to alert you when the number of available threads drops below a particular level. Specifically, the Integration Server creates a journal log entry stating Available Threads Warning Threshold Usage Exceeded. When the Integration Server writes this journal log entry, you might want to decrease threads for document processing to allow more threads to be used for other functions. For more information about setting an available threads warning threshold, see the webMethods Integration Server Administrators Guide. Another way to determine when to alter the number of server threads allotted for document processing is to monitor the current number of threads that are processing documents for triggers. The Integration Server Administrator displays this value in the Current Threads field located under Document Processing on the Settings > Resources > Trigger Management page. Note: Other ways to control the resources used for document processing are: adjusting execution threads for concurrent triggers and suspending or resuming document processing for triggers. For more information about adjusting trigger queue capacity, see Decreasing Document Processing for Concurrent Triggers on page 18. For more information about suspending (or resuming) document processing, see Suspending and Resuming Document Processing on page 20.

Decreasing Document Processing for Concurrent Triggers

You can reduce the amount of server resources consumed by document processing by decreasing the rate of processing for concurrent triggers. Specifically, you can reduce the maximum number of threads that can process documents for a concurrent trigger at one time. The Integration Server Administrator provides an Execution Threads Throttle that you can use to reduce the execution threads for all concurrent triggers by the same percentage. For example, if you set the Execution Threads Throttle to 50% of maximum, the

ISDEV-DS-61SP2-20060623

18

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Integration Server reduces the maximum execution threads for all concurrent triggers by half. A concurrent trigger with a maximum execution threads value of 6, has an adjusted maximum execution threads value of 3. By decreasing parallel processing for concurrent triggers, you can: Free up server threads and memory to perform other functions, such as answering HTTP requests or retrieving documents. Prevent concurrent triggers from monopolizing the threads allotted for document processing. The number of server threads that the server dispatches to process documents for serial and concurrent triggers cannot exceed the value established by the maximum execution threads percentage. If you reduce the number of threads allowed for document processing, and concurrent triggers continue to consume the maximum execution threads possible (according to their configured settings), serial triggers must wait longer for server threads to become available. This is especially likely if the Integration Server contains concurrent triggers that execute long-running services. To decrease parallel execution threads for concurrent triggers 1 2 3 4 Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management, and then click Edit Global Trigger Controls. Under Document Processing, in the Execution Threads Throttle field, select the percentage of the configured maximum execution threads value at which you want to the server to function. The Integration Server applies this percentage to the maximum execution threads value for all concurrent triggers. Click Save Changes.

Notes:

The Execution Threads Throttle value is maintained across server restarts and package reloads. Serial triggers always process one document at a time. The Execution Threads Throttle property does not affect serial triggers. If the percentage by which you reduce trigger execution threads does not resolve to a whole number, Integration Server rounds up or rounds down to the nearest whole number. However, if rounding down would set the value to 0, the Integration Server rounds up to 1. For example, if you reduce Execution Threads Throttle to 10% of maximum, a concurrent trigger with a maximum execution threads value of 12 would have an adjusted value of 1 (Integration Server rounds 1.2 down to 1). A concurrent trigger with a maximum execution threads value of 4 would have an adjusted value of 1 (Integration Server rounds 0.4 up to 1).

ISDEV-DS-61SP2-20060623

19

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

When you reduce the Execution Threads Throttle and save your changes, the Integration Server does not terminate threads currently executing concurrent triggers to meet the adjusted maximum. Integration Server allows server threads processing documents for concurrent triggers to execute to completion. Integration Server waits until the number of threads executing for a concurrent trigger is less than the adjusted maximum before dispatching another server thread to process a document for that trigger. If you suspend document processing (trigger execution) and do not suspend document retrieval, Integration Server will fill all the trigger queues to capacity. Full trigger queues consume more memory than empty trigger queues. You can also reduce the number of concurrent execution threads for a trigger by reducing the capacity of a trigger queue below the maximum number of concurrent execution threads for the trigger. The maximum number of dispatched threads for a trigger cannot exceed the trigger queues capacity. For more information about reducing trigger queue capacity, see Decreasing the Capacity of Trigger Document Stores on page 11. If you use the Execution Threads Throttle as part of your capacity planning process and you determine that the configured values for Maximum execution threads need to change, you can use webMethods Developer to set the new maximum execution threads values for each concurrent trigger.

Suspending and Resuming Document Processing

You can reduce the amount of server resources that document processing consumes by suspending document processing for one or more triggers. Using the Integration Server Administrator, you can: Suspend or resume document processing for all triggers. Suspend or resume document processing for specific triggers. The following sections provide more information about these options.

Suspending and Resuming Document Processing for all Triggers

When you suspend document processing for all triggers, the Integration Server stops dispatching server threads to process documents stored in trigger queues. Server resources, such as threads and memory, that might have been used for document processing will be available for other tasks. Document processing remains suspended until you specifically resume it. Suspending document processing for all triggers is a quick way to make server resources available. This can be especially helpful in a situation in which the Integration Server is functioning under heavy load and additional resources need to be available immediately.

ISDEV-DS-61SP2-20060623

20

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Suspending or resuming document processing can be a temporary or permanent change. (The Integration Server considers a document processing change to be permanent if you selected the Apply Change Permanently check box when you made the change.) If the change is temporary, the Integration Server reverts to the permanent document processing state when the Integration Server restarts or you reload a package. When you reload a package, the Integration Server reverts only the triggers contained in that package to the permanent document processing state. For example, suppose that you temporarily suspend document processing for all triggers. If you reload the package OrderProcessing, the Integration Server resumes document processing for the triggers in the OrderProcessing package only. Tip! On the Settings > Resources > Trigger Management page under Document Processing, the Integration Server Administrator indicates a temporary document processing change by displaying an asterisk (*) next to the trigger status in the Active column. When you are ready to resume document processing, you might want to resume it gradually. For example, you might set the Execution Threads Throttle to a low percentage, resume document processing for all triggers, and then gradually move the Execution Threads Throttle up to 100%. Alternatively, you might selectively resume individual triggers. For example, you might want to resume document processing for those triggers that are part of a critical or high priority process. To suspend or resume document processing for all triggers
.

1 2 3 4 5

Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management. Under Individual Trigger Controls, in the Active column located under Document Processing, click edit all. In the Processing State list, do the following: Select... Active Suspended To... Resume document processing for all of the triggers on the Integration Server. Suspend document processing for all of the triggers on the Integration Server.

If you want the state change to be permanent and maintained after the Integration Server restarts, select the Apply Change Permanently check box. If you do not select this check box, the Integration Server considers the change to be temporary. Click Save Changes.

ISDEV-DS-61SP2-20060623

21

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Notes: Integration Server will not suspend or resume document processing for a locked or disabled trigger. Integration Server does not suspend (or resume) document processing for triggers that have been excluded from trigger management changes using the watt.server.trigger.managementUI.excludeList. For more information about this property, see Appendix 1, Server Configuration Parameters. Suspending or resuming document processing affects all documents in the triggers queue on the Integration Server, including documents retrieved from the Broker and from local publishing. When you suspend document processing, the Integration Server will not dispatch any more server threads to process documents. Any server threads currently processing documents for triggers will execute to completion. This includes documents that are being retried. When you suspend document processing but do not suspend document retrieval, documents will collect in trigger queues until the trigger queues are at maximum capacity or document processing resumes. If the server restarts before document processing resumes, volatile documents are discarded. When you resume document processing the Integration Server resumes document processing at the percentage specified by the Execution Threads Throttle.

Suspending and Resuming Document Processing for Specific Triggers

Sometimes, instead of suspending or resuming document processing for all triggers, you might want to suspend or resume processing for a specific trigger. Following are examples of situations where you might want to suspend document processing for specific triggers. When a back-end system becomes unresponsive or requires maintenance, you might want to suspend document processing for triggers that interact with that back-end system. If the back-end system is not available because of maintenance or failure, trigger services that interact with the system would probably not execute successfully. Suspending document processing for the associated triggers allows for more effective resource utilization because you keep resources that would have been used for unsuccessful document processing available for other tasks. After suspending document processing for all triggers, you might resume document processing for specific triggers. If the server is operating under a heavy load, you might first suspend all document processing and then gradually resume document processing, starting with the triggers involved in critical processes.

ISDEV-DS-61SP2-20060623

22

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

To suspend or resume document processing for a trigger 1 2 3 4 5 Open the Integration Server Administrator if it is not already open. In the Settings menu of the Navigation panel, click Resources. Click Trigger Management. Under Individual Trigger Controls, locate the row containing the trigger for which you want to suspend document processing. In the Active column located under Document Processing, click Yes or No. (The Active column displays Yes if document processing is active for the trigger; No if document processing is suspended. An asterisk (*) appears next to the status if the document processing state is temporary.) In the Processing State list, do the following: Select... Active Suspended 7 To... Resume document processing for this trigger. Suspend document processing for this trigger.

If you want the state change to be permanent and maintained after the Integration Server restarts, select the Apply Change Permanently check box. If you do not select this check box, the Integration Server considers the change to be temporary. Click Save Changes.

Notes: Integration Server will not suspend or resume document processing for the specified trigger if the trigger is locked by a user. When you resume document processing for a concurrent trigger, the Execution Threads Throttle determines the maximum number of documents that can be processed in parallel. In a flow service, you can suspend or resume document processing for individual triggers by invoking the pub.trigger:suspendProcessing service or the pub.trigger:resumeProcessing service, respectively. You can filter the list of displayed triggers using the
watt.server.trigger.managementUI.excludeList property. For more information about this property, see Server Configuration Parameters on page 24.

ISDEV-DS-61SP2-20060623

23

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

Server Configuration Parameters

Note: The information in this section supplements the contents of Appendix B: Server Configuration Parameters in the webMethods Integration Server Administrators Guide webMethods Integration Server 6.1 SP2 includes the following new server parameters. watt.net.ftpPassiveLocalAddr Specifies the host name or IP address a PORT command should send when an FTP port is running in passive mode. When running in passive mode, the FTP port sends a PORT command to the FTP client. The PORT command specifies the address and port to which the client should connect to create a data connection. If the FTP port is behind a NAT server, however, the address of the host on which the Integration Server runs is not visible to the FTP client. Consequently the PORT command will not contain the information the client needs to connect to the server. Use this property to specify the contents of the PORT command. There is no default value. Note: The passive mode address for an individual FTP port can now be specified on the FTP port set up page (Security > Ports > Edit FTP Port Configuration). Using this method, a different passive mode address can be specified for each FTP port. If an address is specified on the FTP port set up page and in the watt.net.ftpPassiveLocalAddr property, the PORT command uses the value specified in the watt.net.ftpPassiveLocalAddr property. watt.net.primaryListener Specifies the primary port for the Integration Server. You can set the primary port on the Security > Ports > Change Primary Port page. By default, Integration Server designates an HTTP port at 5555 as the primary port. Important! webMethods recommends that you set the primary port via the Security > Ports > Change Primary Port page instead of specifying the primary port using the watt.net.primaryListener property. watt.server.auditSync By default, the Integration Server writes audit data to transient storage before writing it to persistent storage. In some high volume, multi-user environments, this behavior can slow performance. This property specifies how the Integration Server writes audit information. When this value is set to false (the default), the Integration Server writes audit data first to transient storage, then to persistent storage. When this value is set to true, the Integration Server writes audit data directly to persistent storage. watt.server.date.suppressPatternError Specifies how Integration Server responds when no parameters are passed to the pub.date:dateTimeFormat service. When this parameter is set to true, the pub.date:dateTimeFormat service returns a null value for the value parameter if nothing is

ISDEV-DS-61SP2-20060623

24

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

passed in to the service. When this parameter is set to false, Integration Server throws the exception [ISS.0086.9189] Invalid date pattern or input when no values are passed to the service. The default is false. watt.server.dispatcher.join.reaperDelay Specifies the frequency (in milliseconds) with which Integration Server removes expired JoinManager entries from the repository. The cleanup logic executes as a background task. The default is 30 minutes (1800000 milliseconds). watt.server.trigger.managementUI.excludeList Specifies a comma-delimited list of triggers to exclude from the Trigger Management pages in the Integration Server Administrator. The Integration Server also excludes these triggers from trigger management changes that suspend or resume document retrieval or document processing for all triggers. The Integration Server does not exclude these triggers from changes to capacity, refill level, or maximum execution threads that are made using the global trigger controls (Queue Capacity Throttle and Trigger Execution Threads Throttle). You can specify the fully qualified names of all the triggers that you want to exclude. You can also use pattern matching to exclude a group of triggers by specifying the beginning portion of the fully qualified name and following it with an asterisk (*). The Integration Server excludes all triggers that begin with the supplied pattern. For example, if you want to exclude all triggers located in the pub.prt folder, specify:
watt.server.trigger.managementUI.excludeList = pub.prt*

watt.server.trigger.monitoringInterval Specifies the interval, measured in seconds, at which Integration Server executes resource monitoring services. A resource monitoring service is a service that you create to check the availability of resources used by a trigger service. When it suspends a trigger because all retry attempts have failed, Integration Server executes the resource monitoring service to determine if all the resources are available. The default is 60 seconds.

webMethods Access, webMethods Administrator, webMethods Broker, webMethods Dashboard, webMethods Developer, webMethods Fabric, webMethods Glue, webMethods Installer, webMethods Integration Server, webMethods Mainframe, webMethods Manager, webMethods Modeler, webMethods Monitor, webMethods Optimize, webMethods Portal, webMethods Servicenet, webMethods Trading Networks, and webMethods Workflow are trademarks of webMethods, Inc. webMethods and the webMethods logo are registered trademarks of webMethods, Inc. Copyright 2006 by webMethods, Inc. All rights reserved, including the right of reproduction in whole or in part in any form.

ISDEV-DS-61SP2-20060623

25

webMethods Integration Server and webMethods Developer Version 6.1 Service Pack 2

ISDEV-DS-61SP2-20060623

26