Sie sind auf Seite 1von 40
1
1
1
1
1

LOAD & PERFORMANCE TESTING

Contents

ERROR! BOOKMARK NOT DEFINED.

1

STRESS, LOAD, SOAK, SPIKE TESTS LOAD PROFILES

ERROR! BOOKMARK NOT DEFINED.

2

LNP TESTING LIFE CYCLE

ERROR! BOOKMARK NOT DEFINED.

3

EXAMPLES OF LNP TESTING OBJECTIVES

ERROR! BOOKMARK NOT DEFINED.

3.1

RESPONSE TIME

ERROR! BOOKMARK NOT DEFINED.

3.2

RELIABILITY

ERROR! BOOKMARK NOT DEFINED.

3.3

CONFIGURATION SIZING

ERROR! BOOKMARK NOT DEFINED.

3.4

CAPACITY PLANNING

ERROR! BOOKMARK NOT DEFINED.

3.5

REGRESSION

ERROR! BOOKMARK NOT DEFINED.

3.6

BOTTLENECK IDENTIFICATION

ERROR! BOOKMARK NOT DEFINED.

3.7

SCALABILITY

ERROR! BOOKMARK NOT DEFINED.

BPEL PM: INTRODUCTION

ERROR! BOOKMARK NOT DEFINED.

1

BPEL PM PROCESS TYPES

ERROR! BOOKMARK NOT DEFINED.

1.1

BY INTERFACE

ERROR! BOOKMARK NOT DEFINED.

1.2

BY DURABILITY

ERROR! BOOKMARK NOT DEFINED.

2

TERMINOLOGY

ERROR! BOOKMARK NOT DEFINED.

3

DIFFERENCE BETWEEN ASYNC & SYNC BPEL

ERROR! BOOKMARK NOT DEFINED.

4

SOA 11G CONFIGURATION FILES AND LOG FILES LOCATION

ERROR! BOOKMARK NOT DEFINED.

5

DEFAULT DATA SOURCES

ERROR! BOOKMARK NOT DEFINED.

6

PERFORMANCE OBJECTIVE

ERROR! BOOKMARK NOT DEFINED.

7

WHAT TO TUNE

ERROR! BOOKMARK NOT DEFINED.

8

SYSTEM SETUP

ERROR! BOOKMARK NOT DEFINED.

9

IMPORTANT POINTS

ERROR! BOOKMARK NOT DEFINED.

10

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

JVM TUNING

ERROR! BOOKMARK NOT DEFINED.

1

PAUSE TIMES AND THROUGHPUT

ERROR! BOOKMARK NOT DEFINED.

1.1

GARBAGE COLLECTION VS. THROUGHPUT

ERROR! BOOKMARK NOT DEFINED.

1.2

DEFRAGMENTATION PAUSES AND THROUGHPUT

ERROR! BOOKMARK NOT DEFINED.

2

PERFORMANCE AND MEMORY FOOTPRINT

ERROR! BOOKMARK NOT DEFINED.

3

JVM CHOICE

ERROR! BOOKMARK NOT DEFINED.

4

JROCKITS TUNING

ERROR! BOOKMARK NOT DEFINED.

4.1

BOARDING INTO SHIP – STEP 1

ERROR! BOOKMARK NOT DEFINED.

4.1.1

Heap Size

Error! Bookmark not defined.

4.1.2

Garbage Collection

Error! Bookmark not defined.

4.1.3

Nursery Size

Error! Bookmark not defined.

4.1.4

Pause Target

Error! Bookmark not defined.

4.2

LEAVING THE SHORE – STEP 2

ERROR! BOOKMARK NOT DEFINED.

4.2.1

Lazy Unlocking

Error! Bookmark not defined.

4.2.2

Call Profiling

Error! Bookmark not defined.

4.2.3

Large Pages

Error! Bookmark not defined.

4.3

IN THE HIGH SEAS – STEP 3

ERROR! BOOKMARK NOT DEFINED.

4.3.1

Compaction

Error! Bookmark not defined.

4.3.2

Thread Local Area size

Error! Bookmark not defined.

5

HOTSPOTS TUNING

ERROR! BOOKMARK NOT DEFINED.

5.1

GARBAGE COLLECTION STRATEGIES

ERROR! BOOKMARK NOT DEFINED.

5.1.1

Serial GC Strategy

Error! Bookmark not defined.

5.1.2

Parallel GC Strategy

Error! Bookmark not defined.

5.1.3

Parallel GC strategy

Error! Bookmark not defined.

6

WHERE TO SPECIFY JVM AND ITS PARAMETERS

ERROR! BOOKMARK NOT DEFINED.

7

OUTOFMEMORYERROR

ERROR! BOOKMARK NOT DEFINED.

7.1

JAVA.LANG.OUTOFMEMORYERROR: JAVA HEAP SPACE

ERROR! BOOKMARK NOT DEFINED.

7.2

JAVA.LANG.OUTOFMEMORYERROR: PERMGEN SPACE

ERROR! BOOKMARK NOT DEFINED.

7.3

JAVA.LANG.OUTOFMEMORYERROR: UNABLE TO CREATE NEW NATIVE THREAD

ERROR! BOOKMARK NOT DEFINED.

7.4

JAVA.LANG.OUTOFMEMORYERROR: REQUESTED XXX BYTES FOR CHUNKPOOL::ALLOCATE. OUT OF SWAP SPACE ERROR! BOOKMARK NOT DEFINED.

7.5

ANALYSIS

ERROR! BOOKMARK NOT DEFINED.

8

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

WEBLOGIC SERVER TUNING

15

1

THREAD MANAGEMENT/WORK MANAGER

21

1.1

REQUEST CLASSES

22

1.2

CONSTRAINTS

24

1.3

WORK MANGER TYPES

25

1.3.1

Default Thread Pool

25

1.3.2

SOAWorkManager

26

2

NETWORK I/O

28

2.1

MUXERS

28

2.2

CHUNK PARAMETERS

29

2.3

CONNECTION BACKLOG BUFFERING

30

2.4

CACHED CONNECTIONS

30

3

TUNING DATA SOURCE CONNECTION POOLS

30

3.1

STATEMENT CACHE

31

3.1.1

Usage Restrictions for Statement Cache

32

3.2

CONNECTION TESTING OPTIONS FOR A DATA SOURCE

32

3.2.1

Automatic Testing

33

3.3

ENABLING CONNECTION CREATION RETRIES

34

3.4

ENABLING CONNECTION REQUESTS TO WAIT FOR A CONNECTION

34

3.5

XA TRANSACTION TIMEOUT

35

3.6

AUTOMATICALLY RECOVERING LEAKED CONNECTIONS

35

3.7

LIMITING STATEMENT PROCESSING TIME

36

3.8

PINNED TO THREAD PROPERTY

36

3.9

GRIDLINK

37

3.10

SINGLE CLIENT ACCESS NAME

38

3.11

LOGGING LAST RESOURCE

38

4 DATABASE MODE

39

 

5 REFERENCE

40

BPEL ENGINE TUNING

ERROR! BOOKMARK NOT DEFINED.

1

REAL TIME VIEWING OF PERFORMANCE METRICS

ERROR! BOOKMARK NOT DEFINED.

1.1

FUSION MIDDDLEWARE CONTROL

ERROR! BOOKMARK NOT DEFINED.

1.2

DYNAMIC MEMORY SERVICE

ERROR! BOOKMARK NOT DEFINED.

2

BPEL ENGINE THREADING MODEL

ERROR! BOOKMARK NOT DEFINED.

2.1

DISPATCHER MAXIMUM REQUEST DEPTH

ERROR! BOOKMARK NOT DEFINED.

2.2

INVOKEMESSAGES IN CACHE

ERROR! BOOKMARK NOT DEFINED.

3

TRANSACTION TIMEOUT

ERROR! BOOKMARK NOT DEFINED.

3.1

SYNCMAXWAITTIME

ERROR! BOOKMARK NOT DEFINED.

3.2

BPEL EJBS

ERROR! BOOKMARK NOT DEFINED.

3.3

GLOBAL TRANSACTION TIMEOUT AT WEBLOGIC DOMAIN LEVEL

ERROR! BOOKMARK NOT DEFINED.

3.4

WEBSERVICE READ AND CONNECTION TIMEOUT

ERROR! BOOKMARK NOT DEFINED.

3.5

RELATIONSHIP AMONG TIMEOUT PARAMETERS

ERROR! BOOKMARK NOT DEFINED.

3.6

WLS TIMEOUT PARAMETERS

ERROR! BOOKMARK NOT DEFINED.

3.6.1

Complete Message Timeout

Error! Bookmark not defined.

3.6.2

Idle Connection Timeout:

Error! Bookmark not defined.

3.6.3

Tunneling Client Timeout

Error! Bookmark not defined.

3.6.4

Post Timeout

Error! Bookmark not defined.

3.6.5

Duration

Error! Bookmark not defined.

3.6.6

Tunneling Client Timeout

Error! Bookmark not defined.

4

BPEL PM AUDIT AND LOGGING

ERROR! BOOKMARK NOT DEFINED.

4.1

AUDIT TUNING

ERROR! BOOKMARK NOT DEFINED.

4.1.1

SOA Infrastructure level

Error! Bookmark not defined.

4.1.2

BPEL Engine Level

Error! Bookmark not defined.

4.1.3

AuditStorePolicy

Error! Bookmark not defined.

4.1.4

AuditFlushByteThreshold

Error! Bookmark not defined.

4.1.5

AuditFlushEventThreshold

Error! Bookmark not defined.

4.1.6

AsyncAuditPersisterThreads

Error! Bookmark not defined.

4.1.7

AsyncAuditQueueSize

Error! Bookmark not defined.

4.2

LOGGING TUNING

ERROR! BOOKMARK NOT DEFINED.

5

BPEL ENGINE DATABASE SCHEMA

ERROR! BOOKMARK NOT DEFINED.

5.1

TWO SCHEMAS

ERROR! BOOKMARK NOT DEFINED.

5.2

PURGE

ERROR! BOOKMARK NOT DEFINED.

5.3

PARTITIONING

ERROR! BOOKMARK NOT DEFINED.

5.3.1

Partial Partitioning

Error! Bookmark not defined.

5.3.2

Complete Partitioning

Error! Bookmark not defined.

6

MDS SCHEMA TUNING

ERROR! BOOKMARK NOT DEFINED.

6.1

PURGING

ERROR! BOOKMARK NOT DEFINED.

6.2

STATISTIC COLLECTION

ERROR! BOOKMARK NOT DEFINED.

6.3

OPTIMIZING PATH_FULLNAME TABLE

ERROR! BOOKMARK NOT DEFINED.

7

TECHNOLOGY ADAPTERS

ERROR! BOOKMARK NOT DEFINED.

7.1

FILE AND FTP ADAPTER

ERROR! BOOKMARK NOT DEFINED.

7.1.1

Threads

Error! Bookmark not defined.

7.1.2

Inbound Throttling

Error! Bookmark not defined.

7.1.3

Outbound Throttling

Error! Bookmark not defined.

7.1.4

Outbound Performance Best Practices

Error! Bookmark not defined.

7.1.5

Tuning number of files to be processed at a time

Error! Bookmark not defined.

7.2

DATABASE ADAPTER

ERROR! BOOKMARK NOT DEFINED.

7.2.1

Basic considerations

Error! Bookmark not defined.

7.2.2

Existence Checking

Error! Bookmark not defined.

7.2.3

Throttling

Error! Bookmark not defined.

7.2.4

Adapter Threads

Error! Bookmark not defined.

7.3

JMS ADAPTER

ERROR! BOOKMARK NOT DEFINED.

7.3.1

Receive Threads

Error! Bookmark not defined.

7.4

AQ ADAPTER

ERROR! BOOKMARK NOT DEFINED.

7.4.1

Dequeue Threads

Error! Bookmark not defined.

7.4.2

Dequeue Threads and Number of database connection

Error! Bookmark not defined.

7.5

MQ ADAPTER

ERROR! BOOKMARK NOT DEFINED.

7.5.1

Dequeue Threads

Error! Bookmark not defined.

7.6

SOCKET ADAPTER

ERROR! BOOKMARK NOT DEFINED.

8

BPEL PM – MISCELLANEOUS PROPERTIES

ERROR! BOOKMARK NOT DEFINED.

8.1

INSTANCE KEY BLOCK SIZE

ERROR! BOOKMARK NOT DEFINED.

8.2

AUTOMATIC RECOVERY ATTEMPTS

ERROR! BOOKMARK NOT DEFINED.

8.3

DISABLE BPEL MONITORS AND SENSORS

ERROR! BOOKMARK NOT DEFINED.

9

BEST PRACTICES: BPEL PROCESS/COMPOSITE/COMPONENT

ERROR! BOOKMARK NOT DEFINED.

9.1

LARGE PAYLOADS

ERROR! BOOKMARK NOT DEFINED.

9.2

PERSIST ONLY THE FAULTED BPEL INSTANCES

ERROR! BOOKMARK NOT DEFINED.

9.3

DO NOT VALIDATE INCOMING XML

ERROR! BOOKMARK NOT DEFINED.

9.4

IDEMPOTENENCY

ERROR! BOOKMARK NOT DEFINED.

9.5

NONBLOCKINGINVOKE

ERROR! BOOKMARK NOT DEFINED.

10

DATABASE TUNING

ERROR! BOOKMARK NOT DEFINED.

10.1

INITIALIZATION PARAMETERS

ERROR! BOOKMARK NOT DEFINED.

10.2

HUGE PAGES

ERROR! BOOKMARK NOT DEFINED.

10.3

DB CONTENTION

ERROR! BOOKMARK NOT DEFINED.

10.3.1

Wait events

Error! Bookmark not defined.

10.4

SECUREFILES

ERROR! BOOKMARK NOT DEFINED.

10.4.1

Enable SecureFiles for SOAINFRA schema

Error! Bookmark not defined.

10.4.2

Enabling Encryption

Error! Bookmark not defined.

10.5

SQL PLAN

ERROR! BOOKMARK NOT DEFINED.

11

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

EM FUSION MIDDLEWARE CONTROL AND WLS ADMIN CONSOLE

ERROR! BOOKMARK NOT DEFINED.

1 STUCK THREAD MAX TIME

2 STUCK THREAD TIMER INTERVAL

3 TUNE DUMP CONFIGURATION

4 DATA DISPLAY OPTION FOR EM CONSOLE

5 CACHE TIMEOUT FOR DISCOVERY

6 PERMSIZE OF ADMIN SERVER

7 DMS’S FREQUENCY

8 UN-TARGET DMS FROM SERVERS

9 BPEL RECOVERY CONSOLE OPTION

10 INDEX CREATION

11 DB MANUAL STATS COLLECTION

12 REFERENCE

DATA COLLECTION

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

1

BEFORE LNP TEST

ERROR! BOOKMARK NOT DEFINED.

1.1

INFRASTRUCTURE

ERROR! BOOKMARK NOT DEFINED.

1.2

JVM-HOTSPOT-<<MANAGEDSERVER>> AND JVM-JROCKIT-<<MANAGEDSERVER>>

ERROR!

BOOKMARK NOT

DEFINED.

1.3

WLS-THREADING-<<MANAGEDSERVER>>

ERROR! BOOKMARK NOT DEFINED.

1.4

WLS-NETWORKIO-<<MANAGEDSERVER>>

ERROR! BOOKMARK NOT DEFINED.

1.5

WLS-DATASOURCE-<<MANAGEDSERVER>>

ERROR! BOOKMARK NOT DEFINED.

1.6

BPEL PM-<<MANAGEDSERVER>>

ERROR! BOOKMARK NOT DEFINED.

1.7

TECHNOLOGY ADAPTERS

ERROR! BOOKMARK NOT DEFINED.

1.8

BPEL PM – COMPOSITE

ERROR! BOOKMARK NOT DEFINED.

1.9

DATABASE

ERROR! BOOKMARK NOT DEFINED.

2

DURING LNP TEST

ERROR! BOOKMARK NOT DEFINED.

2.1

CPU AND RAM USAGE AND MEMORY AVAILABILITY

ERROR! BOOKMARK NOT DEFINED.

2.2

JVM’S HEAP USAGE, GARBAGE COLLECTION AND ANY MEMORY LEAK IF ANY

ERROR! BOOKMARK NOT DEFINED.

2.3

DATABASE CONNECTIONS AVAILABILITY AND USAGE

ERROR! BOOKMARK NOT DEFINED.

2.4

AVAILABILITY OF PERIPHERAL RESOURCES LIKE JMS QUEUES, AQ AND THEIR USAGE PATTERN

ERROR! BOOKMARK

NOT DEFINED.

 

3

AFTER LNP TEST

ERROR! BOOKMARK NOT DEFINED.

3.1

GET FULL DUMP OF FEW IMPORTANT TABLES

ERROR! BOOKMARK NOT DEFINED.

3.2

EXECUTE SQL QUERIES AS AND WHEN REQUIRED

ERROR! BOOKMARK NOT DEFINED.

3.3

ANALYSIS OF AWR REPORT

ERROR! BOOKMARK NOT DEFINED.

3.4

ANALYSIS OF SOA SERVER MEMORY USAGE

ERROR! BOOKMARK NOT DEFINED.

4

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

LNP TEST CASE DESIGN AND LOAD TEST SELECTION

ERROR! BOOKMARK NOT DEFINED.

1 LNP TEST CASE DESIGN

2 DESIGN OF EXPERIMENT

3 REFERENCE

APPENDIX A: WEBLOGIC SERVER OVERVIEW

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

1 WLS CONFIGURATION

2 DOMAIN

3 SERVER

4 ADMINISTRATION SERVER

5 MANAGED SERVER

6 ADMINISTRATION SERVER TO MANAGED SERVER INTERACTION

7 CLUSTER

8 NODE MANAGER

9 MACHINE

APPENDIX B: AUDITING IN BPEL PM

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

1 AUDIT LEVELS

2 ORDER OF PRECEDENCE FOR AUDIT LEVEL SETTINGS

APPENDIX C: ANTI PATTERNS

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

1 SYNCHRONOUS – ASYNCHRONOUS

2 OVER USE OF ASYNCHRONOUS PROCESSES

3 OVER USE OF DURABLE PROCESSES

4 NO FAULT HANDLING

5 SYNCHRONOUS FAULT HANDLING

6 TO MANY RETRIES

7 CHATTING BPEL PROCESS (CALL BACK)

8 OVER USE OF FLOWN

9 LOOPS AND MORE LOOPS

10 SYNCHRONOUS AND ASYNCHRONOUS PROCESSES ON SAME MANAGED SERVER/CLUSTER

DEFINED.

11 DURABLE AND TRANSIENT PROCESSES ON SAME MANAGED SERVER/CLUSTER

12 STICKY LOAD BALANCER

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

13

NOT KEEPING ASPECT RATIO

ERROR! BOOKMARK NOT DEFINED.

APPENDIX D: SQL QUERIES

ERROR! BOOKMARK NOT DEFINED.

1

EM CONSOLE SQL QUERIES

ERROR! BOOKMARK NOT DEFINED.

1.1

RECOVERY CONSOLE QUERIES

ERROR! BOOKMARK NOT DEFINED.

1.2

RECENT FAULT AND REJECTED MESSAGES QUERY

ERROR! BOOKMARK NOT DEFINED.

1.3

RECENT COMPOSITE INSTANCE QUERY

ERROR! BOOKMARK NOT DEFINED.

1.4

INSTANCE TAB PAGE QUERY

ERROR! BOOKMARK NOT DEFINED.

1.5

INSTANCE TAB PAGE SEARCH QUERY BASED ON NAME VS TITLE QUERY

ERROR! BOOKMARK NOT DEFINED.

1.6

FAULT AND REJECTED MESSAGE TAB PAGE QUERIES

ERROR! BOOKMARK NOT DEFINED.

1.6.1

Parent query

Error! Bookmark not defined.

1.6.2

Child query

Error! Bookmark not defined.

2

MISCELLANEOUS

ERROR! BOOKMARK NOT DEFINED.

2.1

STORED PROCEDURE TO CONVERT BLOB IN STRING

ERROR! BOOKMARK NOT DEFINED.

2.2

QUERY TO FIND PERCENTAGE OF FREE SPACE

ERROR! BOOKMARK NOT DEFINED.

2.3

QUERY TO FIND THE WAIT EVENTS FOR LGWR USING ITS SID

ERROR! BOOKMARK NOT DEFINED.

2.4

QUERY TO MONITOR REDO BUFFER ALLOCATION RETRIES

ERROR! BOOKMARK NOT DEFINED.

2.5

SQL STATEMENT TO RECLAIM SPACE AFTER PURGING

ERROR! BOOKMARK NOT DEFINED.

2.6

QUERY TO FIND OUT TOTAL SESSIONS ON A DATABASE

ERROR! BOOKMARK NOT DEFINED.

2.7

QUERY TO FIND OUT UTILIZATION OF PROCESSES AND SESSIONS IN A DATABASE

ERROR! BOOKMARK NOT DEFINED.

2.8

FIND OUT THE PROCESS INSTANCE FROM A CONVERSATION ID WHEN THERE IS NO INSTANCE NUMBER SHOWING IN

THE LOG FILE (BPEL INSTANCE ID FOR A TIMES OUT ITEM)

2.9 QUERY TO GET AUDIT DETAILS FROM AUDIT_DETAILS TABLE

2.10 QUERY TO GET AUDIT DETAILS FROM AUDIT_TRAIL TABLE

2.11 QUERY TO GET XML MESSAGE WITH THE GIVEN INSTANCE ID

2.12 QUERY TO GET XML MESSAGE WITH A GIVEN INSTANCE NAME

2.13 QUERY TO GET PAYLOAD SIZE OF MESSAGE

2.14 QUERY TO GET EXECUTION TIME OF BPEL INSTANCES

2.15 QUERY TO GET THE EXECUTION TIME OF BPEL INSTANCES AND TO FIND THE PARENT THAT HAS INITIALIZED THE

COMPOSITE

2.16 QUERY TO IDENTIFY ALL THE FAULTS FOR THE MESSAGES THAT WERE SITTING IN BPEL ENGINE LEVEL RECOVERY AS

UNDELIVERED INVOKES

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

APPENDIX E: BIG OR LARGE OR HUGE PAGES

ERROR! BOOKMARK NOT DEFINED.

1 LINUX

2 WINDOWS

3 SOLARIS

4 REFERENCE:

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

APPENDIX F: ORA-01438: VALUE LARGER THAN SPECIFIED PRECISION ALLOWED

ERROR! BOOKMARK

NOT DEFINED.

5 WHAT IS THE ERROR IN LOGS?

6 EFFECTS

7 CAUSE

8 SOLUTION

APPENDIX G: LOBS IN THE SOAINFRA SCHEMA

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

APPENDIX H: AWR, ADDM, & ASH REPORTS

ERROR! BOOKMARK NOT DEFINED.

1

AWR REPORT

ERROR! BOOKMARK NOT DEFINED.

2

ADDM REPORT

ERROR! BOOKMARK NOT DEFINED.

3

ASH REPORT

ERROR! BOOKMARK NOT DEFINED.

4

AWR REPORT ANALYSIS

ERROR! BOOKMARK NOT DEFINED.

4.1

SQL STATEMENTS ORDERED BY ELAPSED TIME

ERROR! BOOKMARK NOT DEFINED.

4.2

SQL STATEMENTS ORDERED BY CPU TIME

ERROR! BOOKMARK NOT DEFINED.

4.3

SQL STATEMENTS ORDERED BY GETS

ERROR! BOOKMARK NOT DEFINED.

4.4

SQL STATEMENTS ORDERED BY READS

ERROR! BOOKMARK NOT DEFINED.

4.5

SQL STATEMENTS ORDERED BY EXECUTIONS

ERROR! BOOKMARK NOT DEFINED.

4.6

SQL STATEMENTS ORDERED BY PARSE CALLS

ERROR! BOOKMARK NOT DEFINED.

5

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

APPENDIX I: MONITORING SCRIPTS

ERROR! BOOKMARK NOT DEFINED.

1 DATABASE MONITORING

2 JMS MONITORING

3 AQ MONITORING

APPENDIX J: HOW TO MONITOR SOA SERVER MEMORY USAGE

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR! BOOKMARK NOT DEFINED.

1 SETUP: JCONSOLE OR VISUALVM (INSTALLED LOCALLY)

2 SETUP: JVISUALVM (INSTALLED AT REMOTE MACHINE)

3 SETUP: JROCKIT MISSION CONTROL (INSTALLED AT REMOTE MACHINE)

4 REFERENCE

APPENDIX K: HEAP DUMP FILES ANALYSIS: JROCKIT AND HOTSPOT JVMS

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

ERROR!

BOOKMARK NOT

DEFINED.

1 EXAMPLE ANALYSIS OF A HEAP DUMP FILE USING ECLIPSE MEMORY ANALYZER

2 REFERENCE

ERROR! BOOKMARK NOT DEFINED. ERROR! BOOKMARK NOT DEFINED.

APPENDIX L: CAPACITY PLANNING

ERROR! BOOKMARK NOT DEFINED.

1

CAPACITY PLANNING FOR BPEL PM

ERROR! BOOKMARK NOT DEFINED.

1.1

DETERMINING PERFORMANCE GOALS AND OBJECTIVES – CURRENT & FUTURE

ERROR! BOOKMARK NOT DEFINED.

1.2

MEASURING PERFORMANCE METRICS

ERROR! BOOKMARK NOT DEFINED.

1.3

IDENTIFYING BOTTLENECKS

ERROR! BOOKMARK NOT DEFINED.

1.4

IMPLEMENTING A CAPACITY MANAGEMENT PLAN

ERROR! BOOKMARK NOT DEFINED.

2

REFERENCE

ERROR! BOOKMARK NOT DEFINED.

Exhibits

Exhibit 1: Stress, Load, Soak, Spike Tests Load Profiles Exhibit 2: LnP Testing Life Cycle and SDLC Exhibit 3: BPEL Processes Exhibit 4: JVM Tuning Tradeoffs Exhibit 5: JVM's Garbage Collection and Throughput Exhibit 6: jRockit Attributes Exhibit 7: HostSpot Attributes

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

Exhibit 8: WLS internal applications at startup

- 1

16

Exhibit 9: WLS internal applications at startup

- 2

16

Exhibit 10: WLS internal applications at startup

- 3

17

Exhibit 11: WLS internal applications at startup - 4

18

Exhibit 12: WLS internal applications at startup - 5

19

Exhibit 13: WLS internal applications at startup - 6

20

Exhibit 14: SOAWorkManager - 1

27

Exhibit 15: SOAWorkManager - 2

27

Exhibit 16: SOAWorkManager - 3

27

Exhibit 17: WLS Muxers

29

Exhibit 18: FMC Login Exhibit 19: FMC Farm Summary Exhibit 20: FMC Performance Summary - 1 Exhibit 21: FMC Performance Summary - 2 Exhibit 22: FMC Performance Summary - 3 Exhibit 23: Dynamic Memory Service Exhibit 24: BPEL Engine Threading Model Exhibit 25: Dispatcher Maximum Request Depth - 1 Exhibit 26: Dispatcher Maximum Request Depth - 2 Exhibit 27: InvokeMessages In Cache Exhibit 28: syncMaxWaitTime - 1 Exhibit 29: syncMaxWaitTime - 2 Exhibit 30: syncMaxWaitTime - 3 Exhibit 31: syncMaxWaitTime – 4 Exhibit 32: BPEL EJBs - 1

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

Exhibit 33: BPEL EJBs - 2 Exhibit 34: BPEL EJBs - 3 Exhibit 35: Global transaction timeout at WLS Domain Level - 1 Exhibit 36: Global transaction timeout at WLS Domain Level - 2 Exhibit 37: WLS Timeout parameters - 1 Exhibit 38: WLS Timeout parameters - 2 Exhibit 39: WLS Timeout parameters - 3 Exhibit 40: Tunneling Client Timeout - 1 Exhibit 41: Tunneling Client Timeout - 2 Exhibit 42: Tunneling Client Timeout - 3 Exhibit 43: SOA Infrastructure level audit tuning - 1 Exhibit 44: SOA Infrastructure level audit tuning - 2 Exhibit 45: AuditConfig - 1 Exhibit 46: AuditConfig - 2 Exhibit 47: AuditConfig - 3 Exhibit 48: AuditConfig - 4 Exhibit 49: AuditConfig - 5 Exhibit 50: Payload Validation - 1 Exhibit 51: Payload Validation - 2 Exhibit 52: AuditStorePolicy - 1 Exhibit 53: AuditStorePolicy - 2 Exhibit 54: Logging Tuning - 1 Exhibit 55: Logging Tuning - 2 Exhibit 56: Logging Tuning - 3 Exhibit 57: Logging Tuning - 4 Exhibit 58: MDS Purging - 1 Exhibit 59: MDS Purging - 2 Exhibit 60: Stuck Thread Timer Interval - 1 Exhibit 61: : Stuck Thread Timer Interval - 2 Exhibit 62: : Stuck Thread Timer Interval - 3 Exhibit 63: Data Display Option for EM Console - 1 Exhibit 64: : Data Display Option for EM Console - 2 Exhibit 65: : Data Display Option for EM Console - 3 Exhibit 66: Un-Target DMS from servers - 1 Exhibit 67: Un-Target DMS from servers - 2 Exhibit 68: : Un-Target DMS from servers - 3 Exhibit 69: Un-Target DMS from servers - 4 Exhibit 70: Un-Target DMS from servers - 5 Exhibit 71: BPEL recovery console option - 1 Exhibit 72: BPEL recovery console option - 2

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

Exhibit 73: vmstat Exhibit 74: free Exhibit 75: pmap Exhibit 76: top Exhibit 77: sar-B Exhibit 78: meminfo Exhibit 79: mpstat Exhibit 80: mpstat 2 4 Exhibit 81: df-m Exhibit 82: WebLogic Homes Exhibit 83: Synchronous – Asynchronous - 1 Exhibit 84: Synchronous – Asynchronous - 2 Exhibit 85: Database Monitoring Exhibit 86: JMS Monitoring Exhibit 87: AQ Monitoring Exhibit 88: Setup jConsole or visualVM (installed locally) - 1 Exhibit 89: Setup jConsole or visualVM (installed locally) - 2 Exhibit 90: Setup jConsole or visualVM (installed locally) - 3 Exhibit 91: Setup jConsole or visualVM (installed locally) - 4 Exhibit 92: Heap Dump Files analysis - 1 Exhibit 93: Heap Dump Files analysis - 2 Exhibit 94: Heap Dump Files analysis - 3 Exhibit 95: Heap Dump Files analysis - 4 Exhibit 96: Heap Dump Files analysis - 5 Exhibit 97: Heap Dump Files analysis - 6

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

Snippets

Snippet 1: Sample Work Manager - 1

23

Snippet 2: Sample Work Manager - 2

25

Snippet 3: Sample config.xml

26

Snippet 4: Sample startWeblogic.sh

26

Snippet 5: Using setNull In a Prepared Statement

32

Snippet 6: Sample thread waiting for DB Call to finish

39

Snippet 7: composite.xml Snippet 8: Drop Histrogram Snippet 9: Set Table Preferences Snippet 10: Single Threaded Model - jca file Snippet 11: Partitioned Threaded Model - jca file Snippet 12: Tuning number of files to be processed at a time Snippet 13: JMS Adapter - Receive Thread - composite.xml Snippet 14: AQ Adapter - Dequeue Threads - composite.xml Snippet 15: common properties schema - 1 Snippet 16: common properties schema - 2 Snippet 17: bpel.xml Snippet 18: composite.xml Snippet 19: SecureFiles Snippet 20: BasicFiles to SecureFiles

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

Tables

Table 1: AuditLevel Table 2: BPEL Engine - AuditLevel Table 3: AuditStorePolicy Table 4: SOA Schemas Table 5: SOAINFRA Tables Table 6: MDS Schema tables Table 7: CUBE_INSTANCE Table 8: CUBE_SCOPE Table 9: COMPOSITE_INSTANCE Table 10: COMPOSITE_INSTANCE_FAULT Table 11: AUDIT_TRAIL Table 12: AUDIT_DETAILS Table 13: DLV_MESSAGE Table 14: DLV_SUBSCRIPTION Table 15: DOCUMENT_CI_REF Table 16: DOCUMENT_DLV_MSG_REF Table 17: WORK_ITEM Table 18: XML_DOCUMENT Table 19: XML_DOCUMENT_REF Table 20: HEADERS_PROPERTIES Table 21: CLUSTER_MASTER Table 22: CLUSTER_NODE Table 23: Partial Partitioning Table 24: Complete Partitioning Table 25: MDS_NAMESAPCES Table 26: MDS_PARTITIONS Table 27: MDS_PATHS Table 28: File & FTP Adapter - Inbound Throttling Table 29: : File & FTP Adapter - Outbound Throttling Table 30: : File & FTP Adapter - Outbound Best Practices Table 31: completionPersistPolicy property Table 32: validateSchema value Table 33: Idempotency value Table 34: nonBlockingInvoke value Table 35: Database initialization parameters Table 36: Contention tables Table 37: Tables under potential of index contention

Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined. Error! Bookmark not defined.

WebLogic Server Tuning

WebLogic Server (WLS) is the default option to host BPEL engine in it. This chapter is discussing tuning of WLS form BPEL engine perspective.

The startup script for a managed server, can be located at

<<MW_HOME>>\<<user_projects>>\domain\<<domain-name>>\startWebLogic.sh

OR

<<MW_HOME>>\<<user_projects>>\domain\<<domain-name>>\bin\startWebLogic.sh

Where MW_HOME is the Middleware Home directory containing the Oracle product installation, and domain-name is the name of the domain directory defined by the selected configuration template.

In startWebLogic.sh file one can set JVM parameters, Java version and few more parameters pertaining to WLS.

While starting a managed server (in which BPEL engine is deployed) make sure that it is in production mode.

WLS deploys many internal applications during startup. Many of these internal applications are not needed by BPEL Engine. WLS can be configured to wait and deploy these applications on the first access (on-demand) instead of always deploying them during server startup. This can conserve memory and CPU time during deployment as well as improving startup time and decreasing the base memory footprint for the server.

For a development domain, the default is for WLS to deploy internal applications (console, uddi, wlstestclient, and uddiexplorer) on-demand. For a production-mode domain, the default is for WLS to deploy internal applications as part of server startup.

Log on to the WLS console

Exhibit 1: WLS internal applications at startup - 1 • Click on one of the

Exhibit 1: WLS internal applications at startup

- 1

Click on one of the domain, e.g. myManagedDomain, in the left navigation bar in the WLS console.

in the left navigation bar in the WLS console. Exhibit 2: WLS internal applications at startup

Exhibit 2: WLS internal applications at startup

- 2

Open the Configuration tab and then the General sub-tab.

Exhibit 3: WLS internal applications at startup • Click on Lock & Edit button -

Exhibit 3: WLS internal applications at startup

Click on Lock & Edit button

- 3

Exhibit 4: WLS internal applications at startup - 4 • Deselect the checkbox labeled Enable

Exhibit 4: WLS internal applications at startup - 4

Deselect the checkbox labeled Enable on-demand deployment of internal applications

Exhibit 5: WLS internal applications at startup - 5 • Click on the Save button.

Exhibit 5: WLS internal applications at startup - 5

Click on the Save button.

Exhibit 6: WLS internal applications at startup - 6 • Click on Release Configuration button

Exhibit 6: WLS internal applications at startup - 6

Click on Release Configuration button

Restart the WLS

If set up is using custom application to manage WLS, set

InternalAppsDeployOnDemandEnabled MBean value to false to enable the on-demand

deployment of internal applications.

WLS can be tuned at various levels:

1.

Thread Management

a. Default Thread Pool

b. SOAWorkManager

2. Network I/O

a. Muxers

b. Chunk Parameters

c. Connection Backlog Buffering

d. Cached Connections

3. Database related

a. Connection Pool

b. Transactions

c. Data Source

Enabling of on-demand deployment of internal applications will conserve memory and CPU cycles during deployment as well as improving startup time and decreasing the base memory footprint for the server.

1 Thread Management/Work Manager

WLS has the concept of Work Manager in its thread management. By default, WLS maintains a single thread pool to perform all types (application specific, system specific, etc.) of processing. One can define rules which govern request execution time limits, and rates at which request are entering and leaving pool.

The common thread pool is managed by default work manager. The common thread pool changes its size automatically to maximize throughput. WLS continuously does statistical analysis on historical data and adjust thread counts accordingly.

Since thread pool size changes dynamically, in most of the cases, one need not to tune Work Manager. But in certain conditions such as widely fluctuating load, running other applications along with BPEL engine on the same managed server, response time target is firm, and a minimum thread constraint needs to be specified to avoid server deadlock, custom work mangers should be set up to manage threads at granular level.

There is difference between the general thread pool and the work managers. The general thread pool represents the number of threads allocated for the server. Work Managers are dedicated to applications.

WLS Administrator can configure a set of thread usage guidelines and associate them with one or more applications, or with particular application components. For example, one can associate one set of scheduling guidelines for one application, and another set of guidelines for other application. At run-time, WLS uses these guidelines to assign pending work and enqueued requests to execution threads.

A work manager has following components:

Fair Share Request Class

Response Time Request Class

Context Request Class

Min Threads Constraint

Max Threads Constraint

Capacity Constraint

1.1 Request Classes

A request class expresses a scheduling guideline that WLS uses to allocate threads to requests. Request classes help ensure that high priority work is scheduled before less important work, even if the high priority work is submitted after the lower priority work. WLS takes into account how long it takes for requests to each module to complete

There are multiple types of request class, each of which expresses a scheduling guideline in different terms. A Work Manager may specify only one request class.

fair-share-request-class—Specifies the average thread-use time required to process requests.

For example, assume that WLS is running two modules. The Work Manager for ModuleA specifies a fair-share-request-class of 80 and the Work Manager for ModuleB

specifies a fair-share-request-class of 20.

During a period of sufficient demand, with a steady stream of requests for each module such that the number requests exceed the number of threads; WLS will allocate 80% and 20% of the thread-usage time to ModuleA and ModuleB respectively.

Note: The value of a fair share request class is specified as a relative value, not a percentage. Therefore, in the above example, if the request classes were defined as 400 and 100, they would still have the same relative values.

response-time-request-class—This type of request class specifies a response time target in milliseconds. Response time targets are not applied to individual requests. Instead, WLS computes a tolerable waiting time for requests with that class by subtracting the observed average thread use time from the response time goal, and schedules requests so that the average wait for requests with the class is proportional to its tolerable waiting time.

For example, given that ModuleA and ModuleB in the previous example, have response time goals of 2000 ms and 5000 ms, respectively, and the actual thread use time for an individual request is less than its response time target. During a period of sufficient demand, with a steady stream of requests for each module such that the number

requests exceed the number of threads, and no “think time” delays between response and request, WLS will schedule requests for ModuleA and ModuleB to keep the average response time in the ratio 2:5. The actual average response times for ModuleA and ModuleB might be higher or lower than the response time goals, but WLS will strive to keep ratio of target 2:5. For example, if the average response time for ModuleA requests is 1,000 ms, then average response time for ModuleB requests will be approximate 2,500 ms.

context-request-class—This type of request class assigns request classes to requests based on context information, such as the current user or the current user’s group.

For example, context-request-class assigns a request class to requests based on the value of the request’s subject and role properties.

A context request class defines request classes in an application’s deployment descriptor based on a user’s context. For example:

<work-manager> <name>responsetime_workmanager</name> <response-time-request-class> <name>my_response_time</name>

<goal-ms>2000</goal-ms>

</response-time-request-class>

</work-manager>

<work-manager>

<name>context_workmanager</name>

<context-request-class>

<name>test_context</name>

<context-case>

<user-name>system</user-name>

<request-class-name>high_fairshare</request-class-name>

</context-case>

<context-case>

<group-name>everyone</group-name>

<request-class-name>low_fairshare</request-class-name>

</context-case>

</context-request-class>

</work-manager>

Snippet 1: Sample Work Manager - 1

Above, as explained the request classes based on fair share and response time by relating the scheduling to other work using the same request class. A mix of fair share and response time request classes is scheduled with a marked bias in favor of response time scheduling.

1.2

Constraints

A constraint defines minimum and maximum numbers of threads allocated to execute requests and the total number of requests that can be queued or executing before WLS begins rejecting requests.

One can define the following types of constraints:

max-threads-constraint constraint limits the number of concurrent threads executing requests from the constrained work set. The default is unlimited. For example, consider a constraint defined with maximum threads of 10 and shared by 3 entry points. The scheduling logic ensures that not more than 10 threads are executing requests from the three entry points combined.

A max-threads-constraint can be defined in terms of the availability of resource that requests depend upon, such as a connection pool.

A max-threads-constraint might, but does not necessarily, prevent a request class

from taking its fair share of threads or meeting its response time goal. Once the constraint is reached the server does not schedule requests of this type until the number of concurrent executions falls below the limit. The server then schedules work based on the fair share or response time target.

min-threads-constraint constraint guarantees a number of threads the server will allocate to affected requests to avoid deadlocks. The default is zero. A min-threads- constraint value of one is useful, for example, for a replication update request, which is called synchronously from a peer.

A min-threads-constraint might not necessarily increase a fair share. This type of

constraint has an effect primarily when the server instance is close to a deadlock condition. In that case, the constraint will cause WLS to schedule a request even if requests in the service class have more requests than its fair share recently.

Capacity constraint causes the server to reject requests only when it has reached its capacity. The default is -1. Note that the capacity includes all requests, queued or executing, from the constrained work set. Work is rejected either when an individual capacity threshold is exceeded or if the global capacity is exceeded. This constraint is independent of the global queue threshold.

In response to stuck threads, one can define a Stuck Thread Work Manager component that can shut down the Work Manager, move the application into admin mode, or mark the server instance as failed.

For instance, the Work Manager defined in Listing 2.2 shuts down the Work Manager when two threads are stuck for longer than 30 seconds.

<work-manager> <name>stuckthread_workmanager</name> <work-manager-shutdown-trigger>

<max-stuck-thread-time>30</max-stuck-thread-time>

<stuck-thread-count>2</stuck-thread-count>

</work-manager-shutdown-trigger>

</work-manager>

1.3 Work Manger Types

Snippet 2: Sample Work Manager - 2

Essentially, there are three types of Work Managers, each one characterized by its scope (domain level, application level, and module level) and how it is defined and used. The three types are:

The default Work Manager

Global Work Managers

Application-scoped Work Managers

One can override the behavior of the default Work Manager by creating and configuring a global Work Manager called default. This allows one to control the default thread-handling behavior of WLS.

The configuration of work manager is done in one of the following configuration files:

config.xmlWork Managers specified in config.xml can be assigned to any application, or application component, in the domain. WLS Administration Console can be used to define a Work Manager.

weblogic-application.xmlWork Managers specified at the application level can be assigned to that application, or any component of that application.

weblogic-ejb-jar.xml or weblogic.xmlWork Managers specified at the

component-level can be assigned to that component.

weblogic.xmlWork Managers specified for a Web Application.

1.3.1 Default Thread Pool

Default Thread pool comes with 5 threads which might not be sufficient for high volume scenario. To increase count one can modify the count using two methods:

1. In config.xml add following lines in server definition

<server>

<name>myMangedServer</name>

<self-tuning-thread-pool-size-min>150</self-tuning-thread-pool-size-

min>

<self-tuning-thread-pool-size-max>200</self-tuning-thread-pool-size-

max>

[

]

</server>

Snippet 3: Sample config.xml

2. Modify startWeblogic.sh to add minimum pool size parameter to pass to JVM

# START WEBLOGIC

echo "starting weblogic with Java version:"

${JAVA_HOME}/bin/java ${JAVA_VM} -version

if [ "${WLS_REDIRECT_LOG}" = "" ] ; then

echo "Starting WLS with line:"

${JAVA_HOME}/bin/java ${JAVA_VM} ${MEM_ARGS} - Dweblogic.threadpool.MinPoolSize=150 -Dweblogic.Name=${SERVER_NAME} - Djava.security.policy=${WL_HOME}/server/lib/weblogic.policy ${JAVA_OPTIONS} ${PROXY_SETTINGS} ${SERVER_CLASS}

else

echo "Redirecting output from WLS window to ${WLS_REDIRECT_LOG}"

${JAVA_HOME}/bin/java ${JAVA_VM} ${MEM_ARGS} - Dweblogic.Name=${SERVER_NAME} - Djava.security.policy=${WL_HOME}/server/lib/weblogic.policy ${JAVA_OPTIONS} ${PROXY_SETTINGS} ${SERVER_CLASS} >"${WLS_REDIRECT_LOG}" 2>&1fi

Snippet 4: Sample startWeblogic.sh

As a good practice, set a higher than default "minimal value" for the thread pool but no maximal value, let WLS allow to sort it out by itself.

1.3.2

SOAWorkManager

With the deployment of BPEL Engine in WLS, a work manager (wm/SOAWorkManager) gets created. This work manager is type of Global Work manager.

Exhibit 7: SOAWorkManager - 1 SOAWorkManager is essentially empty Work Manger targeted to a specific

Exhibit 7: SOAWorkManager - 1

SOAWorkManager is essentially empty Work Manger targeted to a specific cluster (all servers in cluster).

targeted to a specific cluster (all servers in cluster). Exhibit 8: SOAWorkManager - 2 Exhibit 9:

Exhibit 8: SOAWorkManager - 2

targeted to a specific cluster (all servers in cluster). Exhibit 8: SOAWorkManager - 2 Exhibit 9:

Exhibit 9: SOAWorkManager - 3

For the purpose of tuning of BPEL engine, one needs to focus on SOAWorkManager

Worker Manager.

2 Network I/O

2.1

Muxers

WLS uses software modules called muxers to read incoming requests on the server and incoming responses on the client. WLS supports the following muxers:

Java Muxer

Native Muxers

Non-Blocking IO Muxer

A "muxer" is a multiplexer, which is a mechanism for combining several streams of data on to a single channel. WLS will be using these to exchange data with itself, or with other nodes in the cluster. At any given time, a number of those will be "blocked", since they have nothing to do.

WLS selects which muxer implementation is using the following criteria:

If NativeIOEnabled is false and MuxerClass is null, the Java Socket Muxer is used.

If NativeIOEnabled is true and MuxerClass is null, native muxers are used, if available for platform.

If the Muxer Class attribute is set to weblogic.socket.NIOSocketMuxer or -

Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer flag is set, the Non-

Blocking IO Muxer is used.

Java Muxer blocks on reads until there is data to be read from a socket. This behavior does not scale well when there are a large number of sockets and/or when data arrives infrequently at sockets. This is typically not an issue for clients, but it can create a huge bottleneck for a server.

If the Enable Native IO parameter is not selected, the server instance exclusively

uses the Java Muxer. This might be acceptable if there are a small number of clients and the rate at which requests arrive at the server is fairly high.

Exhibit 10: WLS Muxers Under these conditions, the Java muxer performs as well as it

Exhibit 10: WLS Muxers

Under these conditions, the Java muxer performs as well as it eliminates Java Native Interface (JNI) overhead. Unlike native muxers, the number of threads used to read requests is not fixed and is tunable for Java muxers by configuring the Percent Socket Readers parameter setting in the Administration Console.

Ideally, one should configure this parameter so the number of threads roughly equals the number of remote concurrently connected clients up to 50% of the total thread pool size. Each thread waits for a fixed amount of time for data to become available at a socket. If no data arrives, the thread moves to the next socket.

Native muxers use platform-specific native binaries to read data from sockets. Native muxers provide superior scalability because they implement a non-blocking thread model. When a native muxer is used, the server creates a fixed number of threads dedicated to reading incoming requests. One must ensure that to use Native I/O, the native library must be present in the server’s shared library path. This is set up with the default scripts. When the server does not find the native library, it throws an error java.lang.UnsatisfiedLinkError: no muxer in java.library.path and then loads the Java version of the muxer.

WLS also provides a non-blocking IO implementation of Native Muxers which may provide enhanced performance for certain load scenarios.

On several platforms, WLS has performance packs. One of the major components of performance pack is native muxer implementation. Naturally, with performance pack, WLS performance improves dramatically if used with Native Muxers.

2.2 Chunk Parameters

A chunk is a unit of memory that the WLS network layer, both on the client and server side, uses to read data from and write data to sockets. To reduce memory allocation costs, a server instance maintains a pool of these chunks. For applications that handle large amounts of data per request, increasing the value on both the client and server sides can boost performance. The default chunk size is about 4K. Use the following properties to tune the chunk size and the chunk pool size:

weblogic.Chunksize: Sets the size of a chunk (in bytes). The primary situation in which this may need to be increased is if request sizes are large. It should be set to values that are multiples of the network's maximum transfer unit (MTU), after subtracting from the value any Ethernet or TCP header sizes. Set this parameter to the same value on the client and server.

weblogic.utils.io.chunkpoolsize: Sets the maximum size of the chunk pool. The default value is 2048. The value may need to be increased if the server starts to allocate and discard chunks in steady state. To determine if the value needs to be increased, monitor the CPU profile or use a memory/ heap profiler for call stacks invoking the constructor

weblogic.utils.io.Chunk.

weblogic.PartitionSize: Sets the number of pool partitions used (default is 4). The chunk pool can be a source of significant lock contention as each request to access to the pool must be synchronized. Partitioning the thread pool spreads the potential for contention over more than one partition.

2.3 Connection Backlog Buffering

One can tune the number of connection requests that a WLS instance will accept before refusing additional requests. The Accept Backlog parameter specifies how many Transmission Control Protocol (TCP) connections can be buffered in a wait queue. This fixed- size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet.

2.4 Cached Connections

Use the http.keepAliveCache.socketHealthCheckTimeout system property for tuning the

behavior of how a socket connection is returned from the cache when keep-alive is enabled when using HTTP 1.1 protocol. By default, the cache does not check the health condition before returning the cached connection to the client for use. Under some conditions, such as due to an unstable network connection, the system needs to check the connection's health condition before returning it to the client. To enable this behavior (checking the health condition), set

http.keepAliveCache.socketHealthCheckTimeout to a value greater than 0.

3 Tuning Data Source Connection Pools

This section discusses how to tune the connection pool attributes in JDBC data sources in WLS domain to improve application and system performance.

3.1

Statement Cache

Usage of prepared statement or callable statement in an application results in considerable processing overhead on database, and communion between WLS & database server. To minimize the processing costs, WLS can cache prepared and callable statements used in applications.

When an application calls any of the statements stored in the cache, WLS reuses the statement stored in the cache. Reusing prepared and callable statements reduces CPU usage on the database server, improving performance for the current statement.

Each connection in a data source has its own individual cache of prepared and callable statements used on the connection. However, statement cache can be configured per data source. For statement cache configuration there are two algorithms:

Least Recently Used (LRU)

Fixed

LRU: As name suggest, LRU caches least recently used statements used on the collection until the statement cache size is reached. When an application calls Conn.prepareStatement(), WLS checks to see if the statement is stored in the statement cache. If so, WLS returns the cached statement. If the statement is not in the cache, and the cache is full (number of statements in the cache = statement cache size), WLS determines which existing statement in the cache was the least recently used and replaces that statement in the cache with the new statement.

Fixed: As name suggest FIXED algorithm enables WLS to cache first (till statement cache size is reached) statements used on the connection. Statements used after the statement cache size is reached are not cached.

LRU algorithm is preferred because rarely used statements will eventually be replaced in the cache with frequently used statements.

Statement Cache Size

The Statement Cache Size determines the total number of statements to cache for each connection in each instance of the data source.

If Statement cache Size is set to 0 then, statement cache is disabled. The default value is 10.

By caching statements, application performance increases. However, RDBMS may maintain a cursor for each open statement and this applies to statements in cache. Excessive value of statement cache size may exceed the limit of RDBMS.

For example:

Number of managed servers in cluster = 4

Statement Cache Size = 10

Number of connections = 20

Then, maximum number of cursor open due to this connection due to cache = 4*10*20 = 800

One needs to be aware that, there may be multiple connections to a RDBMS from an application and multiple applications might be using same RDBMS.

3.1.1 Usage Restrictions for Statement Cache

Using the statement cache can increase performance, but there are few restrictions.

Calling a cached Statement after a DDL operation

Prepared statements stored in the cache refer to specific database objects at the time the prepared statement is cached. If DDL (Data Definition Language) operations are performed over the database objects referenced in statement cache, statements may fail at the time of running them.

Using setNull In a Prepared Statement

If statement cache has a prepared statement that uses a setNull bind variable, set the variable to the proper data type. If generic data type is used data may be truncated or the statement may fail when it runs with a value other than null.

java.sql.Types.Long salary=null

.

.

.

if (salary == null) setNull(2,int)//This is incorrect

else setLong(2, salary)

Instead, use the following:

if (salary == null) setNull(2,long)//This is correct else setLong(2, salary)

Snippet 5: Using setNull In a Prepared Statement

3.2 Connection Testing Options for a Data Source

To check and maintain a data source health, WLS has two options:

Automatic testing: In this case health check and maintenance responsibility is delegated to WLS.

Manual testing: In this case application code is responsible for health check and maintenance.

3.2.1 Automatic Testing

For automatic testing, one has following knobs to turn:

To use automatic testing feature, test table name must be supplied. If one wants not to use default SQL (select 1 from dual) then desired SQL should also be specified.

Test Frequency: This attribute specifies the number of seconds between tests of unused connections. WLS tests unused connections and if find any faulty connections then replace them with healthy ones. If value is 0 or less, automatic testing is disabled.

Test Reserved Connections: This option enables WLS to test connection before giving to client application.

This option reduces performance but ensures that connection handed over to client is healthy one.

Seconds to Trust an Idle Pool Connection: This option specifies the number of seconds after a connection has been proven to be OK to be trusted by WLS as OK and skip automatic testing.

The duration selected for Seconds to Trust an Idle Pool Connection presents performance optimization opportunity.

Connection Testing Semantics

When WLS tests database connections in a data source, it reserves a connection from the data source, runs a small query on the connection, then returns the connection to the pool in the data source. WLS instance tracks statistics on the pool status, including the amount of time a required to complete a connection test, the number of connections waiting for a connection, and the number of connections being tested. The history of recent test connection behavior is used to calculate the amount of time the server instance waits until a connection test is determined to have failed.

If a thread appears to be taking longer than normal to connect, the server instance may delay testing on other threads until the abnormally long-running test completes. If that thread hangs too long in connection testing (10 seconds by default), a pool may declare a RDBMS connectivity failure, disable itself, and kill all connections, whether unreserved or in application hands.

This is very rare, and is intended to relieve the otherwise interminable hangs that can be caused by network cable disconnects and other problems that can lock any JVM thread which is doing a

call in a socket read that the JVM will be unable to break until the OS TCP limit is hit (typically 10 minutes). If a pool disables itself in this manner, it will periodically try to reconnect to the RDBMS (every 5 seconds by default). Once a new connection can be made, the pool will re- enable itself, and subsequent connection requests will be served as normal (with the pool repopulating itself as load requires).

Note: Even one is using automatic testing and relying on WLS for connections’ health, applications must not assume healthy connection in all cases and handle the exceptions that may arise due to unhealthy/faulty connections.

3.3 Enabling Connection Creation Retries

Connection Creation Retry Frequency option specifies the number of seconds between attempts to establish connection to the RDBMS.

If Connection Creation Retry option is enabled and if the database is unavailable at the time of data source creation/deployment or server startup or initial capacity is increased, WLS attempts to create connections in the pool again after the number of seconds specified, and will continue to attempt to create the connections until it succeeds.

When Connection Creation Retry Frequency is set to zero, connection creation retry is disabled. Default value is 0.

3.4 Enabling Connection Requests to Wait for a Connection

WLS JDBC data sources have two attributes that one can set to enable connection requests to wait for a connection from a data source.

Connection Reserve Timeout: When an application requests a connection from a data source, if all connections in the data source are in use and if the data source has reached to its maximum capacity, the application will get a ConnectionUnavailableSQLException. To avoid this, configure the Connection Reserve Timeout value (in seconds) so that connection requests will wait for a connection to become available. After the expiry of Connection Reserve Timeout and if no connection becomes available, the request will fail and the

application will get a PoolLimitSQLExceptionexception.

To disable Connection Reserve Timeout, set it to -1. To make client application wait indefinitely for connection, set Connection Reserve Timeout to 0. The default value is 10 seconds.

Maximum Value: 2147483647

Maximum Waiting for Connection: A connection request that wait for a connection blocks a thread. If too many connection requests concurrently wait for a connection and block threads, system performance may degrade. To limit number of waiting threads manipulate the Maximum

Waiting for Connection attribute, which limits the number connection requests that can concurrently wait for a connection.

The default value is MAX-INT which means no limits. Value 0 means no request will wait for connection. If the maximum number of requests has reached to value of Maximum Waiting for Connection then SQLException is thrown for new requests.

3.5 XA Transaction Timeout

The number of seconds to set as the transaction branch timeout. If set, this value is passed as the transaction timeout value in the XAResource.setTransactionTimeout() call on the XA resource manager, typically the JDBC driver.

When this value is set to 0, the WLS Transaction Manager passes the global WLS transaction timeout in seconds in the method. If set, this value should be greater than or equal to the global WLS transaction timeout.

Note: Enablement of XaSetTransactionTimeout is must to enable setting the transaction branch timeout

3.6 Automatically Recovering Leaked Connections

A leaked connection is a connection that was not properly returned to the connection pool in the data source. To recover leaked connections, specify a value

for InActiveConnectionTimeout on the JDBC Data Source.

WLS forcibly returns a connection to the data source when there is no activity on a reserved connection for the number of seconds that specified in InActiveConnectionTimeout attribute.

Default value is 0 which means this feature is disabled.

Note: The actual timeout could exceed the configured value. The internal data source maintenance thread runs every 5 seconds. When it reaches the InActiveConnectionTimeout (for example 30 seconds), it checks for inactive connections. To avoid timing out a connection that was reserved just before the current check or just after the previous check, the server gives an inactive connection a "second chance." On the next check, if the connection is still inactive, the server times it out and forcibly returns it to the data source. On average, there could be a delay of 50% more than the configured value.

Automatic recovery of leaked connection must be enabled in any high performance setup.

Minimum Value: 0

Maximum Value: 2147483647

3.7

Limiting Statement Processing Time

Statement Timeout attribute value specifies the number of seconds a SQL query may take to execute. If query execution consume more than the time specified, JDBC driver will throw an exception.

The default value for Statement Timeout is -1 which means SQL query will not time out due to longer execution duration.

Setting up value of Statement Timeout in high volume transaction scenario is performance booster at the cost of increased risk of failure of SQL queries.

Maximum value: 2147483647

Changes take effect after restart of managed server.

3.8 Pinned To Thread Property

When Pinned To Thread is enabled ( by setting it to true), WLS pins a database connection from the data source to an execution thread the first time an application uses the thread to reserve a connection. When the application finishes using the connection and calls connection.close(), which otherwise returns the connection to the data source, WLS keeps the connection with the execute thread and does not return it to the data source. When an application subsequently requests a connection using the same execute thread, WLS provides the connection already reserved by the thread. There is no locking contention on the data source that occurs when multiple threads attempt to reserve a connection at the same time and there is no contention for threads that attempt to reserve the same connection from a limited number of database connections.

Note: Pinned To Thread property has no effect with multi data sources, Oracle RAC, and IdentityPool.

Since the nature of connection pooling behavior is changed when Pinned-To-Thread is enabled, some connection pool attributes or features behave differently or are disabled to suit the behavior change:

Maximum Capacity is ignored. The number of connections in a connection pool equals the greater of either the initial capacity or the number of connections reserved from the connection pool.

Shrinking does not apply to connection pools because connections are never returned to the connection pool. Effectively, they are always reserved.

When a connection pool is Reset, the reset connections from the connection pool are marked as Test Needed. The next time each connection is reserved, WLS tests the connection and recreates it if necessary. Connections are not tested synchronously when one reset the connection pool. This feature requires that Test Connections on Reserve is enabled and a Test Table Name or query is specified.

Additional Database Resource Costs When Pinned To Thread is Enabled

When Pinned To Thread is enabled, the Maximum Capacity attribute specified is ignored. The maximum numbers of connections are calculated differently.

Number of servers in cluster = n

Number of threads to execute a connection = m

Number of connections reserved by each thread = p

Number of maximum connections required = n*m*p

While using Pinned To Thread property, consider large number of connections and open cursors required by RDBMS.

Note: Connections are never returned to the connection pool, so the connection pool can never shrink to reduce the number of connections and associated resources in use. This cost can be reduced by setting a driver parameter onePinnedConnectionOnly. When onePinnedConnectionOnly is set to true, only the first connection requested is pinned to the thread. Any additional connections required by the thread are taken from and returned to the connection pool as needed.

If WLS and RDBMS setup can handle the additional resource requirements, use the Pinned To Thread option to increase performance. If WLS or RDBMS cannot handle the additional resource requirements (throw errors after enabling Pinned To Thread) do not use Pinned To Thread.

3.9

GridLink

GridLink is data source implementation targeted to Oracle RAC cluster. It responds to FAN events to provide Fast Connection Failover (FCF), Runtime Connection Load-Balancing (RCLB), and RAC instance graceful shutdown. XA affinity is supported at the global transaction Id level.

A GridLink data source uses Fast Connection Failover to:

Provide rapid failure detection

Abort and remove invalid connections from the connection pool

Perform graceful shutdown for planned and unplanned Oracle RAC node outages

Adapt to changes in topology, such as adding or removing a node

Distribute runtime work requests to all active Oracle RAC instances, including those rejoining a cluster

GridLink data sources use runtime connection load balancing to distribute connections to Oracle RAC instances based on Oracle FAN events issued by the database. This simplifies data

source configuration and improves performance as the database drives load balancing of connections through the GridLink data source, independent of the database topology. Runtime Connection Load Balancing allows WLS to:

Adjust the distribution of work based on back end node capacities such as CPU, availability, and response time

React to changes in Oracle RAC topology

Manage pooled connections for high performance and scalability