Executive summary............................................................................................................................... 2 Kernel recommendations for EBS ........................................................................................................... 2 HP-UX process management parameters ............................................................................................. 3 HP-UX file system parameters............................................................................................................. 4 HP-UX inter-process communication parameters ................................................................................... 6 Tracking kernel parameters with kcusage and kcalarm............................................................................. 7 Sample shell script ............................................................................................................................... 8 Summary ............................................................................................................................................ 9 For more information.......................................................................................................................... 10
Executive summary This document contains recommendations for kernel parameters on HP-UX servers running Oracle E-Business Suite (EBS) software applications. The EBS software deployments are structured around Oracle Fusion Middleware platform; the installation is divided into a database tier and an applications tier, hosting services such as Java, Oracle Forms and Oracle Reports, along with other business logic components. This paper is intended to give an overview of base kernel parameters that should be set for EBS 11i or R12 systems. A system tuning exercise is always recommended before going into live production. The kernel settings offered in this paper are meant to give a starting point for future system tuning. HP also offers system performance tuning services for EBS systems. For more information on this service or other Oracle-specific services, please contact HP at services.oracleapps@hp.com. Target audience: The target audience for this paper is comprised of system administrators and technical managers for Oracle E-business Suite applications deployments running on HP-UX 11i. Kernel recommendations for EBS E-Business Suite supports all current versions of HP-UX 11i (Version 1 {aka 11.11}, Version 2 {aka 11.23}, and Version 3 {aka 11.31}); therefore, the recommended values selected for the different kernel parameters are appropriate for all versions. The operating system versions are specified by their customer-level designations (HP-UX 11i v1, v2 and v3). An nc annotation for a given kernel parameter implies no changes necessary for the specified operating system version, typically because the default setting is adequate. A useful feature of HP-UX 11i v2 and later is complete documentation for all kernel parameters via the man (1M) interface. For example, the command man maxuprc will display the man page for the maxuprc kernel parameter. The order of the changes listed in this document is important; making the changes in the order given will avoid frustrating input value out of bounds and violates interdependency rule messages from the System Administration Manager (SAM) in HP-UX 11i v1 or kcweb in HP-UX 11i v2 or later. Kernel parameters can also be changed from the command line with kmtune (HP-UX 11i v1) or kctune (HP-UX 11i v2 or later). See the appropriate HP-UX man pages for details. A sample shell script that uses the kctune commands is given at the end of this document. HP-UX 11i supports some dynamic kernel parameters (indicated in the tables with a yes in the Dyn column). The error checking by the SAM process (HP-UX 11i v1 only) may prevent changing those parameters until AFTER the reboot forcing the incorporation of the other changes. A second reboot is not required; what is required is a second invocation of SAM. Note Using a formula for some kernel parameters is a useful method for reducing the number of knobs to turn when tuning a server for a particular workload. The default kernel parameters for HP-UX 11i v2 and v3 have virtually eliminated the use of formulas to allow for greater customization of kernel services. This document restores some formula values to simplify server management for well-characterized EBS deployments. Beginning with RDBMS 9i, Oracle has distributed a utility to perform pre-installation checks on the target server. This utility can be very helpful at identifying appropriate kernel settings. Refer to MetaLink note: 334563.1 for the Oracle 10g version. 2 HP-UX process management parameters The kernel parameters that control HP-UX process management features are shown in Table 1. Table 1. Process management kernel parameters Parameter Dyn Default Value Recommended Value maxuprc 11i v1 : 75 v2/v3 : 256 (NPROC/2) -> 2048 maxusers (11i v1 only) 32 128 nproc (nc for v2/v3) 11i v1 : (20+8*MAXUSERS) v2/v3 : 4200 4096 or greater nkthread
maxssiz 0x00800000 -> 8MB 0x04000000 -> 64 MB maxssiz_64bit (nc for v2/v3) 11i v1 : 0x00800000 -> 8 MB v2/v3 : 0x10000000 -> 256 MB 0x10000000 -> 256 MB maxtsiz 11i v1 : 0x04000000 -> 64 MB v2/v3 : 0x06000000 -> 96 MB 0x40000000 -> 1 GB swapmem_on 1 (but may be 0 for some configurations) 1 (set to 0 only if device swap >= server memory) swchunk 2048 16384 vps_ceiling 16 256
Rationale for recommendations: The changes to the proc and thread parameters above are necessary to allow sufficient kernel resources for a typical production deployment. Servers that support multiple environments, such as a development and a unit test environment may need slightly higher values. Error messages in the Oracle log files should provide insight if the configured parameters are insufficient. For example, error messages noting process table full or fork of child process failed are usually indications that nproc and/or maxuprc need to be increased. 3 The max*siz parameters limit the virtual address space of the processes running on the server. Oracle processes can have fairly large virtual memory footprints, so failure to increase these parameters can result in failure to launch all the Oracle processes. The vps_ceiling parameter enables the kernel to more efficiently map physical memory to virtual address space for those large-footprint applications. Increasing swchunk facilitates adding swap space, if adding becomes necessary. Configuring additional storage as device swap does not require a reboot; modifying swchunk does. Note Beginning with the September 2008 release of 11i v3 (11.31.0809), the HP-UX kernel supports the base_pagesize parameter to modify the smallest block of physical memory that can be allocated to running processes. Changing this parameter from its default value of 4 kilobytes is not recommended for Oracle EBS systems at this time. HP-UX file system parameters This next set of parameters controls how the kernel manages the servers file system resources. Table 2. File system resource management kernel parameters Parameter Dyn Default value Recommended value max_async_ports 50 200 maxfiles_lim (nc for v2/v3) yes 11i v1 : 1024 v2/v3 : 4096 4096 maxfiles (nc for v2/v3) 11i v1 : 60 v2/v3 : 2048 2048 nfile <complex formula> <double NPROC scaling factor> or (4*NPROC + 2048) nflocks 200 (2*NPROC) ninode <complex formula> <double NPROC scaling factor> or (2*NPROC + 1024) vx_ninode 1 0 (8*NPROC) dbc_min_pct 2 (filecache_min) yes 5 2 dbc_max_pct (filecache_max) yes 50 20 (or lower)
1 a patch included with HP-UX 11i v3 update 3 (11.31.0809) eliminates the need to modify vx_ninode 2 dbc_min_pct and dbc_max_pct have been replaced with filecache_min and filecache_max for HP-UX 11.31 4 Rationale for recommendations: The max_async_ports is required only for servers running Oracle RDBMS. The value needs to be set to the expected maximum number of Oracle RDBMS processes (usually the max_processes parameter from the init*.ora configuration file for the database instance). Servers supporting multiple database instances will use the sum of all the max_processes values. The changes to the maxfiles* parameters are necessary because many Oracle RDBMS and Fusion Middleware programs open dozens or even hundreds of files concurrently. The values for dbc_min_pct and dbc_max_pct (renamed filecache_min and filecache_max for HP-UX 11i v3) need to be set so as to restrict the kernels buffer cache to several hundred megabytes, or around 20% of a 4GB server, 10% of an 8GB server, and 2-3% of a 32GB server. Database servers may require larger file system buffer caches if the database instance is maintained on standard file systems and those file systems are configured so as to enable buffered I/O, which is not a configuration recommended by either Oracle or HP. Setting a non-zero value for vx_ninode also has a positive impact on overall file system performance. An explicit value ensures that the kernels Symantec Veritas file system code does not inadvertently consume more resources than necessary for file-system intensive applications. Changes in HP-UX 11i v2 require that the formula for vx_ninode be tied to a parameter with a fixed value (e.g., NPROC), not one that is itself another formula, such as NFILE. A patch to the Veritas code was added to HP-UX 11i v3, update 3 (11.31.0809) to address this issue; it is not necessary to modify vx_ninode for systems using HP-UX 11.31.0809 or later.
5 HP-UX inter-process communication parameters This table specifies the inter-process communication (IPC) control parameters. Both the Oracle RDBMS product and the Fusion Middleware components use these IPC features. Refer to ipcs (1M) for information on how to monitor the kernels allocation of these resources on a running system. Table 3. Inter-process communication control kernel parameters Parameter Dyn Default value Recommended value msgssz (obsolete in v3) 11i v1 : 8 v2 : 96 512 msgmni 11i v1 : 50 v2/v3 : 512 NPROC msgseg (obsolete in v3) 11i v1 : 2048 v2 : 8192 32767 msgtql 11i v1 : 40 v2/v3 : 1024 NPROC msgmnb yes 16384 1048576 msgmax (obsolete in v3) yes 8192 64000 msgmap (obsolete in v3) 11i v1 : (2+msgtql) v2 : 1026 (2+msgtql) semmns (nc for v2/v3) 11i v1 : 128 v2/v3 : 4096 (4*NPROC) semmni (nc for v2/v3) 11i v1 : 64 v2/v3 : 2048 (2*NPROC) semmnu 11i v1 : 30 v2/v3 : 256 (NPROC-4) semume (nc for v2/v3) 11i v1 : 10 v2/v3 : 100 64 shmmax yes 11i v1 : 0x04000000 v2/v3 : 0x40000000 App Tier: 0x40000000 (1GB) DB Tier : 0x400000000 (16GB) shmmni 11i v1 : 200 v2/v3 : 400 512 shmseg (nc for v2/v3) Yes 11i v1 : 120 v2/v3 : 300 256
6 Tracking kernel parameters with kcusage and kcalarm HP-UX 11i, version 2 and later, support implicit monitoring of the kernel resources. There are several monitoring capabilities available: kcusage(1M) allows users to track whether sufficient kernel resources have been allocated kcalarm(1M) allows users to set thresholds for key kernel tunables When the usage of the monitored tunable parameter exceeds the threshold value, notifications are automatically sent. The kcweb (1M) tool is a graphical front end to many of the kernel configuration tasks common on HP-UX servers; kcweb will plot the data provided by kcusage in any browser window. The use of kcweb also simplifies the creation and management of kernel tunable alarms. A complete discussion of the kc* tools is beyond the scope of this document. Some sample commands are listed below, along with their effects. As with many HP-UX tools, simple experimentation is often the best way to learn about the capabilities of kcusage and kcalarm. Table 4. Sample kc* tools commands Tools Commands kcusage d nfile List the hourly average usage of nfile over the last day (how many open files have been managed by the kernel) kcusage m semmni List the daily average of semmni over the last month (how many semaphore identifiers have been allocated by the kernel) kcusage h t maxuprc List the average maxuprc value over the last hour (5 minute data samples) along with the top 5 users kcalarm a t 90 n email:sys@host: nkthread kcalarm Add an alarm to email sys@host if the number of kernel threads active on a system exceeds 90% of the nkthread value set in the kernel.
Note The second invocation of kcalarm simply lists the active alarms.
kcalarm d nkthread Delete all kernel alarms for nkthread tunable
It is important to remember that kernel monitoring can help identify both under-provisioned and over-provisioned tunables. An over-provisioned tunable can be identified by using kcalarm with a relatively low threshold (i.e. 50%). If the alarm is not triggered, even during times of peak system utilization, reducing that tunable should be considered. Proper allocation of kernel resources ensures that more of the systems capacity is available for the EBS deployment. 7 Sample shell script The following shell script is designed to simplify the process of updating kernel parameters on servers running HP-UX 11i v2. It uses the kctune (1M) utility to update the kernel tunables specified by a simple list. An analogous script using the HP-UX 11i v1 kmtune (1M) utility could be used for those servers.
#! / bi n/ sh # # A ver y si mpl e scr i pt t o updat e ker nel par amet er s on an # 11. 23 syst em. The scr i pt uses " kct une - h" t o updat e t he # set of par amet er s i n t he " kpar m_val ues" ar r ay bel ow. # The - h f l ag ensur es t hat al l t he changes ar e r ol l ed i nt o # t he NEXT r eboot ( si nce t her e ar e st i l l some par amet er # i n our l i st t hat r equi r e a r eboot ) . The K f l ag # pr event s t he unnecessar y backup of t he ker nel conf i gur at i on # f or each and ever y change. # # NOTE: t he out put of kct une i s sent t o a l og because each # successi ve cal l wi t h t he - h f l ag shows ALL pendi ng changes. # So t he out put ( by t he t i me we' ve updat ed our 10- 15 par amet er s) , # i s pr et t y conf usi ng. # # NOTE: dbc_m*_pct i s handl ed speci al l y because most of t en, # t hese par amet er s must be r educed. Thi s gi ves us t he chance # t o guar ant ee a backup of t he r unni ng ker nel conf i g. #
t hi s_pr og=$0
i f [ `i d - u` ! = 0 ] ; t hen echo " Must be r oot t o r un $t hi s_pr og" exi t 1 f i
KCLOG=`di r name $0`" / kcset . l og"
dbc_mi n_t ar get =2 dbc_max_t ar get =5
set - A kpar m_val ues \ " maxupr c=( npr oc/ 2) \ max_t hr ead_pr oc>=1024 \ maxdsi z>=0xC0000000 \ maxdsi z_64bi t >=0x1000000000 \ maxssi z>=0x04000000 \ maxssi z_64bi t >=0x10000000 \ maxt si z>=0x40000000 \ swchunk>=16384 \ vps_cei l i ng>=256 \ max_async_por t s>=128 \ maxf i l es_l i m>=4096 \ maxf i l es>=2048 \ npr oc>=4096 \ nf i l e=( 4*npr oc+2048) \ nf l ocks=( 2*npr oc) \ ni node=( 2*npr oc+1024) \ msgssz>=512 \ msgseg>=32767 \ msgmni =npr oc \ msgt ql =npr oc \ msgmnb>=1048576 \ msgmax>=64000 \ msgmap=( msgt ql +2) \ 8 9 semmns=( npr oc*4) \ semmni =( npr oc*2) \ semmnu=( npr oc- 4) \ semume>=64 \ shmmax>=0x400000000 \ shmmni >=512 \ shmseg>=256 \ vx_ni node=( 8*npr oc) "
# # Loop t hr ough al l par amet er s. # As of 11. 23, kct une r et ur ns 0 or 1 f or success # / usr / bi n/ dat e > $KCLOG echo " " >> $KCLOG f or p i n $kpar m_val ues do echo " kct une - K - h $p" | t ee - a $KCLOG kct une - K - h " $p" >> $KCLOG i f [ $? - gt 1 ] ; t hen echo " \ " kct une $p\ " f ai l ed" | t ee a $KCLOG f i done
# # The dbc par amet er s ar e har der because we need t o # r educe t hem. Thi s gi ves us t he chance t o save a backup # of t he r unni ng copy now. # dbc_mi n=`kct une - P cur r ent dbc_mi n_pct | awk ' / cur r ent / {pr i nt $2}' ` dbc_max=`kct une - P cur r ent dbc_max_pct | awk ' / cur r ent / {pr i nt $2}' `
i f [ $dbc_mi n - gt $dbc_mi n_t ar get ] ; t hen dbc_mi n=$dbc_mi n_t ar get echo " kct une - h dbc_mi n_pct =$dbc_mi n" | t ee - a $KCLOG f i
i f [ $dbc_max - gt $dbc_max_t ar get ] ; t hen dbc_max=$dbc_max_t ar get echo " kct une - h dbc_max_pct =$dbc_max" | t ee - a $KCLOG f i
kct une - B - h dbc_mi n_pct =$dbc_mi n dbc_max_pct =$dbc_max >> $KCLOG i f [ $? - gt 1 ] ; t hen echo " kct une f ai l ed t o updat e dbc_[ mi n/ max] _pct par amet er s" f i
echo " " | t ee - a $KCLOG / usr / bi n/ dat e >> $KCLOG echo " kct une changes compl et e. See $KCLOG f or out put . " | t ee - a $KCLOG echo " I f t her e wer e no er r or s di spl ayed as t he changes wer e i ni t i at ed, " echo " you shoul d now r eboot t he syst emso t hat t he par amet er changes" echo " may t ake ef f ect . " echo " "
Summary Implementing the recommended parameters as described in this paper will help avoid many performance pitfalls common in new E-Business Suite configurations. While it is always recommended to engage in a complete system tuning exercise, the recommendations mentioned in this white paper will, in most cases, provide a stable platform upon which to build an E-Business Suite environment. For more information on this service or other Oracle specific services, please contact HP at services.oracleapps@hp.com. For more information For additional information on Oracle Applications and the HP services that are available, please visit our alliance website at www.hporacleapps.com
To help us improve our documents, please provide feedback at www.hp.com/solutions/feedback
2007-2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Java is a US trademark of Sun Microsystems, Inc. . 4AA1-6355ENW, Revision 3, November 2008